#!/usr/bin/env python
# coding: utf-8

# # Advanced search options (Nestle1904LFT)
# 
# NOTE: This notebook requires significant cleaning and rework.

# ## Table of content <a class="anchor" id="TOC"></a>
# * <a href="#bullet1">1 - Introduction</a>
# * <a href="#bullet2">2 - Load Text-Fabric app and data</a>
# * <a href="#bullet3">3 - Performing the queries</a>
#     * <a href="#bullet3x1">3.1 - TBD</a>
#     * <a href="#bullet3x2">3.2 - Inspecting your query</a>
#     * <a href="#bullet3x3">3.2 - Comparing two lists with query results</a>
# * <a href="#bullet4">4 - Discussion</a>
# * <a href="#bullet5">5 - Atribution and footnotes</a>
# * <a href="#bullet6">6 - Required libraries</a>

# # 1 - Introduction <a class="anchor" id="bullet1"></a>
# ##### [Back to TOC](#TOC)
# 
# TBD

# # 2 - Load Text-Fabric app and data <a class="anchor" id="bullet2"></a>
# ##### [Back to TOC](#TOC)

# In[1]:


get_ipython().run_line_magic('load_ext', 'autoreload')
get_ipython().run_line_magic('autoreload', '2')


# In[2]:


# Loading the Text-Fabric code
# Note: it is assumed Text-Fabric is installed in your environment
from tf.fabric import Fabric
from tf.app import use


# In[3]:


# load the N1904 app and data
N1904 = use ("tonyjurg/Nestle1904LFT", version="0.7", hoist=globals())


# In[4]:


# The following will push the Text-Fabric stylesheet to this notebook (to facilitate proper display with notebook viewer)
N1904.dh(N1904.getCss())


# In[5]:


# Set default view in a way to limit noise as much as possible.
N1904.displaySetup(condensed=True, multiFeatures=False,queryFeatures=False)


# # 3 - Performing the queries <a class="anchor" id="bullet3"></a>
# ##### [Back to TOC](#TOC)

# ## 3.1 - TBD<a class="anchor" id="bullet3x1"></a>
# ##### [Back to TOC](#TOC)
# 
# TBD

# ## 3.2 - Inspecting your query<a class="anchor" id="bullet3x2"></a>
# ##### [Back to TOC](#TOC)
# 
# Each query templace can be inspected by use of [`S.study()`](https://annotation.github.io/text-fabric/tf/search/search.html#tf.search.search.Search.study). This is particulary helpfull in case the query is complicated.

# In[6]:


ComplicatedQuery = '''
wg
/where/
   wg phrasefunction=S
/have/
  /without/
    word sp#conj
  /-/
/-/
  wg phrasefunction=O
'''
S.study(ComplicatedQuery)


# In[5]:


S.showPlan()


# In[6]:


for result in S.fetch(limit=10):
    TF.info(S.glean(result))


# ## 3.3 - Comparing two lists with query results<a class="anchor" id="bullet3x3"></a>
# ##### [Back to TOC](#TOC)
# 
# Using python standard functions, it apears to be easy to verify if the result of two queries are the same or different. However, it is good to have a closer look at the matter since there are a few pitfalls that could lead to false results.

# In[7]:


# define query template 1
SomeQuery ='''
phrase phrasefunction=V
    word lemma=λέγω
'''
# now create two lists with identical query results and compare result lists
SomeResult1=N1904.search(SomeQuery)
SomeResult2=N1904.search(SomeQuery)
print(f'Same result? {SomeResult1 == SomeResult2}')


# This is exactly what we would expect. But comparing lists can be tricky. Consider the following two queries.

# In[25]:


# define query template 1
Query1 ='''
phrase
  a:word sp=prep
  b:word sp=adj
  c:word sp=noun 
'''

# define query template 2
Query2 ='''
phrase
  a:word sp=prep
  b:word sp=noun
  c:word sp=adj
'''

# create and compare result lists
ResultQuery1=N1904.search(Query1)
ResultQuery2=N1904.search(Query2)

print(f'Same result? {ResultQuery1 == ResultQuery2}')


# This method of comparing the lists results identifies a difference between the two lists. However, upon closer examination, that may or may not be the case, depending on what is understood as difference. The 'problem' here is that both ResultQuery1 and ResultQuery2 are lists of **ordered tuples**. The swapping of the feature conditions 'sp=adj' and 'sp=noun' did not result in a different set of results; only the presentation of rhe result differed. 

# In[33]:


# create 2 result lists
ResultQuery3=N1904.search(Query1,sort=True)
ResultQuery4=N1904.search(Query1,sort=False)

# compare unsorted lists
print(f'Unsorted lists: Same result ? {ResultQuery3 == ResultQuery4}')

# sort both lists on the first tuple 
SortedResultQuery3 = sorted(ResultQuery3, key=lambda x: x[0])
SortedResultQuery4 = sorted(ResultQuery4, key=lambda x: x[0])

# compare sorted lists
print(f'Sorted lists: Same result ? {SortedResultQuery3 == SortedResultQuery4}')


# Unexpectedly the python list compare still viewed the two lists as different. But why? Python does report that the two lists are different because the comparison of lists (SortedResultQuery3 == SortedResultQuery4) checks for the equality of the list objects, not their contents.
# 
# The search() function in Text-Fabric returns a list of nodes or tuples representing search results. Even if the search criteria and the data are the same, the two lists, ResultQuery3 and ResultQuery4, are distinct list objects. Hence, when comparing them directly using the == operator, Python considers them as different objects, resulting in False.
# 
# To compare the content of the lists, it is advices to first onvert them to sets and compare those sets. See following example: 

# In[35]:


# Convert tuples to sets
set1 = set(tuple(item) for item in ResultQuery3)
set2 = set(tuple(item) for item in ResultQuery4)

# Compare the sets
if set1 == set2:
    print("Lists ResultQuery3 and ResultQuery4 are equal.")
else:
    print("Lists ResultQuery3 and ResultQuery4 are not equal.")


# Now, let's compare ResultQuery1 and ResultQuery2 again by first converting them to sets. 

# In[38]:


# Convert tuples to sets
set1 = set(tuple(item) for item in ResultQuery1)
set2 = set(tuple(item) for item in ResultQuery2)

# Compare the sets
if set1 == set2:
    print("Lists ResultQuery1 and ResultQuery2 are equal.")
else:
    print("Lists ResultQuery1 and ResultQuery2 are not equal.")


# This is indeed the result we expected (see earlier mentioned reasons).

# ## 3.4 - Using search qualifiers<a class="anchor" id="bullet3x4"></a>
# ##### [Back to TOC](#TOC)
# 
# A search template can also use the following:

# In[7]:


ErgetaiQuery = '''
word word=ἔρχεται
/with/
book=Mark chapter=6 verse=1 
/or/
book=Revelation chapter=1 verse=7 
/-/
'''

ErgetaiResult = N1904GBI.search(ErgetaiQuery) 
# returns list of ordered tuples

Maybe discuss:

query = '''
node feature1=A|B
node feature2=X|Y|Z
'''
# # 4 - Discussion<a class="anchor" id="bullet4"></a>
# ##### [Back to TOC](#TOC)
# 
# TBA

# # 5 - Attribution and footnotes<a class="anchor" id="bullet5"></a>
# ##### [Back to TOC](#TOC)
# 
# NA

# # 6 - Required libraries <a class="anchor" id="bullet6"></a>
# ##### [Back to TOC](#TOC)
# 
# The scripts in this notebook require (beside `text-fabric`) the following Python libraries to be installed in the environment:
# 
#     {none}
# 
# You can install any missing library from within Jupyter Notebook using either`pip` or `pip3`.

# In[ ]: