You might want to consider the start of this tutorial.
Short introductions to other TF datasets:
or the
Maybe you know MQL. It is the search template language implemented by Ulrik Sandborg-Petersen in Emdros, and used by SHEBANQ.
TF search templates have been inspired by MQL, but they are different.
This notebook shows examples of real-life MQL queries on SHEBANQ. and expresses them as Text-Fabric search templates.
See also the SHEBANQ tutorial by Bas Meeuse translated to TF.
%load_ext autoreload
%autoreload 2
from tf.app import use
from tf.core.helpers import project
We want to use a fixed data version, not a changing one, for the purposes of this notebook. Also, we pick it from our cloned repo. So, if you want to run this, you need to have cloned the bhsa repo to your computer.
VERSION = "2017"
A = use("ETCBC/bhsa:clone", hoist=globals(), version=VERSION)
This is Text-Fabric 9.2.3 Api reference : https://annotation.github.io/text-fabric/tf/cheatsheet.html 121 features found and 0 ignored
Oliver Glanz: PP with adjective followed by noun
select all objects where
[phrase FOCUS typ = PP
[word sp= prep]
[word sp=adjv]
[word sp=subs]
]
64 results having 251 words.
query = """
phrase typ=PP
word sp=prep
<: word sp=adjv
<: word sp=subs
"""
results = A.search(query)
1.44s 64 results
The number of results is right. The number of words that SHEBANQ reports is the number of words in the phrases of the result. Let us count them:
print(sum([len(L.d(r[0], otype="word")) for r in results]))
251
This means: 220 verses, with 458 result words, and since there are exactly 2 words in each result, we expect 229 results.
queryTight = """
phrase
=: w1:word
<: w2:word
w1 .lex. w2
"""
resultsTight = A.search(queryTight)
2.41s 206 results
We miss 23 results. How can that be?
Well, in MQL
[phrase]
[word]
[word]
means that the two words are adjacent in the phrase. If the phrase has gaps, words around a gap are still adjacent.
In TF it is different, there such words are not adjacent.
Let's lift the requirement of adjacency from the TF query:
queryLoose = """
phrase
=: w1:word
< w2:word
w1 .lex. w2
"""
resultsLoose = A.search(queryLoose)
2.33s 6735 results
Way to many results.
Let's filter these results and only retain those where w1
and w2
are adjacent in the MQL sense.
def filterTight(results):
good = []
for (p, w1, w2) in results:
phraseWords = set(L.d(p, otype="word"))
if all(w not in phraseWords for w in range(w1 + 1, w2)):
good.append((p, w1, w2))
return good
len(filterTight(resultsLoose))
229
So it looks like we have the same set in our hands.
There must be 23 cases of phrases with an intervening phrase after the first word, after which the phrase resumes with a word of the same lexeme. Let's identify those.
From search Gaps we pick the query that finds all gapped phrases, and we store it in a set.
queryGap = """
p:phrase
wPreGap:word
wLast:word
:=
wGap:word
wPreGap <: wGap
wGap < wLast
p || wGap
"""
gaps = A.search(queryGap, shallow=True)
2.50s 671 results
Now we modify our loose query to run over gapped phrases only:
queryLooseGap = """
phrasegap
=: w1:word
< w2:word
w1 .lex. w2
"""
resultsLooseGap = A.search(queryLooseGap, sets=dict(phrasegap=gaps))
0.52s 492 results
And apply the filter again:
resultsGap = filterTight(resultsLooseGap)
len(resultsGap)
24
Now we have one more than expected. It could be that one of the gapped phrases has an adjacent pair that does not lie around a gap.
Indeed, result 9 is such a case.
A.show(resultsGap, start=8, end=10, condenseType="clause")
result 8
result 9
result 10
We look for words with the same value for the g_cons
feature but with a different lex
feature.
There is a word pair in Jeremiah 1:11-12 that has this.
An obvious query to try is the following:
query = """
verse book=Jeremia chapter=1 verse=11|12
w1:word lex*
< w2:word
w1 .g_cons=g_cons. w2
w1 .lex#lex. w2
"""
results = A.search(query)
0.54s 0 results
But the thing is, one of those words is in verse 11, and the other in verse 12. The query as it stands, requires both words to be in the same verse.
We could write a bit more intricate query, that requires the word pair between the last word of verse 10 and the first one of verse 13, but this query turns out to be inefficient. It finds a result in a few seconds (that is already way to slow) and then it spends ages to search a vast space where no results are to be found.
query = """
verse book=Jeremia chapter=1 verse=10
start:word
:=
verse book=Jeremia chapter=1 verse=13
=: end:word
start
< w1:word
< w2:word
< end
w1 .g_cons=g_cons. w2
w1 .lex#lex. w2
"""
S.study(query)
0.00s Checking search template ... 0.00s Setting up search space for 8 objects ... 0.38s Constraining search space with 11 relations ... 0.45s 2 edges thinned 0.45s Setting up retrieval plan with strategy small_choice_multi ... 0.50s Ready to deliver results from 1706340 nodes Iterate over S.fetch() to get the results See S.showPlan() to interpret the results
firstResult = next(S.fetch())
A.prettyTuple(firstResult, 1, condenseType="clause", skipCols="1 3")
result 1
An alternative is to look in a whole chapter, and to require the words not to be too far apart, e.g. at most 20.
query = """
chapter book=Jeremia chapter=1
w1:word
<20: w2:word
w1 < w2
w1 .g_cons=g_cons. w2
w1 .lex#lex. w2
"""
Note that <20:
relaxes to adjacency condition <:
with 20 slots in both directions.
If there is a pair of words within a slot distance of 20, then the reversed pair also satisfies that condition.
Hence we put in an extra w1 < w2
, so that every pair occurs only once.
results = A.search(query)
0.23s 1 result
A.show(results, condenseType="clause", skipCols="1")
result 1
We can do so for the whole book of Jeremiah.
query = """
book book=Jeremia
w1:word
<20: w2:word
w1 < w2
w1 .g_cons=g_cons. w2
w1 .lex#lex. w2
"""
results = A.search(query)
2.93s 277 results
A.show(results, condenseType="clause", end=3, skipCols="1")
result 1
result 2
result 3
Let's try the whole Bible in this way:
query = """
w1:word
<20: w2:word
w1 < w2
w1 .g_cons=g_cons. w2
w1 .lex#lex. w2
"""
results = A.search(query)
15s 3423 results
TF is struggling with it, but it delivers!
A.show(results, condenseType="clause", end=3)
result 1
result 2
result 3
Martijn Naaijer: Object clauses with >CR
Select all objects where
[clause rela = Objc
[word focus first lex = '>CR']
]
157 results
query = """
verse
clause rela=Objc
=: word lex=>CR
"""
results = A.search(query)
A.table(results, end=10)
0.47s 96 results
n | p | verse | clause | word |
---|---|---|---|---|
1 | Genesis 14:24 | אֲשֶׁ֣ר אָֽכְל֣וּ הַנְּעָרִ֔ים | אֲשֶׁ֣ר | |
2 | Genesis 18:17 | אֲשֶׁ֖ר אֲנִ֥י עֹשֶֽׂה׃ | אֲשֶׁ֖ר | |
3 | Genesis 24:3 | אֲשֶׁ֨ר לֹֽא־תִקַּ֤ח אִשָּׁה֙ לִבְנִ֔י מִבְּנֹות֙ הַֽכְּנַעֲנִ֔י | אֲשֶׁ֨ר | |
4 | Genesis 34:11 | אֲשֶׁ֥ר תֹּאמְר֛וּ אֵלַ֖י | אֲשֶׁ֥ר | |
5 | Genesis 39:23 | אֲשֶׁר־ה֥וּא עֹשֶׂ֖ה | אֲשֶׁר־ | |
6 | Genesis 41:28 | אֲשֶׁ֧ר הָאֱלֹהִ֛ים עֹשֶׂ֖ה | אֲשֶׁ֧ר | |
7 | Genesis 41:55 | אֲשֶׁר־יֹאמַ֥ר לָכֶ֖ם | אֲשֶׁר־ | |
8 | Genesis 44:5 | אֲשֶׁ֥ר עֲשִׂיתֶֽם׃ | אֲשֶׁ֥ר | |
9 | Exodus 4:12 | אֲשֶׁ֥ר תְּדַבֵּֽר׃ | אֲשֶׁ֥ר | |
10 | Exodus 5:21 | אֲשֶׁ֧ר הִבְאַשְׁתֶּ֣ם אֶת־רֵיחֵ֗נוּ בְּעֵינֵ֤י פַרְעֹה֙ וּבְעֵינֵ֣י עֲבָדָ֔יו | אֲשֶׁ֧ר |
We have fewer cases: 96 instead of 157.
We are working on the ETCBC version 2017, and the query has been executed against 4b.
There have been coding updates that are relevant to this query, e.g. in Genesis 43:27, which is in the results
on SHEBANQ, but not here. In 2017 the rela
is Attr
, and not Objc
:
query = """
verse book=Genesis chapter=43 verse=27
clause
=: word lex=>CR
"""
results = A.search(query)
A.show(results)
0.45s 1 result
result 1
Cody Kingham: Mutual Information Hierarchies. p.18n49. First Person Verbs in Narrative
SELECT ALL OBJECTS WHERE
[book
[clause txt = 'N'
[word FOCUS sp = verb
[word ps = p1
]
]
]
]
OR
[book
[clause txt = '?N'
[word FOCUS sp = verb
[word ps = p1
]
]
]
]
273 results.
query = """
book
clause txt=N|?N
word sp=verb ps=p1
"""
results = A.search(query)
A.table(results, end=5)
0.56s 272 results
n | p | book | clause | word |
---|---|---|---|---|
1 | Deuteronomy 2:1 | Deuteronomy | וַנֵּ֜פֶן | נֵּ֜פֶן |
2 | Deuteronomy 2:1 | Deuteronomy | וַנִּסַּ֤ע הַמִּדְבָּ֨רָה֙ דֶּ֣רֶךְ יַם־ס֔וּף | נִּסַּ֤ע |
3 | Deuteronomy 2:1 | Deuteronomy | וַנָּ֥סָב אֶת־הַר־שֵׂעִ֖יר יָמִ֥ים רַבִּֽים׃ ס | נָּ֥סָב |
4 | Deuteronomy 2:8 | Deuteronomy | וַֽנַּעֲבֹ֞ר מֵאֵ֧ת אַחֵ֣ינוּ בְנֵי־עֵשָׂ֗ו מִדֶּ֨רֶךְ֙ הָֽעֲרָבָ֔ה מֵאֵילַ֖ת וּמֵעֶצְיֹ֣ן גָּ֑בֶר ס | נַּעֲבֹ֞ר |
5 | Deuteronomy 2:8 | Deuteronomy | וַנֵּ֨פֶן֙ | נֵּ֨פֶן֙ |
One result less. Again, a coding difference between versions. Exercise: find out where that happened.
Reinoud Oosting: to go + object marker
Select all objects
where
[clause
[phrase function = Pred OR function = PreC
[word FOCUS sp = verb AND vs = qal AND lex = "HLK[" ]
]
..
[phrase FOCUS
[word First lex = ">T"]
]
]
OR
[clause
[phrase FOCUS
[word First lex = ">T" ]
]
..
[phrase function = Pred OR function = PreC
[word FOCUS sp = verb AND vs = qal AND lex = "HLK["]
]
]
4 results.
This is a case where we can simplify greatly because we are not hampered by automatic constraints on the order of the phrases.
query = """
clause
p1:phrase function=Pred|PreC
word sp=verb vs=qal lex=HLK[
p2:phrase
=: word lex=>T
p1 # p2
"""
results = A.search(query)
A.show(sorted(results), condensed=False, condenseType="clause")
1.25s 4 results
result 1
result 2
result 3
result 4
Reinoud Oosting: To establish covenant
select all objects
where
[clause
[phrase function = Pred OR function = PreC
[word FOCUS sp = verb AND vs = hif AND lex = "QWM[" ]
]
..
[phrase function = Objc
[word FOCUS lex = "BRJT/" ]
]
]
OR
[clause
[phrase function = Objc
[word FOCUS lex = "BRJT/" ]
]
..
[phrase function = Pred OR function = PreC
[word FOCUS sp = verb AND vs = hif AND lex = "QWM["]
]
]
13 results
query = """
clause
phrase function=Pred|PreC
word sp=verb vs=hif lex=QWM[
phrase function=Objc
word lex=BRJT/
"""
results = A.search(query)
resultsx = sorted(
(L.u(r[0], otype="verse") + r for r in results), key=lambda r: N.sortKey(r[0])
)
A.table(sorted(resultsx))
A.show(resultsx, start=4, end=4)
1.25s 13 results
n | p | verse | clause | phrase | word | phrase | word |
---|---|---|---|---|---|---|---|
1 | Genesis 6:18 | וַהֲקִמֹתִ֥י אֶת־בְּרִיתִ֖י אִתָּ֑ךְ | הֲקִמֹתִ֥י | הֲקִמֹתִ֥י | אֶת־בְּרִיתִ֖י | בְּרִיתִ֖י | |
2 | Genesis 9:9 | הִנְנִ֥י מֵקִ֛ים אֶת־בְּרִיתִ֖י אִתְּכֶ֑ם וְאֶֽת־זַרְעֲכֶ֖ם אַֽחֲרֵיכֶֽם׃ | מֵקִ֛ים | מֵקִ֛ים | אֶת־בְּרִיתִ֖י | בְּרִיתִ֖י | |
3 | Genesis 9:11 | וַהֲקִמֹתִ֤י אֶת־בְּרִיתִי֙ אִתְּכֶ֔ם | הֲקִמֹתִ֤י | הֲקִמֹתִ֤י | אֶת־בְּרִיתִי֙ | בְּרִיתִי֙ | |
4 | Genesis 17:7 | וַהֲקִמֹתִ֨י אֶת־בְּרִיתִ֜י בֵּינִ֣י וּבֵינֶ֗ךָ וּבֵ֨ין זַרְעֲךָ֧ אַחֲרֶ֛יךָ לְדֹרֹתָ֖ם לִבְרִ֣ית עֹולָ֑ם | הֲקִמֹתִ֨י | הֲקִמֹתִ֨י | אֶת־בְּרִיתִ֜י | בְּרִיתִ֜י | |
5 | Genesis 17:19 | וַהֲקִמֹתִ֨י אֶת־בְּרִיתִ֥י אִתֹּ֛ו לִבְרִ֥ית עֹולָ֖ם לְזַרְעֹ֥ו אַחֲרָֽיו׃ | הֲקִמֹתִ֨י | הֲקִמֹתִ֨י | אֶת־בְּרִיתִ֥י | בְּרִיתִ֥י | |
6 | Genesis 17:21 | וְאֶת־בְּרִיתִ֖י אָקִ֣ים אֶת־יִצְחָ֑ק | אָקִ֣ים | אָקִ֣ים | אֶת־בְּרִיתִ֖י | בְּרִיתִ֖י | |
7 | Exodus 6:4 | וְגַ֨ם הֲקִמֹ֤תִי אֶת־בְּרִיתִי֙ אִתָּ֔ם | הֲקִמֹ֤תִי | הֲקִמֹ֤תִי | אֶת־בְּרִיתִי֙ | בְּרִיתִי֙ | |
8 | Leviticus 26:9 | וַהֲקִימֹתִ֥י אֶת־בְּרִיתִ֖י אִתְּכֶֽם׃ | הֲקִימֹתִ֥י | הֲקִימֹתִ֥י | אֶת־בְּרִיתִ֖י | בְּרִיתִ֖י | |
9 | Deuteronomy 8:18 | לְמַ֨עַן הָקִ֧ים אֶת־בְּרִיתֹ֛ו | לְמַ֨עַן הָקִ֧ים | הָקִ֧ים | אֶת־בְּרִיתֹ֛ו | בְּרִיתֹ֛ו | |
10 | 2_Kings 23:3 | לְהָקִ֗ים אֶת־דִּבְרֵי֙ הַבְּרִ֣ית הַזֹּ֔את | לְהָקִ֗ים | הָקִ֗ים | אֶת־דִּבְרֵי֙ הַבְּרִ֣ית הַזֹּ֔את | בְּרִ֣ית | |
11 | Jeremiah 34:18 | אֲשֶׁ֤ר לֹֽא־הֵקִ֨ימוּ֙ אֶת־דִּבְרֵ֣י הַבְּרִ֔ית | הֵקִ֨ימוּ֙ | הֵקִ֨ימוּ֙ | אֶת־דִּבְרֵ֣י הַבְּרִ֔ית | בְּרִ֔ית | |
12 | Ezekiel 16:60 | וַהֲקִמֹותִ֥י לָ֖ךְ בְּרִ֥ית עֹולָֽם׃ | הֲקִמֹותִ֥י | הֲקִמֹותִ֥י | בְּרִ֥ית עֹולָֽם׃ | בְּרִ֥ית | |
13 | Ezekiel 16:62 | וַהֲקִימֹותִ֥י אֲנִ֛י אֶת־בְּרִיתִ֖י אִתָּ֑ךְ | הֲקִימֹותִ֥י | הֲקִימֹותִ֥י | אֶת־בְּרִיתִ֖י | בְּרִיתִ֖י |
result 4
Reinoud Oosting: To find grace in sight of
select all objects
where
[clause
[phrase FOCUS function = Pred OR function = PreC
[word sp = verb AND vs = qal AND lex = "MY>[" ]
]
..
[phrase function = Objc
[word FOCUS lex = "XN/" ]
]
[phrase function = Cmpl
[word FOCUS lex = "B"]
[word FOCUS lex = "<JN/"]
]
]
OR
[clause
[phrase function = Objc
[word FOCUS lex = "XN/" ]
]
[phrase function = Cmpl
[word FOCUS lex = "B"]
[word FOCUS lex = "<JN/"]
..
[phrase function = Pred OR function = PreC
[word FOCUS sp = verb AND vs = qal AND lex = "MY>["]
]
]
]
38 results
query = """
clause
p1:phrase function=Pred|PreC
word sp=verb vs=qal lex=MY>[
p2:phrase function=Objc
word lex=XN/
p3:phrase function=Cmpl
word lex=B
<: word lex=<JN/
p2 << p3
"""
results = A.search(query)
resultsx = sorted(
(L.u(r[0], otype="verse") + r for r in results), key=lambda r: N.sortKey(r[0])
)
A.table(resultsx)
2.25s 40 results
n | p | verse | clause | phrase | word | phrase | word | phrase | word | word |
---|---|---|---|---|---|---|---|---|---|---|
1 | Genesis 6:8 | וְנֹ֕חַ מָ֥צָא חֵ֖ן בְּעֵינֵ֥י יְהוָֽה׃ פ | מָ֥צָא | מָ֥צָא | חֵ֖ן | חֵ֖ן | בְּעֵינֵ֥י יְהוָֽה׃ פ | בְּ | עֵינֵ֥י | |
2 | Genesis 18:3 | אִם־נָ֨א מָצָ֤אתִי חֵן֙ בְּעֵינֶ֔יךָ | מָצָ֤אתִי | מָצָ֤אתִי | חֵן֙ | חֵן֙ | בְּעֵינֶ֔יךָ | בְּ | עֵינֶ֔יךָ | |
3 | Genesis 19:19 | הִנֵּה־נָ֠א מָצָ֨א עַבְדְּךָ֣ חֵן֮ בְּעֵינֶיךָ֒ | מָצָ֨א | מָצָ֨א | חֵן֮ | חֵן֮ | בְּעֵינֶיךָ֒ | בְּ | עֵינֶיךָ֒ | |
4 | Genesis 30:27 | אִם־נָ֛א מָצָ֥אתִי חֵ֖ן בְּעֵינֶ֑יךָ | מָצָ֥אתִי | מָצָ֥אתִי | חֵ֖ן | חֵ֖ן | בְּעֵינֶ֑יךָ | בְּ | עֵינֶ֑יךָ | |
5 | Genesis 32:6 | לִמְצֹא־חֵ֖ן בְּעֵינֶֽיךָ׃ | לִמְצֹא־ | מְצֹא־ | חֵ֖ן | חֵ֖ן | בְּעֵינֶֽיךָ׃ | בְּ | עֵינֶֽיךָ׃ | |
6 | Genesis 33:8 | לִמְצֹא־חֵ֖ן בְּעֵינֵ֥י אֲדֹנִֽי׃ | לִמְצֹא־ | מְצֹא־ | חֵ֖ן | חֵ֖ן | בְּעֵינֵ֥י אֲדֹנִֽי׃ | בְּ | עֵינֵ֥י | |
7 | Genesis 33:10 | אִם־נָ֨א מָצָ֤אתִי חֵן֙ בְּעֵינֶ֔יךָ | מָצָ֤אתִי | מָצָ֤אתִי | חֵן֙ | חֵן֙ | בְּעֵינֶ֔יךָ | בְּ | עֵינֶ֔יךָ | |
8 | Genesis 33:15 | לָ֣מָּה זֶּ֔ה אֶמְצָא־חֵ֖ן בְּעֵינֵ֥י אֲדֹנִֽי׃ | אֶמְצָא־ | אֶמְצָא־ | חֵ֖ן | חֵ֖ן | בְּעֵינֵ֥י אֲדֹנִֽי׃ | בְּ | עֵינֵ֥י | |
9 | Genesis 34:11 | אֶמְצָא־חֵ֖ן בְּעֵינֵיכֶ֑ם | אֶמְצָא־ | אֶמְצָא־ | חֵ֖ן | חֵ֖ן | בְּעֵינֵיכֶ֑ם | בְּ | עֵינֵיכֶ֑ם | |
10 | Genesis 39:4 | וַיִּמְצָ֨א יֹוסֵ֥ף חֵ֛ן בְּעֵינָ֖יו | יִּמְצָ֨א | יִּמְצָ֨א | חֵ֛ן | חֵ֛ן | בְּעֵינָ֖יו | בְּ | עֵינָ֖יו | |
11 | Genesis 47:25 | נִמְצָא־חֵן֙ בְּעֵינֵ֣י אֲדֹנִ֔י | נִמְצָא־ | נִמְצָא־ | חֵן֙ | חֵן֙ | בְּעֵינֵ֣י אֲדֹנִ֔י | בְּ | עֵינֵ֣י | |
12 | Genesis 47:29 | אִם־נָ֨א מָצָ֤אתִי חֵן֙ בְּעֵינֶ֔יךָ | מָצָ֤אתִי | מָצָ֤אתִי | חֵן֙ | חֵן֙ | בְּעֵינֶ֔יךָ | בְּ | עֵינֶ֔יךָ | |
13 | Genesis 50:4 | אִם־נָ֨א מָצָ֤אתִי חֵן֙ בְּעֵ֣ינֵיכֶ֔ם | מָצָ֤אתִי | מָצָ֤אתִי | חֵן֙ | חֵן֙ | בְּעֵ֣ינֵיכֶ֔ם | בְּ | עֵ֣ינֵיכֶ֔ם | |
14 | Exodus 33:12 | וְגַם־מָצָ֥אתָ חֵ֖ן בְּעֵינָֽי׃ | מָצָ֥אתָ | מָצָ֥אתָ | חֵ֖ן | חֵ֖ן | בְּעֵינָֽי׃ | בְּ | עֵינָֽי׃ | |
15 | Exodus 33:13 | אִם־נָא֩ מָצָ֨אתִי חֵ֜ן בְּעֵינֶ֗יךָ | מָצָ֨אתִי | מָצָ֨אתִי | חֵ֜ן | חֵ֜ן | בְּעֵינֶ֗יךָ | בְּ | עֵינֶ֗יךָ | |
16 | Exodus 33:13 | לְמַ֥עַן אֶמְצָא־חֵ֖ן בְּעֵינֶ֑יךָ | אֶמְצָא־ | אֶמְצָא־ | חֵ֖ן | חֵ֖ן | בְּעֵינֶ֑יךָ | בְּ | עֵינֶ֑יךָ | |
17 | Exodus 33:16 | כִּֽי־מָצָ֨אתִי חֵ֤ן בְּעֵינֶ֨יךָ֙ אֲנִ֣י וְעַמֶּ֔ךָ | מָצָ֨אתִי | מָצָ֨אתִי | חֵ֤ן | חֵ֤ן | בְּעֵינֶ֨יךָ֙ | בְּ | עֵינֶ֨יךָ֙ | |
18 | Exodus 33:17 | כִּֽי־מָצָ֤אתָ חֵן֙ בְּעֵינַ֔י | מָצָ֤אתָ | מָצָ֤אתָ | חֵן֙ | חֵן֙ | בְּעֵינַ֔י | בְּ | עֵינַ֔י | |
19 | Exodus 34:9 | אִם־נָא֩ מָצָ֨אתִי חֵ֤ן בְּעֵינֶ֨יךָ֙ | מָצָ֨אתִי | מָצָ֨אתִי | חֵ֤ן | חֵ֤ן | בְּעֵינֶ֨יךָ֙ | בְּ | עֵינֶ֨יךָ֙ | |
20 | Numbers 11:11 | וְלָ֛מָּה לֹא־מָצָ֥תִי חֵ֖ן בְּעֵינֶ֑יךָ | מָצָ֥תִי | מָצָ֥תִי | חֵ֖ן | חֵ֖ן | בְּעֵינֶ֑יךָ | בְּ | עֵינֶ֑יךָ | |
21 | Numbers 11:15 | אִם־מָצָ֥אתִי חֵ֖ן בְּעֵינֶ֑יךָ | מָצָ֥אתִי | מָצָ֥אתִי | חֵ֖ן | חֵ֖ן | בְּעֵינֶ֑יךָ | בְּ | עֵינֶ֑יךָ | |
22 | Numbers 32:5 | אִם־מָצָ֤אנוּ חֵן֙ בְּעֵינֶ֔יךָ | מָצָ֤אנוּ | מָצָ֤אנוּ | חֵן֙ | חֵן֙ | בְּעֵינֶ֔יךָ | בְּ | עֵינֶ֔יךָ | |
23 | Deuteronomy 24:1 | אִם־לֹ֧א תִמְצָא־חֵ֣ן בְּעֵינָ֗יו | תִמְצָא־ | תִמְצָא־ | חֵ֣ן | חֵ֣ן | בְּעֵינָ֗יו | בְּ | עֵינָ֗יו | |
24 | Judges 6:17 | אִם־נָ֛א מָצָ֥אתִי חֵ֖ן בְּעֵינֶ֑יךָ | מָצָ֥אתִי | מָצָ֥אתִי | חֵ֖ן | חֵ֖ן | בְּעֵינֶ֑יךָ | בְּ | עֵינֶ֑יךָ | |
25 | 1_Samuel 1:18 | תִּמְצָ֧א שִׁפְחָתְךָ֛ חֵ֖ן בְּעֵינֶ֑יךָ | תִּמְצָ֧א | תִּמְצָ֧א | חֵ֖ן | חֵ֖ן | בְּעֵינֶ֑יךָ | בְּ | עֵינֶ֑יךָ | |
26 | 1_Samuel 16:22 | כִּֽי־מָ֥צָא חֵ֖ן בְּעֵינָֽי׃ | מָ֥צָא | מָ֥צָא | חֵ֖ן | חֵ֖ן | בְּעֵינָֽי׃ | בְּ | עֵינָֽי׃ | |
27 | 1_Samuel 20:3 | כִּֽי־מָצָ֤אתִי חֵן֙ בְּעֵינֶ֔יךָ | מָצָ֤אתִי | מָצָ֤אתִי | חֵן֙ | חֵן֙ | בְּעֵינֶ֔יךָ | בְּ | עֵינֶ֔יךָ | |
28 | 1_Samuel 20:29 | אִם־מָצָ֤אתִי חֵן֙ בְּעֵינֶ֔יךָ | מָצָ֤אתִי | מָצָ֤אתִי | חֵן֙ | חֵן֙ | בְּעֵינֶ֔יךָ | בְּ | עֵינֶ֔יךָ | |
29 | 1_Samuel 25:8 | וְיִמְצְא֨וּ הַנְּעָרִ֥ים חֵן֙ בְּעֵינֶ֔יךָ | יִמְצְא֨וּ | יִמְצְא֨וּ | חֵן֙ | חֵן֙ | בְּעֵינֶ֔יךָ | בְּ | עֵינֶ֔יךָ | |
30 | 1_Samuel 27:5 | אִם־נָא֩ מָצָ֨אתִי חֵ֤ן בְּעֵינֶ֨יךָ֙ | מָצָ֨אתִי | מָצָ֨אתִי | חֵ֤ן | חֵ֤ן | בְּעֵינֶ֨יךָ֙ | בְּ | עֵינֶ֨יךָ֙ | |
31 | 2_Samuel 14:22 | כִּי־מָצָ֨אתִי חֵ֤ן בְּעֵינֶ֨יךָ֙ | מָצָ֨אתִי | מָצָ֨אתִי | חֵ֤ן | חֵ֤ן | בְּעֵינֶ֨יךָ֙ | בְּ | עֵינֶ֨יךָ֙ | |
32 | 2_Samuel 15:25 | אִם־אֶמְצָ֥א חֵן֙ בְּעֵינֵ֣י יְהוָ֔ה | אֶמְצָ֥א | אֶמְצָ֥א | חֵן֙ | חֵן֙ | בְּעֵינֵ֣י יְהוָ֔ה | בְּ | עֵינֵ֣י | |
33 | 2_Samuel 16:4 | אֶמְצָא־חֵ֥ן בְּעֵינֶ֖יךָ | אֶמְצָא־ | אֶמְצָא־ | חֵ֥ן | חֵ֥ן | בְּעֵינֶ֖יךָ | בְּ | עֵינֶ֖יךָ | |
34 | 1_Kings 11:19 | וַיִּמְצָא֙ הֲדַ֥ד חֵ֛ן בְּעֵינֵ֥י פַרְעֹ֖ה מְאֹ֑ד | יִּמְצָא֙ | יִּמְצָא֙ | חֵ֛ן | חֵ֛ן | בְּעֵינֵ֥י פַרְעֹ֖ה | בְּ | עֵינֵ֥י | |
35 | Proverbs 3:4 | וּמְצָא־חֵ֖ן וְשֵֽׂכֶל־טֹ֑וב בְּעֵינֵ֖י אֱלֹהִ֣ים וְאָדָֽם׃ פ | מְצָא־ | מְצָא־ | חֵ֖ן וְשֵֽׂכֶל־טֹ֑וב | חֵ֖ן | בְּעֵינֵ֖י אֱלֹהִ֣ים וְאָדָֽם׃ פ | בְּ | עֵינֵ֖י | |
36 | Ruth 2:2 | אַחַ֕ר אֲשֶׁ֥ר אֶמְצָא־חֵ֖ן בְּעֵינָ֑יו | אֶמְצָא־ | אֶמְצָא־ | חֵ֖ן | חֵ֖ן | בְּעֵינָ֑יו | בְּ | עֵינָ֑יו | |
37 | Ruth 2:10 | מַדּוּעַ֩ מָצָ֨אתִי חֵ֤ן בְּעֵינֶ֨יךָ֙ | מָצָ֨אתִי | מָצָ֨אתִי | חֵ֤ן | חֵ֤ן | בְּעֵינֶ֨יךָ֙ | בְּ | עֵינֶ֨יךָ֙ | |
38 | Ruth 2:13 | אֶמְצָא־חֵ֨ן בְּעֵינֶ֤יךָ | אֶמְצָא־ | אֶמְצָא־ | חֵ֨ן | חֵ֨ן | בְּעֵינֶ֤יךָ | בְּ | עֵינֶ֤יךָ | |
39 | Esther 5:8 | אִם־מָצָ֨אתִי חֵ֜ן בְּעֵינֵ֣י הַמֶּ֗לֶךְ | מָצָ֨אתִי | מָצָ֨אתִי | חֵ֜ן | חֵ֜ן | בְּעֵינֵ֣י הַמֶּ֗לֶךְ | בְּ | עֵינֵ֣י | |
40 | Esther 7:3 | אִם־מָצָ֨אתִי חֵ֤ן בְּעֵינֶ֨יךָ֙ | מָצָ֨אתִי | מָצָ֨אתִי | חֵ֤ן | חֵ֤ן | בְּעֵינֶ֨יךָ֙ | בְּ | עֵינֶ֨יךָ֙ |
Two results more. Spot the differences.
SELECT ALL OBJECTS WHERE
[clause
[phrase function IN (Subj)
[phrase_atom NOT rela IN (Appo,Para,Spec)
[word FOCUS pdp IN (subs,nmpr,prps,prde,prin,adjv)
]
]
]
NOTEXIST [phrase function IN (Pred)]
..
NOTEXIST [phrase function IN (Pred)]
[phrase function IN (PreC)
NOTEXIST [word pdp IN (prep)]
[word FOCUS pdp IN (subs,nmpr,prin,adjv) AND ls IN (card,ordn)]
]
]
2303 results with 2129 words in those results.
We can deal with NOTEXIST
by means of the quantifier /without/
.
We can also state that features do not have certain values.
And we play with the spatial relations.
query = """
clause
phrase function=Subj
/without/
<: phrase function=Pred
/-/
phrase_atom rela#Appo|Para|Spec
word pdp=subs|nmpr|prps|prde|prin|adjv
<< phrase function=PreC
/without/
:> phrase function=Pred
/-/
/without/
word pdp=prep
/-/
word pdp=subs|nmpr|prin|adjv ls=card|ordn
"""
results = A.search(query)
clauses = project(results, 1)
print(f"{len(clauses)} clauses in results")
1.42s 2288 results 519 clauses in results
We have 15 results less than the MQL query on SHEBANQ.
Let us have a look at some results words and compare them with the result words on SHEBANQ. It is handy to fetch from SHEBANQ the CSV file with query results.
We have fetched them and stored them in fromShebanq.csv
in the same directory.
It is a list of words occurring in results, so let's see which clauses are in the SHEBANQ results.
book,chapter,verse,monad,text,ktv,phtext,phsep
Genesis,5,4,2169,יְמֵי־,,yᵊmê-,
Genesis,5,4,2170,אָדָ֗ם ,,ʔāḏˈām,
Genesis,5,4,2175,שְׁמֹנֶ֥ה ,,šᵊmōnˌeh,
Genesis,5,4,2176,מֵאֹ֖ת ,,mēʔˌōṯ,
Genesis,5,5,2185,כָּל־,,kol-,
Genesis,5,5,2186,יְמֵ֤י ,,yᵊmˈê,
Genesis,5,5,2187,אָדָם֙ ,,ʔāḏˌām,
Genesis,5,5,2190,תְּשַׁ֤ע ,,tᵊšˈaʕ,
Genesis,5,5,2191,מֵאֹות֙ ,,mēʔôṯ,
Genesis,5,5,2194,שְׁלֹשִׁ֖ים ,,šᵊlōšˌîm,
shebanqClauses = set()
with open("fromShebanq.csv") as fh:
for (i, line) in enumerate(fh):
if i == 0:
continue
fields = line.split(",")
word = int(fields[3])
clause = L.u(word, otype="clause")[0]
shebanqClauses.add(clause)
len(shebanqClauses)
519
That looks good: both methods yield the same amount of clauses.
But we need to be thorough.
clauses == shebanqClauses
False
See? They are not the same clauses.
Let's spot the differences.
tfNotMql = clauses - shebanqClauses
mqlNotTf = shebanqClauses - clauses
print(f"Results of TF but not MQL: {sorted(tfNotMql)}")
print(f"Results of MQL but not TF : {sorted(mqlNotTf)}")
Results of TF but not MQL: [482510, 485154, 507046] Results of MQL but not TF : [427990, 452091, 511757]
First we do the results that TF provides, but not MQL.
A.displaySetup(extraFeatures="ls")
newResults = [r for r in results if r[0] in tfNotMql]
newResults
[(482510, 813849, 1076419, 275732, 813850, 275736), (482510, 813849, 1076419, 275733, 813850, 275736), (485154, 821404, 1084317, 288698, 821405, 288703), (485154, 821404, 1084317, 288701, 821405, 288703), (507046, 879338, 1143889, 375038, 879339, 375041), (507046, 879338, 1143889, 375039, 879339, 375041)]
We are going to inspect them clause by clause. Note that we have two results per clause, the only difference between the two results is in column 4, which corresponds to the word in the Subj phrase.
A.show(newResults, condensed=True, withNodes=True, condenseType="clause")
clause 1
clause 2
clause 3
In all three cases we see a Pred phrase somewhere after the PreC
phrase.
The NOTEXIST
of MQL works a bit subtle: the not-exists claim holds from the place where it appears till the end
of the surrounding context.
So, in fact, the second NOTEXIST
is redundant. Following the MQL query, the clause cannot have a Pred phrase beyond
the Subj phrase.
Before we remedy our TF query to match this effect, let us inspect the clauses delivered by MQL, but not by TF.
Most of the effort in the code below is to furnish appropriate highlighting.
def showClause(clause):
highlights = {}
for phrase in L.d(clause, otype="phrase"):
pf = F.function.v(phrase)
if pf == "Subj":
highlights[phrase] = "cyan"
for phraseAtom in L.d(phrase, otype="phrase_atom"):
rela = F.rela.v(phraseAtom)
if rela in {"Appo", "Para", "Spec"}:
continue
words = L.d(phraseAtom, otype="word")
for word in words:
pdp = F.pdp.v(word)
if pdp in {"subs", "nmpr", "prps", "prde", "prin", "adjv"}:
highlights[word] = "yellow"
elif pf == "PreC":
highlights[phrase] = "lightskyblue"
words = L.d(phrase, otype="word")
if any(F.pdp.v(word) == "prep" for word in words):
continue
for word in words:
pdp = F.pdp.v(word)
ls = F.ls.v(word)
if ls in {"card", "ord"} and pdp in {"subs", "nmpr", "prin", "adjv"}:
highlights[word] = "yellow"
elif pf == "Pred":
highlights[phrase] = "coral"
A.pretty(clause, withNodes=True, highlights=highlights)
mqlClauses = sorted(mqlNotTf)
We inspect the cases one by one:
showClause(mqlClauses[0])
What could be wrong here? The only violation could be in the gap. What happens before the PreC
phrase?
If there is an adjacent Pred phrase, it explains why this does not show up in the TF query results.
Let's find out.
xPhrase = L.u(2189, otype="phrase")[0]
A.pretty(
xPhrase,
withNodes=True,
highlights={xPhrase: "coral" if F.function.v(xPhrase) == "Pred" else "lightyellow"},
)
Clearly, this is the culprit. it is in the same clause.
showClause(mqlClauses[1])
Again, a gap just before the Prec
phrase. Indeed:
xPhrase = L.u(132678, otype="phrase")[0]
A.pretty(
xPhrase,
withNodes=True,
highlights={xPhrase: "coral" if F.function.v(xPhrase) == "Pred" else "lightyellow"},
)
showClause(mqlClauses[2])
We are getting used to it!
xPhrase = L.u(403004, otype="phrase")[0]
A.pretty(
xPhrase,
withNodes=True,
highlights={xPhrase: "coral" if F.function.v(xPhrase) == "Pred" else "lightyellow"},
)
But no, here we have a different cause. Probably a Pred phrase right after the Subj phrase.
xPhrase = L.u(402999, otype="phrase")[0]
A.pretty(
xPhrase,
withNodes=True,
highlights={xPhrase: "coral" if F.function.v(xPhrase) == "Pred" else "lightyellow"},
)
We have seen all the causes why the TF search and the MQL query produced different results.
Now we are going to remedy the TF query, such that it produces the same results as the MQL.
Let us start with what we just saw: when we stipulate the non-existence of a Pred phrase, we only claim that such a phrase does not occur in the same clause.
Then we remove the second non-existence claim of a Pred phrase, since the MQL query just stipulates that there is no Pred phrase after the Subj phrase.
But then we can make the quantifier much simpler. Instead of applying it to the Subj phrase, we apply it to the enclosing clause. That will solve the problem of phrases outside the clause in one go!
query = """
c:clause
/without/
phrase function=Subj
<< phrase function=Pred
/-/
p:phrase function=Subj
phrase_atom rela#Appo|Para|Spec
word pdp=subs|nmpr|prps|prde|prin|adjv
<< phrase function=PreC
/without/
word pdp=prep
/-/
word pdp=subs|nmpr|prin|adjv ls=card|ordn
"""
results = A.search(query)
clauses = project(results, 1)
print(f"{len(clauses)} clauses in results")
clauses == shebanqClauses
1.43s 2303 results 519 clauses in results
True
And this is in exact agreement with the MQL query.
As a bonus, let's study this query in order to see what the quantifiers are doing.
S.study(query)
0.00s Checking search template ... 0.00s Setting up search space for 6 objects ... | 0.00s "Quantifier on "c:clause" | | /without/ | | c:clause | | phrase function=Subj | | << phrase function=Pred | | /-/ | | 0.70s 4985 nodes to exclude | 0.71s reduction from 88101 to 83116 nodes | 0.00s "Quantifier on "parent:phrase function=PreC" | | /without/ | | parent:phrase function=PreC | | word pdp=prep | | /-/ | | 0.79s 6370 nodes to exclude | 0.81s reduction from 19383 to 13013 nodes 1.22s Constraining search space with 6 relations ... 1.64s 5 edges thinned 1.65s Setting up retrieval plan with strategy small_choice_multi ... 1.65s Ready to deliver results from 5321 nodes Iterate over S.fetch() to get the results See S.showPlan() to interpret the results
query = """
verse
clause
clause_atom
phrase
phrase_atom
word lex=JC/|>JN/
"""
results = A.search(query)
A.table(sorted(results), end=7)
0.53s 926 results
n | p | verse | clause | clause_atom | phrase | phrase_atom | word |
---|---|---|---|---|---|---|---|
1 | Genesis 2:5 | וְאָדָ֣ם אַ֔יִן | וְאָדָ֣ם אַ֔יִן | אַ֔יִן | אַ֔יִן | אַ֔יִן | |
2 | Genesis 5:24 | וְאֵינֶ֕נּוּ | וְאֵינֶ֕נּוּ | אֵינֶ֕נּוּ | אֵינֶ֕נּוּ | אֵינֶ֕נּוּ | |
3 | Genesis 7:8 | אֲשֶׁ֥ר אֵינֶ֖נָּה טְהֹרָ֑ה | אֲשֶׁ֥ר אֵינֶ֖נָּה טְהֹרָ֑ה | אֵינֶ֖נָּה | אֵינֶ֖נָּה | אֵינֶ֖נָּה | |
4 | Genesis 11:30 | אֵ֥ין לָ֖הּ וָלָֽד׃ | אֵ֥ין לָ֖הּ וָלָֽד׃ | אֵ֥ין | אֵ֥ין | אֵ֥ין | |
5 | Genesis 18:24 | אוּלַ֥י יֵ֛שׁ חֲמִשִּׁ֥ים צַדִּיקִ֖ם בְּתֹ֣וךְ הָעִ֑יר | אוּלַ֥י יֵ֛שׁ חֲמִשִּׁ֥ים צַדִּיקִ֖ם בְּתֹ֣וךְ הָעִ֑יר | יֵ֛שׁ | יֵ֛שׁ | יֵ֛שׁ | |
6 | Genesis 19:31 | וְאִ֨ישׁ אֵ֤ין בָּאָ֨רֶץ֙ | וְאִ֨ישׁ אֵ֤ין בָּאָ֨רֶץ֙ | אֵ֤ין | אֵ֤ין | אֵ֤ין | |
7 | Genesis 20:7 | וְאִם־אֵֽינְךָ֣ מֵשִׁ֗יב | וְאִם־אֵֽינְךָ֣ מֵשִׁ֗יב | אֵֽינְךָ֣ | אֵֽינְךָ֣ | אֵֽינְךָ֣ |
advanced sets relations quantifiers from MQL
You master the theory.
In practice, there are pitfalls:
CC-BY Dirk Roorda