Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move BooleanScorer to work on top of Scorers rather than BulkScorers. #13931

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Move BooleanScorer to work on top of Scorers rather than BulkScorers. #13931

wants to merge 5 commits into from
+118 −197

Conversation

jpountz
Copy link
Contributor

@jpountz jpountz commented Oct 18, 2024

I was looking at some queries where Lucene performs significantly worse than Tantivy at https://tantivy-search.github.io/bench/, and found out that we get quite some overhead from implementing BooleanScorer on top of BulkScorer (effectively implemented by DefaultBulkScorer since it only runs term queries as boolean clauses) rather than Scorer directly.

The CountOrHighHigh and CountOrHighMed tasks are a bit noisy on my machine, so I did 3 runs on wikibigall, and all of them had speedups for these two tasks, often with a very low p-value.

In theory, this change could make things slower when the inner query has a specialized bulk scorer, such as MatchAllDocsQuery or a conjunction. It does feel right to optimize for term queries though.

@jpountz
Move BooleanScorer to work on top of Scorers rather than BulkScorers.
55eeb3f 
I was looking at some queries where Lucene performs significantly worse than
Tantivy at https://tantivy-search.github.io/bench/, and found out that we get
quite some overhead from implementing `BooleanScorer` on top of `BulkScorer`
(effectively implemented by `DefaultBulkScorer` since it only runs term queries
as boolean clauses) rather than `Scorer` directly.

The `CountOrHighHigh` and `CountOrHighMed` tasks are a bit noisy on my machine,
so I did 3 runs on wikibigall, and all of them had speedups for these two
tasks, often with a very low p-value.

In theory, this change could make things slower when the inner query has a
specialized bulk scorer, such as `MatchAllDocsQuery` or a conjunction. It does
feel right to optimize for term queries though.
@jpountz
Copy link
Contributor Author

jpountz commented Oct 18, 2024 

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                        Wildcard      159.36      (2.9%)      156.17      (5.3%)   -2.0% (  -9% -    6%) 0.232
                      AndHighLow      746.04      (4.5%)      732.70      (4.1%)   -1.8% (  -9% -    7%) 0.290
                CountAndHighHigh       96.45      (3.7%)       95.82      (2.7%)   -0.7% (  -6% -    6%) 0.611
             And2Terms2StopWords      209.77      (5.5%)      208.53      (5.5%)   -0.6% ( -10% -   10%) 0.784
                       And3Terms      228.00      (7.2%)      226.66      (7.4%)   -0.6% ( -14% -   15%) 0.838
               HighTermTitleSort       88.93      (2.9%)       88.48      (4.3%)   -0.5% (  -7% -    6%) 0.720
                       CountTerm     5973.32      (5.9%)     5943.88      (4.9%)   -0.5% ( -10% -   10%) 0.817
                        HighTerm      342.55      (5.2%)      341.44      (5.4%)   -0.3% ( -10% -   10%) 0.878
                 CountAndHighMed      227.59      (4.3%)      227.14      (6.2%)   -0.2% ( -10% -   10%) 0.924
                         Prefix3      221.09      (6.7%)      220.79      (5.8%)   -0.1% ( -11% -   13%) 0.956
                     AndHighHigh      126.14      (4.5%)      126.07      (2.4%)   -0.1% (  -6% -    7%) 0.967
                         LowTerm      526.10      (4.2%)      526.53      (3.9%)    0.1% (  -7% -    8%) 0.959
                       OrHighLow      521.66      (3.9%)      523.36      (2.9%)    0.3% (  -6% -    7%) 0.808
                          OrMany       24.25      (4.3%)       24.35      (2.7%)    0.4% (  -6% -    7%) 0.769
               HighTermMonthSort      790.01      (1.9%)      793.38      (2.4%)    0.4% (  -3% -    4%) 0.619
                        PKLookup      190.87      (3.0%)      191.74      (3.0%)    0.5% (  -5% -    6%) 0.697
                   OrHighNotHigh      227.35      (4.1%)      228.77      (4.7%)    0.6% (  -7% -    9%) 0.718
                          IntNRQ      257.99      (7.3%)      260.43      (6.4%)    0.9% ( -11% -   15%) 0.725
                      OrHighHigh      123.13      (5.3%)      124.34      (4.2%)    1.0% (  -8% -   11%) 0.601
                    OrNotHighLow      696.35      (5.1%)      703.21      (4.3%)    1.0% (  -8% -   10%) 0.594
                         MedTerm      422.03      (6.3%)      427.16      (5.1%)    1.2% (  -9% -   13%) 0.589
                      TermDTSort      216.84      (3.9%)      220.04      (4.4%)    1.5% (  -6% -   10%) 0.362
              Or2Terms2StopWords      215.40      (5.4%)      218.92      (4.4%)    1.6% (  -7% -   12%) 0.395
                    OrNotHighMed      262.76      (5.7%)      267.10      (3.8%)    1.7% (  -7% -   11%) 0.385
                   OrNotHighHigh      201.90      (5.5%)      205.25      (2.6%)    1.7% (  -6% -   10%) 0.324
                       OrHighMed      253.32      (6.8%)      258.46      (9.1%)    2.0% ( -12% -   19%) 0.519
            HighTermTitleBDVSort       31.08      (5.5%)       31.80      (6.3%)    2.3% (  -8% -   14%) 0.314
                      AndHighMed      227.41      (3.8%)      232.96      (6.1%)    2.4% (  -7% -   12%) 0.219
                     OrStopWords       66.05      (7.2%)       67.77      (4.0%)    2.6% (  -7% -   14%) 0.251
                    OrHighNotLow      305.24      (6.5%)      313.48      (5.1%)    2.7% (  -8% -   15%) 0.238
           HighTermDayOfYearSort      313.33      (6.7%)      321.85      (6.7%)    2.7% (  -9% -   17%) 0.300
                        Or3Terms      230.36      (5.1%)      237.14      (5.1%)    2.9% (  -6% -   13%) 0.142
                      OrHighRare      243.26      (5.7%)      250.89      (5.8%)    3.1% (  -7% -   15%) 0.163
                    AndStopWords       60.96      (6.2%)       63.00      (3.1%)    3.3% (  -5% -   13%) 0.082
                    OrHighNotMed      294.99      (4.9%)      306.41      (5.2%)    3.9% (  -5% -   14%) 0.050
                  CountOrHighMed      170.22      (6.6%)      199.04      (6.5%)   16.9% (   3% -   32%) 0.000
                 CountOrHighHigh      111.03     (10.9%)      132.94      (3.8%)   19.7% (   4% -   38%) 0.000

@jpountz
Copy link
Contributor Author

jpountz commented Oct 21, 2024

I could confirm the speedup on a different machine:

                            TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
             LowIntervalsOrdered        9.98      (4.7%)        9.61      (6.7%)   -3.7% ( -14% -    8%) 0.044
            HighIntervalsOrdered        2.34      (4.6%)        2.26      (5.1%)   -3.1% ( -12% -    6%) 0.043
             MedIntervalsOrdered       11.57      (4.6%)       11.30      (4.1%)   -2.3% ( -10% -    6%) 0.090
                       CountTerm    13086.97      (4.6%)    12820.41      (8.0%)   -2.0% ( -13% -   11%) 0.324
                          IntNRQ      250.03      (8.7%)      245.69     (10.8%)   -1.7% ( -19% -   19%) 0.576
            HighTermTitleBDVSort       34.20      (2.9%)       33.67      (4.0%)   -1.5% (  -8% -    5%) 0.164
                         LowTerm     1671.38      (3.3%)     1650.93      (5.6%)   -1.2% (  -9% -    7%) 0.395
                          Fuzzy2      115.68      (1.4%)      114.62      (2.2%)   -0.9% (  -4% -    2%) 0.109
                          Fuzzy1      157.55      (1.7%)      156.19      (2.0%)   -0.9% (  -4% -    2%) 0.146
                     MedSpanNear        4.34      (2.6%)        4.30      (3.3%)   -0.8% (  -6% -    5%) 0.402
                     CountPhrase        5.18      (7.2%)        5.14      (5.7%)   -0.8% ( -12% -   13%) 0.703
                        PKLookup      457.36      (3.7%)      454.41      (4.1%)   -0.6% (  -8% -    7%) 0.603
                     LowSpanNear        7.29      (2.6%)        7.25      (3.0%)   -0.6% (  -6% -    5%) 0.522
                    HighSpanNear        9.02      (1.7%)        8.97      (2.0%)   -0.5% (  -4% -    3%) 0.402
                CountAndHighHigh       58.51      (1.3%)       58.24      (1.4%)   -0.5% (  -3% -    2%) 0.285
                         Respell       97.94      (1.8%)       97.49      (1.9%)   -0.5% (  -4% -    3%) 0.433
           HighTermDayOfYearSort     2991.77      (4.0%)     2978.80      (4.7%)   -0.4% (  -8% -    8%) 0.754
                      TermDTSort      646.59      (2.4%)      643.93      (3.2%)   -0.4% (  -5% -    5%) 0.645
              Or2Terms2StopWords      422.30      (3.2%)      420.68      (2.3%)   -0.4% (  -5% -    5%) 0.663
                      OrHighHigh       74.97      (3.1%)       74.70      (2.2%)   -0.4% (  -5% -    5%) 0.661
               HighTermMonthSort     1987.11      (2.0%)     1981.26      (2.4%)   -0.3% (  -4% -    4%) 0.673
                 CountAndHighMed      169.81      (1.6%)      169.32      (1.6%)   -0.3% (  -3% -    2%) 0.567
               HighTermTitleSort      205.08      (2.9%)      204.55      (3.1%)   -0.3% (  -6% -    6%) 0.788
                       OrHighLow     1389.51      (3.7%)     1386.35      (4.2%)   -0.2% (  -7% -    8%) 0.857
                       OrHighMed      199.35      (3.2%)      198.92      (2.4%)   -0.2% (  -5% -    5%) 0.807
             And2Terms2StopWords      389.56      (4.1%)      388.90      (3.9%)   -0.2% (  -7% -    8%) 0.893
                         Prefix3      537.90      (2.6%)      537.19      (1.9%)   -0.1% (  -4% -    4%) 0.856
                      HighPhrase       78.58      (3.8%)       78.56      (2.9%)   -0.0% (  -6% -    6%) 0.986
                      AndHighLow     1526.88      (3.6%)     1527.75      (4.2%)    0.1% (  -7% -    8%) 0.963
                     OrStopWords       47.00      (4.8%)       47.07      (3.6%)    0.2% (  -7% -    8%) 0.904
                       And3Terms      274.37      (4.3%)      274.95      (3.4%)    0.2% (  -7% -    8%) 0.863
                        HighTerm      894.11      (6.6%)      897.19      (9.1%)    0.3% ( -14% -   17%) 0.890
                        Or3Terms      218.85      (4.2%)      219.80      (3.1%)    0.4% (  -6% -    8%) 0.712
                 MedSloppyPhrase      115.99      (3.0%)      116.54      (2.4%)    0.5% (  -4% -    5%) 0.580
                         MedTerm      902.19      (5.9%)      906.44      (8.7%)    0.5% ( -13% -   15%) 0.841
                    AndStopWords       40.54      (4.0%)       40.79      (2.7%)    0.6% (  -5% -    7%) 0.566
                     AndHighHigh       95.06      (3.5%)       95.75      (3.0%)    0.7% (  -5% -    7%) 0.483
                      AndHighMed      216.82      (3.6%)      218.42      (3.2%)    0.7% (  -5% -    7%) 0.488
                       LowPhrase       39.04      (2.7%)       39.34      (2.1%)    0.8% (  -3% -    5%) 0.318
                        Wildcard      100.42      (4.4%)      101.19      (3.9%)    0.8% (  -7% -    9%) 0.562
                       MedPhrase       52.82      (3.2%)       53.31      (2.6%)    0.9% (  -4% -    6%) 0.324
                 LowSloppyPhrase       70.20      (2.6%)       70.85      (2.2%)    0.9% (  -3% -    5%) 0.218
                HighSloppyPhrase        7.10      (2.3%)        7.18      (2.2%)    1.1% (  -3% -    5%) 0.114
                    OrNotHighLow     2101.21      (3.9%)     2129.58      (3.7%)    1.3% (  -5% -    9%) 0.257
                      OrHighRare      523.20     (13.1%)      533.40     (10.0%)    1.9% ( -18% -   28%) 0.597
                   OrNotHighHigh      268.31      (3.9%)      276.60      (4.8%)    3.1% (  -5% -   12%) 0.025
                    OrNotHighMed      513.16      (3.3%)      529.63      (4.3%)    3.2% (  -4% -   11%) 0.008
                   OrHighNotHigh      449.82      (4.1%)      466.37      (5.8%)    3.7% (  -6% -   14%) 0.021
                    OrHighNotMed      596.77      (5.4%)      624.91      (8.3%)    4.7% (  -8% -   19%) 0.033
                    OrHighNotLow      664.22      (6.6%)      698.03      (9.1%)    5.1% (  -9% -   22%) 0.044
                  CountOrHighMed      165.48      (4.7%)      175.63      (3.2%)    6.1% (  -1% -   14%) 0.000
                 CountOrHighHigh      125.70      (5.3%)      134.91      (3.4%)    7.3% (  -1% -   16%) 0.000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jpountz