Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Advanced Queries & Boolean Retrieval #125

Draft
wants to merge 19 commits into
base: dev
Choose a base branch
from

Conversation

ppanopticon
Copy link
Member

@ppanopticon ppanopticon commented Nov 27, 2024

Description

The aim of this PR is to refactor queries in general and BooleanQuery in particular with eyes on future changes to the query pipeline, such as, late filtering, filters combined with fulltext or NNS queries and predicate pushdown.

It incorporates the following changes:

  • There is now a clear separation between a Query and a Predicate, the latter of which corresponds to what was considered a Query before the PR. The Query object holds additional information such as limits.
  • The new BooleanPredicate now comes in different flavours: Comparison, And, Or and Not
  • Comparison operators now come in different flavours as well, which correspond to the basic comparison operations provided by most DBMS. This also includes a new Comparison.In predicate.
  • The ProximityPredicate and the SimpleFulltextPredicate now comes with an optional BooleanPredicate filter, with the idea, to execute queries with boundary conditions.

The PostgreSQL database backend has been retro-fitted to support these changes. With this PR, the database layer becomes more powerful. It should be able to:

  • Fetch Descriptor of field A based on a BooleanPredicate that involves other fields.
  • Fetch Retrievable with Descriptor of field A based on a BooleanPredicate that involves other fields.
  • Execute ProximityQuery and FulltextQuery with boundary conditions that involves other fields.

The PR is currently in draft and open for discussion.

ppanopticon and others added 19 commits November 27, 2024 08:31
- There is now a difference between a Query and a Predicate
- SimpleBooleanQuery is now called Comparison (a predicate) and exists in different flavours (with support for IN queries)
- Adds construct for logical connections (AND, OR, NOT).

Signed-off-by: Ralph Gasser <rg@pontius.ch>
…constants.

Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
's idea.

Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
…eader and ScalarDescriptorReader.

Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
Signed-off-by: Ralph Gasser <rg@pontius.ch>
…eature/advanced-boolean

# Conflicts:
#	vitrivr-engine-module-pgvector/src/main/kotlin/org/vitrivr/engine/database/pgvector/descriptor/AbstractDescriptorReader.kt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant