Skip to content

Commit

Permalink
Update FTS with prefix search
Browse files Browse the repository at this point in the history
  • Loading branch information
cbullinger committed Sep 6, 2023
1 parent b0b7e5f commit dbbfbe3
Show file tree
Hide file tree
Showing 5 changed files with 97 additions and 78 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -140,50 +140,59 @@ class ReadTest: RealmTest() {
@Test
fun fullTextSearch () {
// :snippet-start: kotlin-fts-property
class Book() : RealmObject {
class Book : RealmObject {
@PrimaryKey
var _id: ObjectId = ObjectId() // Primary key

var name: String = ""

@FullText // Marks the property with FTS
var genre: String = ""
}
// :snippet-end:

val config = RealmConfiguration.Builder(
schema = setOf(Book::class) // Pass the defined class as the object schema
)
.directory("/tmp/") // default location for jvm is... in the project root
val config = RealmConfiguration.Builder(setOf(Book::class))
.inMemory()
.build()
val realmFts = Realm.open(config)
Log.v("Successfully opened realm: ${realmFts.configuration.name}")

realmFts.writeBlocking {
copyToRealm(Book().apply {
_id = ObjectId()
name = "Peaches"
genre = "fiction"
})
}

// :snippet-start: kotlin-fts-query
// Find all the books with science fiction as the genre
var scienceFiction = realmFts.query<Book>("genre TEXT $0", "science fiction").find()
val realm = Realm.open(config)
Log.v("Successfully opened realm: ${realm.configuration.name}")

// Find all the books with fiction but not science in the genre
var fictionNotScience = realmFts.query<Book>("genre TEXT $0", "fiction -science").find()
// :snippet-end:
runBlocking {
realm.writeBlocking {
copyToRealm(Book().apply {
genre = "sci fi"
})
copyToRealm(Book().apply {
genre = "fiction"
})
copyToRealm(Book().apply {
genre = "science fiction"
})
copyToRealm(Book().apply {
genre = "sci fi fantasy"
}) }

// :snippet-start: kotlin-fts-query
// Find all books with "science fiction" as the genre
val scienceFiction =
realm.query<Book>("genre TEXT $0", "science fiction").find()

// Find all books with "fiction" but not "science" in the genre
val fictionNotScience =
realm.query<Book>("genre TEXT $0", "fiction -science").find()

// Find all books with "sci-" and "fi-" prefixes in the genre
val sciFi =
realm.query<Book>("genre TEXT $0", "sci* fi*").find()
// :snippet-end:

assertEquals(0, scienceFiction.count())
assertEquals(1, fictionNotScience.count())
assertEquals(1, scienceFiction.size) // "science fiction"
assertEquals(1, fictionNotScience.size) // "fiction"
assertEquals(3, sciFi.size) // "sci fi", "science fiction", "sci fi fantasy"

realmFts.writeBlocking {
// fetch all frogs from the realm
val books: RealmResults<Book> = this.query<Book>().find()
// call delete on the results of a query to delete those objects permanently
delete(books)
realm.writeBlocking {
val books = query<Book>().find()
delete(books)
}
}
realmFts.close()
realm.close()
}
}
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
class Book() : RealmObject {
class Book : RealmObject {
@PrimaryKey
var _id: ObjectId = ObjectId() // Primary key

var name: String = ""

@FullText // Marks the property with FTS
var genre: String = ""
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
// Find all the books with science fiction as the genre
var scienceFiction = realmFts.query<Book>("genre TEXT $0", "science fiction").find()
// Find all books with "science fiction" as the genre
val scienceFiction =
realm.query<Book>("genre TEXT $0", "science fiction").find()

// Find all the books with fiction but not science in the genre
var fictionNotScience = realmFts.query<Book>("genre TEXT $0", "fiction -science").find()
// Find all books with "fiction" but not "science" in the genre
val fictionNotScience =
realm.query<Book>("genre TEXT $0", "fiction -science").find()

// Find all books with "sci-" and "fi-" prefixes in the genre
val sciFi =
realm.query<Book>("genre TEXT $0", "sci* fi*").find()
44 changes: 24 additions & 20 deletions source/sdk/kotlin/realm-database/crud/read.txt
Original file line number Diff line number Diff line change
Expand Up @@ -126,32 +126,36 @@ Filter with Full-Text Search

You can use Realm Query Language (RQL) to query on properties that
have :ref:`Full-Text Search Indexes <kotlin-fts-index>` (FTS) on them.
To query these properties, use the ``TEXT`` predicate in your query.

Exclude results for a word by placing the ``-`` character
in front of the word. For example, a search for ``fiction -science`` would
include all search results for ``fiction`` excluding those that include the
word ``science``.
To query these properties, use the ``TEXT`` predicate.

Words in the query are converted to tokens by a tokenizer using the
following rules:

- Tokens can only consist of characters from ASCII and the Latin-1
supplement (western languages). All other characters are considered whitespace.
- Words split by a hyphen (``-``), for example ``full-text``, are split into
two tokens.
- Tokens are diacritics- and case-insensitive.

You can search for an entire word or phrase or limit your results with the
following characters:

- Exclude results for a word by placing the ``-`` character
in front of the word. For example, ``fiction -science`` would
include all search results for ``fiction`` and exclude those that include the
word ``science``.
- In Kotlin SDK version 1.11.0 and later, you can specify prefixes
by placing the ``*`` character at the end of a word.
For example, ``fict*`` would include all search results for
``fiction`` and ``fictitious``. (The Kotlin SDK does *not* currently
support suffix searches.)

In the following example, we query the ``Book.genre`` field:

.. literalinclude:: /examples/generated/kotlin/ReadTest.snippet.kotlin-fts-query.kt
:language: kotlin

Full-Text Search Tokenizer Details
``````````````````````````````````

Full-Text Search (FTS) indexes support:

- Boolean match word searches, not searches for relevance.
- Tokens are diacritics- and case-insensitive.
- Tokens can only consist of characters from ASCII and the Latin-1 supplement (western languages).
- All other characters are considered whitespace. Words split by a hyphen
(``-``) like ``full-text`` are split into two tokens.

For more information on features of the FTS index, see the API reference for
the `@FullText <{+kotlin-local-prefix+}io.realm.kotlin.types.annotations/
-full-text/index.html>`__ index annotation.
Note that FTS only supports Boolean matches and not relevance-based matches.

.. _kotlin-sort-queries:

Expand Down
40 changes: 21 additions & 19 deletions source/sdk/kotlin/realm-database/schemas/property-annotations.txt
Original file line number Diff line number Diff line change
Expand Up @@ -213,33 +213,35 @@ Full-Text Search Indexes
------------------------

In addition to standard indexes, Realm also supports Full-Text Search
(FTS) indexes on ``string`` properties. While you can query a string
(FTS) indexes on ``String`` properties. While you can query a string
field with or without a standard index, an FTS index enables searching
for multiple words and phrases and excluding others.

For more information on querying full-text indexes, see
:ref:`Filter with Full Text Search <kotlin-filter-fts>`.

To create a FTS index on a property, use the `@FullText
To create a FTS index on a ``String`` property, use the `@FullText
<{+kotlin-local-prefix+}io.realm.kotlin.types.annotations
/-full-text/index.html>`__ annotation. This enables full-text queries
on the property. In the following example, we mark the ``genre``
property with the FTS annotation:

.. literalinclude:: /examples/generated/kotlin/ReadTest.snippet.kotlin-fts-property.kt
:language: kotlin
:emphasize-lines: 7-8

You cannot combine `@Index
<{+kotlin-local-prefix+}io.realm.kotlin.types.annotations
/-index/index.html>`__ and `@FullText
<{+kotlin-local-prefix+}io.realm.kotlin.types.annotations
/-full-text/index.html>`__ on the same property.

.. note:: Character Limitations for Full-Text Search Indexes

For Full-Text Search (FTS) indexes, only ASCII and Latin-1 alphanumerical
chars are included in the index (most western languages). See the `@FullText
<{+kotlin-local-prefix+}io.realm.kotlin.types.annotations/
-full-text/index.html>`__ for more information on current FTS index constraints.
:emphasize-lines: 5-6

Note the following constraints on FTS indexes:

- You can only annotate ``String`` properties with ``@FullText``.
- FTS-enabled properties are not indexable. You *cannot* combine `@Index
<{+kotlin-local-prefix+}io.realm.kotlin.types.annotations/-index/index.html>`__
and `@FullText
<{+kotlin-local-prefix+}io.realm.kotlin.types.annotations/-full-text/index.html>`__
on the same property.
- FTS only supports Boolean matches. You cannot sort results by relevance.
- Indexes are diacritics- and case-insensitive.
- Indexes can *only* consist of characters from ASCII and the Latin-1
supplement (western languages). All other characters are considered
whitespace.
- In Kotlin SDK version 1.11.0 and later, you can perform prefix searches.
However, the SDK does not yet support suffix searches.

For more information on querying full-text indexes, refer to
:ref:`Filter with Full Text Search <kotlin-filter-fts>`.

0 comments on commit dbbfbe3

Please sign in to comment.