How to enable FlexSearch "simple" encoder ?

[mpfj]

I'm creating a test site for some documentation.
FlexSearch picks up most items but it's not as "flexible" as I was imagining.

For instance, you can search for "docu" and it will list all instances of (eg) "document".
But if I search for "ment", nothing is listed.
So it seems to only match the beginning of words ... is that correct ?

Also, we have some words that include unicode characters (eg NÅNO).
When I search for just "nano", nothing is listed.
However, on the FlexSearch website, it mentions encoders which would appear to allow better matching for these types of words (see the example here).
Is this supported in hextra ?

Alternatively, is there a way to add "keywords" to index pages so that we could add extra hidden search terms ?

Cheers

[imfing]

Thanks for the detailed feedback!

For instance, you can search for "docu" and it will list all instances of (eg) "document".
But if I search for "ment", nothing is listed.
So it seems to only match the beginning of words ... is that correct ?

that is correct, the tokenization is set to forward which takes less memory footprint

hextra/assets/js/flexsearch.js

Line 176 in 5c7303b

tokenize: 'forward',

https://github.com/nextapps-de/flexsearch#tokenizer-prefix-search

it is not configurable yet, feel free to open an feature request

However, on the FlexSearch website, it mentions encoders which would appear to allow better matching for these types of words (see the example here).
Is this supported in hextra ?

I don't remember the encoders has been enabled by default so I guess it's also something we could potentially support via a config field in hugo.

Alternatively, is there a way to add "keywords" to index pages so that we could add extra hidden search terms ?

not yet, currently it only indexes the content and title, it can be added potentially, but may require users to explicitly set keywords in frontmatter.