Compute an ngram field for all street names #364
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds one new field called
address_parts.street.ngram
It takes advantage of the
fields
mapping to generate ngrams for street names.The ngram field is tokenized using a new tokenizers called
peliasIndexStreetOneEdgeGram
which is mostly the same as thepeliasIndexOneEdgeGram
analyzer except using different synonyms files, so as to produce prefix-ngrams which can be used for autocomplete.The motivation here is to be able to, quite simply and efficiently, improve autocomplete queries which contain street names.
We currently do autocomplete on the
name.default
field, which mixes names and addresses, using this field with amulti_match
will have benefits over the single-field approach:multi_match
queryI suspect the changes required for the queries will be minimal.
Its likely that the index size will be increased after merging this PR because the street names will be indexed using an edge ngram filter twice, once for
name.default
and once forstreet.ngram
.The plan is, following this PR to remove the street names from the
name.default
field, once this has been done the index will return to the previous size (or a very similar on-disk size).The analysis that I've configured in this PR is likely not perfect, but it mirrors what we already have, so the integration will be easier.
Once we've merge this PR and switched the queries we will be much freer to improve individual fields and analysis.
related: #347
related: #359