Skip to content

Commit

Permalink
Adding a getter to FeatureField to allow value access (apache#13111)
Browse files Browse the repository at this point in the history
Getting the access of a FeatureField#value is useful for deduplicating. If you have a sparse vector model and you want to handle multiple inputs from them, you want flexibility in how you de-duplicate the feature dimensions.
  • Loading branch information
benwtrent committed Feb 17, 2024
1 parent d09d6de commit 0ad98cc
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions lucene/core/src/java/org/apache/lucene/document/FeatureField.java
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,16 @@ public TokenStream tokenStream(Analyzer analyzer, TokenStream reuse) {
return stream;
}

/**
* This is useful if you have multiple features sharing a name and you want to take action to
* deduplicate them.
*
* @return the feature value of this field.
*/
public float getFeatureValue() {
return featureValue;
}

private static final class FeatureTokenStream extends TokenStream {
private final CharTermAttribute termAttribute = addAttribute(CharTermAttribute.class);
private final TermFrequencyAttribute freqAttribute = addAttribute(TermFrequencyAttribute.class);
Expand Down

0 comments on commit 0ad98cc

Please sign in to comment.