Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FT.SEARCH gives incomplete result when used with EVAL (Silent error, potentially hard to spot, but hopefully easy to fix) #3706

Open
CodeToFreedom opened this issue Sep 13, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@CodeToFreedom
Copy link

During our migration tests to dragonflydb we came across a quite dangerous silent error which is quite hard to spot as incomplete result.
In short: The lua-script output of FT.SEARCH is cut off and therefore different from the normal FT.SEARCH output.
This could lead to potentially dangerous scenarios where users rely on the output to be correct which it is not.

Steps to reproduce:

-Create test data index:
FT.CREATE idx:testindex ON JSON PREFIX 1 testindex: SCHEMA $.pk AS pk TAG $.color AS color TAG

-Insert test data:

JSON.SET testindex:abc1 . "{\"pk\":\"abc1\",\"color\":\"blue\"}" 
JSON.SET testindex:abc2 . "{\"pk\":\"abc2\",\"color\":\"blue\"}"

Without EVAL/lua the FT.SEARCH command gives the correct result in dragonflydb + redis:

'FT.SEARCH' 'idx:testindex' '((@color:{blue}))' 'LIMIT' '0' '20'
1) "2"
2) "testindex:abc1"
3) 1) "$"
   2) "{\"color\":\"blue\",\"pk\":\"abc1\"}"
4) "testindex:abc2"
5) 1) "$"
   2) "{\"color\":\"blue\",\"pk\":\"abc2\"}"

Wrong output format and incomplete result:

The same command via EVAL/lua script is incorrect in dragonflydb though:

SCRIPT FLAGS f90904580109ee522dcbd414dd3103e5ecdccd6b disable-atomicity
EVAL "return redis.call('FT.SEARCH', 'idx:testindex', '((@color:{blue}))', 'LIMIT', '0', '20')" 0`

Wrong Result (also different output format):
"{\"color\":\"blue\",\"pk\":\"abc2\"}"

Expected result (equal to redis and regular df FT.SEARCH command):

The lua script result should be the exact same output that the FT.SEARCH command above gives.
Here the correct output of redis instance:

EVAL "return redis.call('FT.SEARCH', 'idx:testindex', '((@color:{blue}))', 'LIMIT', '0', '20')" 0
1) "2"
2) "testindex:abc1"
3) 1) "$"
   2) "{\"pk\":\"abc1\",\"color\":\"blue\"}"
4) "testindex:abc2"
5) 1) "$"
   2) "{\"pk\":\"abc2\",\"color\":\"blue\"}"

Dragonfly version used: Most recent version [v1.22.1]

Please let me know in case something is unclear and I try to help as good as possible.
Many thanks for your efforts.

@CodeToFreedom CodeToFreedom added the bug Something isn't working label Sep 13, 2024
@CodeToFreedom CodeToFreedom changed the title FT.SEARCH gives incomplete result when used with EVAL (Silent error + potentially dangerous) FT.SEARCH gives incomplete result when used with EVAL (Silent error, potentially hard to spot, but hopefully easy to fix) Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants