Skip to content

An open-source dataset for e-commerce user intent detection and embedding retrieval

Notifications You must be signed in to change notification settings

jdcomsearch/jd-pretrain-data

Repository files navigation

jd-pretrain-data

jd-pretrain-data is an open-source dataset for user intent detection and embedding retrieval which is collected from search system of JD.com. Due to business information confidentiality, we only show encoded id of search query, item title and product category id. The project contains 5 tab-separated values (TSV) files, each of which has the following headers:

  • pretrain_data.tsv: item title \t product category
  • finetune_data_uid.tsv: query \t aggregated product categories
  • finetune_data_er.tsv: query \t item title
  • eval_overall.tsv: query \t aggregated product categories \t aggregated items' title
  • eval_tail.tsv: same as eval_overall.tsv

User intent detection and embedding retrieval tasks share the same pre-train and evaluation datasets, but use different finetune datasets. Concretely, finetune_data_uid.tsv is the finetune dataset for user intent detection task while finetune_data_er.tsv is for embedding retrieval task. eval_overall.tsv and eval_tail.tsv stand for evaluation dataset on overall and long-tail query respectively.

For more details, feel free to check out our CIKM 2022 paper: https://arxiv.org/pdf/2208.06150.pdf

About

An open-source dataset for e-commerce user intent detection and embedding retrieval

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published