Cross-modal Search for E-commerce: Building and Scaling a Cross-Modal Image Retrieval App

Read blog post about the project.

1. Quickstart

Register or login to Anyscale

If you don't have an Anyscale account, you can register here.

If you already have an account, login here.

Prepare Anyscale Workspace

Step 1

Once you are logged in, go to workspaces by clicking the "Workspaces" tab on the left side of the home screen:

Step 2

Create a new workspace by clicking the "Create Workspace" button:

Step 3

Specify Workspace details as displayed below and click "Create":

Step 4

Wait for the workspace to be created:

(this can take up to one minute)

Note

Your workspace is ready!

Step 5

Open the terminal in the workspace and clone the repository by running the following command:

git clone https://github.com/anyscale/cross-modal-search-ecommerce-project.git

Step 6

Go to the project directory:

cd cross-modal-search-ecommerce-project

2. Set up Pinecone

You'll need Pinecone index. You can register or login using this link.

Step 1

Set the Pinecone API key as an environment variable under the Dependencies section of the workspace

Step 2

Set the Pinecone API key in the embeddings/job.yaml file

name: calc-embeddings
entrypoint: python embeddings/main.py --mode img --model-name openai  #choose model from [openai, fashionclip], choose mode from [img, txt]
runtime_env:
  working_dir: .
  pip: embeddings/requirements.txt
  env_vars:
    PINECONE_API_KEY: <your pinecone api key> # set your pinecone api key here

3. Run an Anyscale Job to generate embeddings

Step 1

Submit the job to generate embeddings by running the following command in your workspace terminal:

make anyscale-job-embeddings

Step 2

Update the mode and model name in the embeddings/job.yaml file to generate embeddings for img and txt modes and openai and fashionclip models. For each mode and model combination, submit a new job by running the following command.

It should be a total of 4 jobs that need to be submitted.

Step 3

Check the status of each job by visiting the Anyscale Job interface

4. Deploy an Anyscale Service to serve the application

Step 1

Deploy the application by running the following command

make deploy-app

Step 2

Check the status of the deployment by visiting the

Step 3

Visit the Application URL to see the application. You can find the URL under the service "Query" dropdown.

Step 4

Query the application by entering a query in the search bar and clicking the search button

---

Created with ♥️ by Anyscale and deepsense.ai