Elastic search global setup requirements for Event Listing Page

[pghorpade]

Description

This Issue documents the requirements for setting up and configuring FTVA search index for Elastic Search.

Index Creation

What is the scope?

• This index will be used for both Site Search and Event Listing Page filtering functionality

How is the Index created?

• We are using Nuxt modules within the FTVA Nuxt repo to create a temporary index for the build and then swapping the index in the alias
• Prerequisite for this is an Alias for FTVA is created using Kibana devtools.

Are we using an Alias like a Library Website?

• Yes

What is Alias name for FTVA for local setup?

• It is apps-dev-parinita-current-ftv-website

What work is completed?

• There is Nuxt module named init.js that creates the temporary index every time a build is executed like pnpm run generate
• after the build is complete, there is another Nuxt module which swaps the index in the alias

What work is remaining?

• Write a ticket to delete the old temporary index which we are swapping out from the alias.
• There is a ticket to add similar functionality to the library website.
• Use Elastic search Rest API to delete the old temp index after removing it from the alias. We need to first fetch the alias with the get request copy the old temp index name to a constant and then perform the delete operation using the constant. APPS-2938

Indexing FTVA Events Content

This is functionality where we will be indexing craft content from all Nuxt page templates including the listing pages and home page for site search and to filter events

Tasks:

1. In the library website we created a nuxt plugin to index craft data to Elastic Search. For FTVA and going forward for other nuxt website we should use composables. So create a composable named useContentIndexer. In this composable instead of checking process.env.NODE_ENV !== 'development' use import.meta.prerender === true
2. Call the composable useContentIndexer on all slug pages built so far for FTVA. Add the call after error checks before creating data refs like

// This will work only on the Event detail slug
if (data.value.ftvaEvent && import.meta.prerender) {
  try {
     // Call the composable to use the indexing function
    const { indexContent } = useContentIndexer()
    // Index the event data using the composable during static build
    await indexContent(data.value.ftvaEvent, route.params.slug)
    console.log('Event indexed successfully during static build')
  } catch (error) {
    console.error('Failed to index event during static build:', error)
  }
}

ES Aggregations

what is the difference between query and aggregation?

Queries - retrieve documents that match the specified criteria.
Aggregation - This presents the summary of your data as metrics, statistics, and other analytics.

For FTVA we need to show some event filters called taglabels in a dropdown component to filter events in the list.

What fields do we need to aggregate for the filters on the event listing page?

1. Event Type
2. Screening Type
3. File Format

what is a sample query to fetch the taglabels aggregated data from ES?

Syntax:

GET Enter_name_of_the_index_here/_search
{
  "aggs": {
    "Name your aggregations here": {
      "Specify the aggregation type here": {
        "field": "Name the field you want to aggregate on here"
      }
    }
  }
}

GET apps-craft-staff-index/_search
{
  "size": 0, // do not return any documents only aggs data
  "aggregations": {
    "Event Type": {
      "terms": {
        "field": "tagLabels.title.keyword",
        "size": 100
      }
    }
  }
}

ES Filtering

This is functionality where we will be setting up ES filter queries used on the Event listing page.

What are we filtering by?

• startDate from ftvaEvent section handle
• tagLabels [which are ftvaEventFilters in craft] from the ftvaEvent section handle

What kind of ES query type is good for filtering content?

• Here's how you can structure the query:

1. startDate: You will use a range query to filter documents by a specific date range (e.g., documents with dates greater than or equal to today's date).
2. tagLabels Term: Since you can have more than one category selected, use a terms query. The terms query allows you to filter documents matching any specified categories.

What Nuxt composable do we need?

• useIndexFilter()

What other work do we need to do?

1. Sorting or ordering the search results
2. Are we returning all fields as search results from the indexed documents?

GET /_search
{
  "query": { 
    "bool": { 
      "filter": [ 
        { "term":  { "sectionHandle.keyword": "ftvaEvent" }},
        { "term":  { "tagLabels.keyword": ["guest speaker","other"] }},
        { "range": { "startDate": { "gte": "2015-01-01" }}}
      ]
    }
  }
}

Exact Date Match

• When using exact date filters in Elasticsearch, it's essential to ensure that the date format in your query matches the format of the date stored in your index. Elasticsearch uses ISO 8601 as the default date format (yyyy-MM-dd for dates, and yyyy-MM-dd'T'HH:mm:ssZ for dates with time).

{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "startDate": "2024-10-17"
          }
        }
      ]
    }
  }
}

Another example of Date Range can be Double Bound Ranges

{
  "range": {
    "event_date": {
      "gte": "2024-01-01",
      "lte": "2024-12-31"
    }
  }
}

Exclusive Bounds

{
  "range": {
    "event_date": {
      "gt": "2024-10-01"
    }
  }
}

All dates up to today

{
  "range": {
    "event_date": {
      "lte": "now"
    }
  }
}

Custom Date Formats

{
  "range": {
    "event_date": {
      "gte": "01/01/2024",
      "lte": "31/12/2024",
      "format": "dd/MM/yyyy"
    }
  }
}

Name and description of any slots needed.

ES Site Search query

Site search is a full-text search that searches everything that is indexed for all content types in Craft for FTVA.
There are many kinds of search queries in ES.

FTVA site search will cover the following features (Need to discuss further with UX to cover all conditions)

1. search across a set of fields
2. the search fields can be boosted
3. enable fuzzy search
4. Do we need query_string or multi_match
5. Do we need a bool query to have must, filter, and should clauses?
6. do we need highlighting?
7. what fields to fetch in the returned results?
8. There will be a Nuxt composable useSiteSearch which exports a function that has this query setup

Page ES Logic:

Questions?

1. Do we need searchconfig?

• Maybe not as filters are only used on the Event listing page, we can hard code es field names as constants in the page itself.

Tasks:

1. Filter Component and Page interactions:

• On the mounted vue lifecycle hook, we will call the composable which performs the aggregation query and returns the parsed data to filter components like DatePicker and filter component for Event Tags.
• When the user selects dates from the date picker and clicks apply or done or selects filters from drop down the components will emit events and data to the event handler on the page, which calls the router.push like we do on the website to update the URL
• Next, after the page reloads, the page will have a watcher for filter query params, which will call a method on the page that will after checking some criteria and assembling some data needed for the query call the composable like useIndexFilters to perform the search
• After the search is performed in the composable and it returns results, the calling function will update data references for the page which will update the page template due to vue reactivity.

Developer Tips

List any developer tips here

1. Explore Elastic Search Query syntax.
2. Beginner's Crash Course to Elastic Stack
3. Tutorial
4. https://www.elastic.co/blog/a-practical-introduction-to-elasticsearch
5. Official Elasticsearch Documentation