Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add more components to the search indexing #541

Merged
merged 38 commits into from
Oct 25, 2024
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
a7cb1ba
fix: add more components to the search indexing
Revathyvenugopal162 Oct 10, 2024
b6b1abb
chore: adding changelog file 541.fixed.md [dependabot-skip]
pyansys-ci-bot Oct 10, 2024
488246e
fix: add more elements
Revathyvenugopal162 Oct 10, 2024
8dbd6c8
Merge branch 'fix/indexing-docs-components' of https://github.com/ans…
Revathyvenugopal162 Oct 10, 2024
c56bba2
Merge branch 'main' into fix/indexing-docs-components
Revathyvenugopal162 Oct 11, 2024
678652b
Merge branch 'main' into fix/indexing-docs-components
Revathyvenugopal162 Oct 11, 2024
99de523
fix: add block list
Revathyvenugopal162 Oct 14, 2024
26c7e03
fix: initial test with group patterns
Revathyvenugopal162 Oct 15, 2024
b012dc8
Merge branch 'main' into fix/indexing-docs-components
Revathyvenugopal162 Oct 16, 2024
2674187
fix: group docs and pattern
Revathyvenugopal162 Oct 16, 2024
7473d88
fix: add nodes
Revathyvenugopal162 Oct 16, 2024
ee4be55
fix: add nodes as seral objects
Revathyvenugopal162 Oct 16, 2024
cf82f5c
fix: add config options
Revathyvenugopal162 Oct 17, 2024
2a2d382
fix: cleanup the codes
Revathyvenugopal162 Oct 17, 2024
908212f
Apply suggestions from code review
Revathyvenugopal162 Oct 17, 2024
95ae207
fix: add all init
Revathyvenugopal162 Oct 17, 2024
adcbb8a
fix: config values
Revathyvenugopal162 Oct 17, 2024
699c47e
Merge branch 'main' into fix/indexing-docs-components
Revathyvenugopal162 Oct 22, 2024
cf9e078
chore: adding changelog file 541.miscellaneous.md [dependabot-skip]
pyansys-ci-bot Oct 22, 2024
d3984a5
fix: update the title name
Revathyvenugopal162 Oct 22, 2024
59dc7a9
fix: update the default minimum patterns
Revathyvenugopal162 Oct 22, 2024
3040dac
Update doc/source/conf.py
Revathyvenugopal162 Oct 22, 2024
cf2b16e
Merge branch 'main' into fix/indexing-docs-components
jorgepiloto Oct 23, 2024
f3b59bd
fix: add desc nodes
Revathyvenugopal162 Oct 24, 2024
7831aba
Merge branch 'fix/indexing-docs-components' of https://github.com/ans…
Revathyvenugopal162 Oct 24, 2024
33f18a0
fix: add desc nodes with achor id
Revathyvenugopal162 Oct 24, 2024
571dfba
fix: add proper title
Revathyvenugopal162 Oct 24, 2024
580b36e
Merge branch 'main' into fix/indexing-docs-components
Revathyvenugopal162 Oct 24, 2024
327dc74
fix: ignore changelog from linkchcek
Revathyvenugopal162 Oct 24, 2024
739fb02
Merge branch 'fix/indexing-docs-components' of https://github.com/ans…
Revathyvenugopal162 Oct 24, 2024
89081f3
fix: anchor to title
Revathyvenugopal162 Oct 24, 2024
eccbb03
Merge branch 'main' into fix/indexing-docs-components
Revathyvenugopal162 Oct 24, 2024
7042a22
docs: add documentation for index patterns
Revathyvenugopal162 Oct 24, 2024
8ed2866
Merge branch 'fix/indexing-docs-components' of https://github.com/ans…
Revathyvenugopal162 Oct 24, 2024
7b8be92
chore: adding changelog file 541.documentation.md [dependabot-skip]
pyansys-ci-bot Oct 24, 2024
0b7f055
Apply suggestions from code review
Revathyvenugopal162 Oct 25, 2024
0b19dab
docs: remove linkchcek
Revathyvenugopal162 Oct 25, 2024
290dcc0
Apply suggestions from code review
Revathyvenugopal162 Oct 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/changelog.d/541.documentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
fix: add more components to the search indexing
11 changes: 11 additions & 0 deletions doc/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@
from sphinx.builders.latex import LaTeXBuilder

from ansys_sphinx_theme import (
ALL_NODES,
PARAGRAPHS,
TITLES,
__version__,
ansys_favicon,
ansys_logo_white,
Expand Down Expand Up @@ -62,9 +65,15 @@
"threshold": 0.2,
"limit": 7,
"minMatchCharLength": 3,
"ignoreLocation": True,
Revathyvenugopal162 marked this conversation as resolved.
Show resolved Hide resolved
},
}

index_patterns = {
"examples/api/": ALL_NODES,
"examples/sphinx_examples/": TITLES + PARAGRAPHS,
}


# Sphinx extensions
extensions = [
Expand Down Expand Up @@ -133,6 +142,8 @@
rst_epilog += f.read()


# linkcheck_exclude_documents = ["changelog"]

Revathyvenugopal162 marked this conversation as resolved.
Show resolved Hide resolved
linkcheck_ignore = [
r"https://sphinxdocs.ansys.com/version/*",
]
Expand Down
17 changes: 17 additions & 0 deletions doc/source/user-guide/options.rst
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,23 @@ the ``delay`` key in the ``static_search`` dictionary with a value specifying
the amount of milliseconds to wait before executing the search. A value of
``0`` disables the debounce function.

To customise the indexing of your documentation, you can use the ``index_patterns`` dictionary in the ``conf.py`` file.
This dictionary contains the paths to the directories you want to index and the type of nodes to index.
The type of nodes can be ``ALL_NODES``, ``TITLES`` or ``PARAGRAPHS``.
The default value is ``TITLES + PARAGRAPHS``.

Here is an example of how to add the ``index_patterns`` dictionary to the `conf.py`` file:

.. code-block:: python

index_patterns = {
"api/": ALL_NODES,
"examples/": TITLES + PARAGRAPHS,
}

The above example indexes all nodes in the ``api/`` directory and only titles and paragraphs in the ``examples/`` directory.


.. note::

All other options are available in the `Fuse.js documentation <https://fusejs.io/api/options.html>`_.
Expand Down
21 changes: 15 additions & 6 deletions src/ansys_sphinx_theme/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,19 @@
import subprocess
from typing import Any, Dict

from docutils.nodes import document
from docutils import nodes
from sphinx import addnodes
from sphinx.application import Sphinx

from ansys_sphinx_theme.extension.linkcode import DOMAIN_KEYS, sphinx_linkcode_resolve
from ansys_sphinx_theme.latex import generate_404 # noqa: F401
from ansys_sphinx_theme.search import create_search_index, update_search_config
from ansys_sphinx_theme.latex import generate_404
from ansys_sphinx_theme.search import (
ALL_NODES,
PARAGRAPHS,
TITLES,
create_search_index,
update_search_config,
)

try:
import importlib.metadata as importlib_metadata
Expand Down Expand Up @@ -170,7 +176,7 @@ def setup_default_html_theme_options(app):


def fix_edit_html_page_context(
app: Sphinx, pagename: str, templatename: str, context: dict, doctree: document
app: Sphinx, pagename: str, templatename: str, context: dict, doctree: nodes.document
) -> None:
"""Add a function that Jinja can access for returning an "edit this page" link .

Expand Down Expand Up @@ -271,7 +277,7 @@ def fix_edit_link_page(link: str) -> str:


def update_footer_theme(
app: Sphinx, pagename: str, templatename: str, context: Dict[str, Any], doctree: document
app: Sphinx, pagename: str, templatename: str, context: Dict[str, Any], doctree: nodes.document
) -> None:
"""Update the version number of the Ansys Sphinx theme in the footer.

Expand Down Expand Up @@ -401,7 +407,7 @@ def convert_pdf_to_png(pdf_path: pathlib.Path, output_dir: pathlib.Path, output_


def add_cheat_sheet(
app: Sphinx, pagename: str, templatename: str, context: Dict[str, Any], doctree: document
app: Sphinx, pagename: str, templatename: str, context: Dict[str, Any], doctree: nodes.document
) -> None:
"""Add a cheat sheet to the left navigation sidebar.

Expand Down Expand Up @@ -578,3 +584,6 @@ def setup(app: Sphinx) -> Dict:
"parallel_read_safe": True,
"parallel_write_safe": True,
}


__all__ = ["__version__", "generate_404", "get_version_match", "TITLEs", "PARAGRAPHS", "ALL_NODES"]
18 changes: 16 additions & 2 deletions src/ansys_sphinx_theme/search/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,13 @@

from sphinx.application import Sphinx

from ansys_sphinx_theme.search.fuse_search import create_search_index
from ansys_sphinx_theme.search.fuse_search import (
ALL_NODES,
LITERAL,
PARAGRAPHS,
TITLES,
create_search_index,
)


def update_search_config(app: Sphinx) -> None:
Expand All @@ -38,7 +44,15 @@ def update_search_config(app: Sphinx) -> None:
theme_static_options["keys"] = ["title", "text"]
theme_static_options["threshold"] = theme_static_options.get("threshold", 0.2)
theme_static_options["limit"] = theme_static_options.get("limit", 10)
app.add_config_value("index_patterns", {}, "html")
Revathyvenugopal162 marked this conversation as resolved.
Show resolved Hide resolved
app.config.html_theme_options["static_search"] = theme_static_options


__all__ = ["create_search_index", "update_search_config"]
__all__ = [
"create_search_index",
"update_search_config",
"LITERAL",
"PARAGRAPHS",
"TITLES",
"ALL_NODES",
]
74 changes: 68 additions & 6 deletions src/ansys_sphinx_theme/search/fuse_search.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,19 @@
import re

from docutils import nodes
from docutils.nodes import Element

PARAGRAPHS = [nodes.paragraph]
TITLES = [nodes.title]
LITERAL = [nodes.literal]
ALL_NODES = [nodes.Text]
DEFAULT_PATTERN = PARAGRAPHS + TITLES + LITERAL


class SearchIndex:
"""Generate a search index for a Sphinx document."""

def __init__(self, doc_name, app):
def __init__(self, doc_name, app, pattern=None):
"""
Initialize the search index object.

Expand All @@ -46,19 +53,31 @@ def __init__(self, doc_name, app):
self.doc_name = doc_name
self.doc_path = f"{self.doc_name}.html"
self.env = app.env
self.theme_options = app.config.html_theme_options.get("static_search", {})
Revathyvenugopal162 marked this conversation as resolved.
Show resolved Hide resolved
self.doc_title = self.env.titles[self.doc_name].astext()
self.doc_tree = self.env.get_doctree(self.doc_name)
self.sections = []
self.pattern = pattern

def build_sections(self):
"""Build sections from the document tree."""
"""Build sections from the document tree, handling subsections and descriptions."""
for node in self.doc_tree.traverse(nodes.section):
subsections = list(node.traverse(nodes.section))

if len(subsections) > 1:
# get only the first section
main_section = subsections[0]
# remove subsections from the main section
for subsection in main_section.traverse(nodes.section):
subsection.parent.remove(subsection)
node = main_section

section_title = node[0].astext()

section_text = "\n".join(
n.astext()
for node_type in [nodes.paragraph, nodes.literal_block]
for n in node.traverse(node_type)
n.astext() for node_type in self.pattern for n in node.traverse(node_type)
)

section_anchor_id = _title_to_anchor(section_title)
self.sections.append(
{
Expand All @@ -68,6 +87,31 @@ def build_sections(self):
}
)

self._process_desc_element(node, section_title)

def _process_desc_element(self, node, title):
"""Process `desc` element within a section."""
jorgepiloto marked this conversation as resolved.
Show resolved Hide resolved
for element in node.traverse(Element):
if element.tagname != "desc":
continue

# id is the id tag of the desc element
section_anchor_id = element.attributes["ids"]
if element.children:
for element_child in element.children:
if element_child.tagname != "desc_signature":
continue
section_anchor_id = element_child.attributes["ids"][0]
section_text = element.astext()
section_title = _desc_anchor_to_title(title, section_anchor_id)
self.sections.append(
{
"title": section_title,
"text": section_text,
"anchor_id": section_anchor_id,
jorgepiloto marked this conversation as resolved.
Show resolved Hide resolved
}
)

def generate_breadcrumbs(self, section_title: str) -> str:
"""
Generate title breadcrumbs from the document structure.
Expand Down Expand Up @@ -122,11 +166,28 @@ def indices(self):
}


def _desc_anchor_to_title(title, anchor):
"""Convert a desc anchor to a title."""
anchor_title = anchor.split(".")[-1]
return f"{title} > {anchor_title}"


def _title_to_anchor(title: str) -> str:
"""Convert a title to a URL-friendly anchor identifier."""
return re.sub(r"[^\w\s-]", "", title.lower().strip().replace(" ", "-"))


def get_pattern_for_each_page(app, doc_name):
"""Get the pattern for each page in the search index."""
patterns = app.env.config.index_patterns or {}
Revathyvenugopal162 marked this conversation as resolved.
Show resolved Hide resolved

for filename, pattern in patterns.items():
if doc_name.startswith(filename):
return pattern

return DEFAULT_PATTERN


def create_search_index(app, exception):
"""
Generate search index at the end of the Sphinx build process.
Expand All @@ -144,7 +205,8 @@ def create_search_index(app, exception):
search_index_list = []

for document in app.env.found_docs:
search_index = SearchIndex(document, app)
pattern = get_pattern_for_each_page(app, document)
search_index = SearchIndex(document, app, pattern)
search_index.build_sections()
search_index_list.extend(search_index.indices)

Expand Down
Loading