Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Crash when a config value is not a string #6

Open
VictorGoubet opened this issue Oct 17, 2024 · 1 comment
Open

bug: Crash when a config value is not a string #6

VictorGoubet opened this issue Oct 17, 2024 · 1 comment
Assignees
Labels
unconfirmed This bug was not reproduced yet

Comments

@VictorGoubet
Copy link

VictorGoubet commented Oct 17, 2024

Description of the bug

In the jinja template, in the config section we iterate throught each key value of the config griffe object and we highlight the value. However the highlight method take a string as input and it will fail if another kind of value is took as input.

Here is the targeted peace of code:

  {% block config scoped %}
    {% if class.extra.griffe_pydantic.config %}
      <p>Config:</p>
      <ul>
        {% for name, value in class.extra.griffe_pydantic.config.items() %}
          <li><code>{{ name }}</code>: {{ value|highlight(language="python", inline=True) }}</li>
        {% endfor %}
      </ul>
    {% endif %}
  {% endblock config %}

Now comes the issue, if you are using pydantic > 2.0 you will probably use the new way to define the config with model_dict being a ConfigDict. When you switch from the old config definition (with class Config) to the new here is the change in the data.extra.griffe_pydantic:

With class Config definition:

data.extra["griffe_pydantic"] ={
    'fields': functools.partial(<function _model_fields at 0x7e6ee10f9bd0>, Class('TermObject', 6, 43)), 
    'validators': functools.partial(<function _model_validators at 0x7e6ee10f9c60>, Class('TermObject', 6, 43))
}

With ConfigDict definition:

data.extra["griffe_pydantic"] ={
     'fields': functools.partial(<function _model_fields at 0x7400568c1bd0>, Class('TermObject', 6, 44)), 
     'validators': functools.partial(<function _model_validators at 0x7400568c1c60>, Class('TermObject', 6, 44)), 
     'config': {
         'json_schema_extra': ExprDict(
             keys=["'example'"], 
             values=[ExprDict(keys=["'usage'", "'limitations'", "'billing'", "'notice_period'"], 
                              values=["'Data can be used for reports, analytics and machine learning use cases.'", "'Not suitable for real-time use cases. Data may not be used to identify individual customers.'", "'5000 USD per month'", "'P3M'"]
                              )
                     ]
             )
         }
}
 

In the first case you can see that there is no section about config, but in the second one there is a section about config. So in the first case the {% if class.extra.griffe_pydantic.config %} condition is not met so no issue, but with newer pydantic version the config now exist and looks like this:

{'json_schema_extra': ExprDict(keys=["'example'"], values=[ExprDict(keys=["'usage'", "'limitations'", "'billing'", "'notice_period'"], values=["'Data can be used for reports, analytics and machine learning use cases.'", "'Not suitable for real-time use cases. Data may not be used to identify individual customers.'", "'5000 USD per month'", "'P3M'"])])}

So here we see that it mapped strings to ExprDict, and highlight method does not like to get ExprDict as input which finally raise our error:

TypeError: expected string or bytes-like object

To Reproduce

Inorder to reproduce you can try to display this pydantic model:

from typing import Optional

from pydantic import BaseModel, ConfigDict, Field


class TermObject(BaseModel):
    """
    Represents the terms and conditions of a data contract.

    This class defines the structure for the 'terms' section of a data contract,
    including details such as usage terms, limitations, billing information, and notice period.
    """

    usage: Optional[str] = Field(
        None,
        description="The way the data is expected to be used. Can contain business and technical information.",
        example="Data can be used for reports, analytics and machine learning use cases.",
    )
    limitations: Optional[str] = Field(
        None,
        description="Restrictions on how the data can be used, including technical limitations or usage restrictions.",
        example="Not suitable for real-time use cases. Data may not be used to identify individual customers.",
    )
    billing: Optional[str] = Field(
        None,
        description="The pricing model for using the data (e.g., free, monthly fee, or metered pay-per-use).",
        example="5000 USD per month",
    )
    notice_period: Optional[str] = Field(
        None,
        description="Period of time required to terminate or modify the data usage agreement. Uses ISO-8601 format.",
        example="P3M",
    )

    model_config = ConfigDict(
        json_schema_extra={
            "example": {
                "usage": "Data can be used for reports, analytics and machine learning use cases.",
                "limitations": "Not suitable for real-time use cases. Data may not be used to identify individual customers.",
                "billing": "5000 USD per month",
                "notice_period": "P3M",
            }
        }
    )
    
    #class Config:
    #    schema_extra = {
    #        "example": {
    #            "usage": "Data can be used for reports, analytics and machine learning use cases.",
    #            "limitations": "Not suitable for real-time use cases. Data may not be used to identify individual customers.",
    #            "billing": "5000 USD per month",
    #            "notice_period": "P3M",
    #        }
    #    }

Full traceback

ERROR   -  Error reading page 'data_contract_models/term_object.md': expected string or bytes-like
           object
Traceback (most recent call last):
  File "/home/xxxxx/docs/.venv/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocs/__main__.py", line 272, in serve_command
    serve.serve(**kwargs)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocs/commands/serve.py", line 85, in serve
    builder(config)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocs/commands/serve.py", line 67, in builder
    build(config, serve_url=None if is_clean else serve_url, dirty=is_dirty)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocs/commands/build.py", line 310, in build
    _populate_page(file.page, config, files, dirty)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocs/commands/build.py", line 167, in _populate_page
    page.render(config, files)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocs/structure/pages.py", line 285, in render
    self.content = md.convert(self.markdown)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/markdown/core.py", line 357, in convert
    root = self.parser.parseDocument(self.lines).getroot()
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/markdown/blockparser.py", line 117, in parseDocument
    self.parseChunk(self.root, '\n'.join(lines))
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/markdown/blockparser.py", line 136, in parseChunk
    self.parseBlocks(parent, text.split('\n\n'))
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/markdown/blockparser.py", line 158, in parseBlocks
    if processor.run(parent, blocks) is not False:
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocstrings/extension.py", line 130, in run
    html, handler, data = self._process_block(identifier, block, heading_level)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocstrings/extension.py", line 230, in _process_block
    rendered = handler.render(data, options)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocstrings_handlers/python/handler.py", line 406, in render
    return template.render(
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/griffe_pydantic/templates/material/pydantic_model.html.jinja", line 1, in top-level template code
    {% extends "_base/pydantic_model.html.jinja" %}
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/griffe_pydantic/templates/material/_base/pydantic_model.html.jinja", line 1, in top-level template code
    {% extends "_base/class.html.jinja" %}
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocstrings_handlers/python/templates/material/_base/class.html.jinja", line 105, in top-level template code
    {% block contents scoped %}
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/griffe_pydantic/templates/material/_base/pydantic_model.html.jinja", line 15, in block 'contents'
    {% block config scoped %}
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/griffe_pydantic/templates/material/_base/pydantic_model.html.jinja", line 20, in block 'config'
    <li><code>{{ name }}</code>: {{ value|highlight(language="python", inline=True) }}</li>
  File "/home/xxxxx/docs/.venv/lib/python3.10/site-packages/mkdocstrings/handlers/rendering.py", line 113, in highlight
    src = textwrap.dedent(src)
  File "/home/xxxxx/.pyenv/versions/3.10.12/lib/python3.10/textwrap.py", line 438, in dedent
    text = _whitespace_only_re.sub('', text)
TypeError: expected string or bytes-like object

Expected behavior

The expected behavior is to not fail even if some values of the config are not dict and to convert it before in string. Moreover the model_config field should not be displayed since it's now a special field dedicated to the config that will already be displayed.

Here is the suggestion:

<li><code>{{ name }}</code>: {{ value|string|highlight(language="python", inline=True) }}</li>

Environment information

python: 3.10.12

Libs:

annotated-types==0.7.0
anyio==4.6.2.post1
babel==2.16.0
certifi==2024.8.30
charset-normalizer==3.4.0
click==8.1.7
colorama==0.4.6
exceptiongroup==1.2.2
fastapi==0.115.2
ghp-import==2.1.0
griffe==1.4.1
griffe-pydantic==1.0.0
h11==0.14.0
idna==3.10
Jinja2==3.1.4
line_profiler==4.1.3
Markdown==3.7
MarkupSafe==3.0.1
mergedeep==1.3.4
mkdocs==1.6.1
mkdocs-autorefs==1.2.0
mkdocs-get-deps==0.2.0
mkdocs-material==9.5.41
mkdocs-material-extensions==1.3.1
mkdocstrings==0.26.2
mkdocstrings-python==1.12.1
packaging==24.1
paginate==0.5.7
pathspec==0.12.1
platformdirs==4.3.6
pydantic==2.9.2
pydantic_core==2.23.4
Pygments==2.18.0
pymdown-extensions==10.11.2
python-dateutil==2.9.0.post0
PyYAML==6.0.2
pyyaml_env_tag==0.1
regex==2024.9.11
requests==2.32.3
six==1.16.0
sniffio==1.3.1
starlette==0.40.0
typing_extensions==4.12.2
urllib3==2.2.3
uvicorn==0.32.0
watchdog==5.0.3

Tell me what you think about, I can do a PR to implement the changes if needed

@VictorGoubet VictorGoubet added the unconfirmed This bug was not reproduced yet label Oct 17, 2024
@pawamoy
Copy link
Member

pawamoy commented Oct 19, 2024

Hi @VictorGoubet, thank you for the detailed bug report! I'll try to reproduce the issue locally (but this looks legit already).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
unconfirmed This bug was not reproduced yet
Projects
None yet
Development

No branches or pull requests

2 participants