Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

An example mypy configuration #31225

Closed
wants to merge 13 commits into from
26 changes: 26 additions & 0 deletions .github/workflows/type_checking.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Type checking

on:
push:
paths:
- '*.py'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't this only trigger when the code is pushed to master?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should trigger when you push to a PR. (I think paths should actually be '**.py'.)

Fwiw, you can specify to only trigger on pushes to master with

  push:
    branches: [master]


jobs:
mypy:
runs-on: ubuntu-latest
steps:
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.7.4
architecture: x64
- name: Checkout
uses: actions/checkout@v1
- name: Install mypy
run: pip install mypy
- name: Run mypy
uses: sasanquaneuf/mypy-github-action@releases/v1
with:
checkName: 'mypy' # NOTE: This needs to be the same as the job name
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
4 changes: 2 additions & 2 deletions corehq/motech/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"""
import datetime
import re
from typing import Any
from typing import Any, Callable, Optional

from dateutil import parser as dateutil_parser

Expand Down Expand Up @@ -101,7 +101,7 @@ def to_datetime_str(value):
return value.isoformat(timespec='milliseconds')


serializers = {
serializers: dict[tuple[Optional[str], Optional[str]], Callable] = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm disappointed that type hints like this are apparently required by mypy. It's ugly and provides very little information that is not already present in the literals below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's more verbosity and no more useful IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to use variables as parameters or default values, and you want a type checker to verify their type, then the type checker needs to know what type they are. You can choose to use this format, or the # type: ... comment format.

Alternatively, if you don't want the type checker to check its type, you can explicitly tell the type checker to ignore it using # type: ignore or (preferred) # type: ignore[assignment].

# (from_data_type, to_data_type): function
(None, COMMCARE_DATA_TYPE_BOOLEAN): to_boolean,
(None, COMMCARE_DATA_TYPE_DECIMAL): to_decimal,
Expand Down
43 changes: 24 additions & 19 deletions corehq/motech/value_source.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
from typing import Any, Dict, List, Optional, Tuple, Union
from __future__ import annotations

from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional, Tuple, Type, TypeVar, Union

import attr
from jsonobject.containers import JsonDict
Expand Down Expand Up @@ -27,32 +30,34 @@
from .serializers import serializers
from .utils import simplify_list

T = TypeVar('T')

@attr.s

@dataclass
class CaseTriggerInfo:
domain = attr.ib()
case_id = attr.ib()
type = attr.ib(default=None)
name = attr.ib(default=None)
owner_id = attr.ib(default=None)
modified_by = attr.ib(default=None)
updates = attr.ib(factory=dict)
created = attr.ib(default=None)
closed = attr.ib(default=None)
extra_fields = attr.ib(factory=dict)
form_question_values = attr.ib(factory=dict)
domain: str
case_id: str
type: Optional[str] = None
name: Optional[str] = None
owner_id: Optional[str] = None
modified_by: Optional[str] = None
updates: dict[str, Any] = field(default_factory=dict)
created: Optional[bool] = None
closed: Optional[bool] = None
extra_fields: dict[str, Any] = field(default_factory=dict)
form_question_values: dict[str, Any] = field(default_factory=dict)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the switch from attrs to dataclasses required by mypy? If yes, can we no longer use attrs if the code is checked by mypy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the switch wasn't required, but it helps mypy to identify the types of fields. In the typing-extensions library and in Python 3.11, PEP 681 is adding more support for authors of libraries like attrs, pydantic, SQLAlchemy and Django ORMs, etc. so that type checkers can check field types.


def __str__(self):
if self.name:
return f'<CaseTriggerInfo {self.case_id} {self.name!r}>'
return f"<CaseTriggerInfo {self.case_id}>"


def recurse_subclasses(cls):
return (
cls.__subclasses__()
+ [subsub for sub in cls.__subclasses__() for subsub in recurse_subclasses(sub)]
)
def recurse_subclasses(cls: Type[T]) -> list[Type[T]]:
return cls.__subclasses__() + [
subsub for sub in cls.__subclasses__()
for subsub in recurse_subclasses(sub)
]


@attr.s(auto_attribs=True, kw_only=True)
Expand Down Expand Up @@ -86,7 +91,7 @@ class ValueSource:
jsonpath: Optional[str] = None

@classmethod
def wrap(cls, data: dict):
def wrap(cls, data: dict) -> ValueSource:
"""
Allows us to duck-type JsonObject, and useful for doing
pre-instantiation transforms / dropping unwanted attributes.
Expand Down
20 changes: 20 additions & 0 deletions mypy.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Global options:

[mypy]
python_version = 3.9
ignore_missing_imports = True
follow_imports = silent

disallow_subclassing_any = True
warn_return_any = True

warn_redundant_casts = True
warn_unused_ignores = True
warn_unused_configs = True
show_error_codes = True

# Per-module options:

[corehq.motech.value_source]
disallow_untyped_defs = True
disallow_any_generics = True
1 change: 1 addition & 0 deletions mypy_typed_modules.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
corehq/motech/value_source.py
14 changes: 11 additions & 3 deletions requirements/dev-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -334,8 +334,12 @@ matplotlib-inline==0.1.3
# via ipython
mccabe==0.6.1
# via flake8
mypy==0.931
# via -r test-requirements.in
mypy-extensions==0.4.3
# via black
# via
# black
# mypy
nose==1.3.7
# via
# -r test-requirements.in
Expand Down Expand Up @@ -633,7 +637,9 @@ tinys3==0.1.12
toml==0.10.2
# via pep517
tomli==2.0.0
# via black
# via
# black
# mypy
toposort==1.7
# via -r base-requirements.in
traitlets==5.1.1
Expand All @@ -647,7 +653,9 @@ turn-python==0.0.1
twilio==6.5.1
# via -r base-requirements.in
typing-extensions==4.1.1
# via black
# via
# black
# mypy
ua-parser==0.10.0
# via user-agents
unidecode==1.2.0
Expand Down
1 change: 1 addition & 0 deletions requirements/test-requirements.in
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
beautifulsoup4
django-nose @ https://github.com/dimagi/django-nose/raw/fast-first-1.4.6.1/releases/django_nose-1.4.6.1-py2.py3-none-any.whl
fakecouch
mypy
nose
nose-exclude
pip-tools>6.4.0
Expand Down
8 changes: 8 additions & 0 deletions requirements/test-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,10 @@ markupsafe==1.1.1
# via
# jinja2
# mako
mypy==0.931
# via -r test-requirements.in
mypy-extensions==0.4.3
# via mypy
nose==1.3.7
# via
# -r test-requirements.in
Expand Down Expand Up @@ -512,6 +516,8 @@ tinys3==0.1.12
# via -r base-requirements.in
toml==0.10.2
# via pep517
tomli==2.0.1
# via mypy
toposort==1.7
# via -r base-requirements.in
tropo-webapi-python==0.1.3
Expand All @@ -520,6 +526,8 @@ turn-python==0.0.1
# via -r base-requirements.in
twilio==6.5.1
# via -r base-requirements.in
typing-extensions==4.1.1
# via mypy
ua-parser==0.10.0
# via user-agents
unidecode==1.2.0
Expand Down