Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Selective checks for Task SDK #43185

Merged
merged 1 commit into from
Oct 19, 2024

Conversation

kaxil
Copy link
Member

@kaxil kaxil commented Oct 19, 2024

Task SDK tests were running as part of "DB tests" with other Core files & providers, this PR changes it so they run separately. It also adds separate mypy checks for it.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@kaxil kaxil added the area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK label Oct 19, 2024
Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one nit, but mostly looks good to me 👍

dev/breeze/tests/test_selective_checks.py Outdated Show resolved Hide resolved
@Lee-W Lee-W self-requested a review October 19, 2024 03:54
@potiuk
Copy link
Member

potiuk commented Oct 19, 2024

Nice! just two small things but LGTM

Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually ... There is another problem. the task-sdk-tests.yml is never called from the ci workflow :)

You need something like k8s tests calling from the main ci.yml @kaxil - because task-sdk-tests.yml is triggered on workflow call.

  tests-kubernetes:
    name: "Kubernetes tests"
    uses: ./.github/workflows/k8s-tests.yml
    needs: [build-info, wait-for-prod-images]
    permissions:
      contents: read
      packages: read
    secrets: inherit
    with:
      runs-on-as-json-default: ${{ needs.build-info.outputs.runs-on-as-json-default }}
      image-tag: ${{ needs.build-info.outputs.image-tag }}
      python-versions-list-as-string: ${{ needs.build-info.outputs.python-versions-list-as-string }}
      kubernetes-versions-list-as-string: ${{ needs.build-info.outputs.kubernetes-versions-list-as-string }}
      kubernetes-combos-list-as-string: ${{ needs.build-info.outputs.kubernetes-combos-list-as-string }}
      include-success-outputs: ${{ needs.build-info.outputs.include-success-outputs }}
      debug-resources: ${{ needs.build-info.outputs.debug-resources }}
    if: >
      ( needs.build-info.outputs.run-kubernetes-tests == 'true' ||
      needs.build-info.outputs.needs-helm-tests == 'true')

@kaxil
Copy link
Member Author

kaxil commented Oct 19, 2024

Actually ... There is another problem. the task-sdk-tests.yml is never called from the ci workflow :)

You need something like k8s tests calling from the main ci.yml @kaxil - because task-sdk-tests.yml is triggered on workflow call.

  tests-kubernetes:
    name: "Kubernetes tests"
    uses: ./.github/workflows/k8s-tests.yml
    needs: [build-info, wait-for-prod-images]
    permissions:
      contents: read
      packages: read
    secrets: inherit
    with:
      runs-on-as-json-default: ${{ needs.build-info.outputs.runs-on-as-json-default }}
      image-tag: ${{ needs.build-info.outputs.image-tag }}
      python-versions-list-as-string: ${{ needs.build-info.outputs.python-versions-list-as-string }}
      kubernetes-versions-list-as-string: ${{ needs.build-info.outputs.kubernetes-versions-list-as-string }}
      kubernetes-combos-list-as-string: ${{ needs.build-info.outputs.kubernetes-combos-list-as-string }}
      include-success-outputs: ${{ needs.build-info.outputs.include-success-outputs }}
      debug-resources: ${{ needs.build-info.outputs.debug-resources }}
    if: >
      ( needs.build-info.outputs.run-kubernetes-tests == 'true' ||
      needs.build-info.outputs.needs-helm-tests == 'true')

Fixed in 77c786d

@kaxil kaxil requested a review from potiuk October 19, 2024 12:52
@kaxil kaxil force-pushed the fix-task-sdk-selective-checks branch from 77c786d to a105e8c Compare October 19, 2024 12:52
@kaxil kaxil force-pushed the fix-task-sdk-selective-checks branch 5 times, most recently from a10130c to 283773e Compare October 19, 2024 23:22
Task SDK tests were running as part of "DB tests" with other Core files & providers, this PR changes it so they run separately.
It also adds separate mypy checks for it.

This commit fixes several other issues to allow running Task SDK  tests separately.
@kaxil kaxil force-pushed the fix-task-sdk-selective-checks branch from 283773e to a648b0b Compare October 19, 2024 23:24
@kaxil kaxil merged commit f0740b3 into apache:main Oct 19, 2024
77 checks passed
@kaxil kaxil deleted the fix-task-sdk-selective-checks branch October 19, 2024 23:49
harjeevanmaan pushed a commit to harjeevanmaan/airflow that referenced this pull request Oct 23, 2024
Task SDK tests were running as part of "DB tests" with other Core files & providers, this PR changes it so they run separately.
It also adds separate mypy checks for it.

This commit fixes several other issues to allow running Task SDK  tests separately.
PaulKobow7536 pushed a commit to PaulKobow7536/airflow that referenced this pull request Oct 24, 2024
Task SDK tests were running as part of "DB tests" with other Core files & providers, this PR changes it so they run separately.
It also adds separate mypy checks for it.

This commit fixes several other issues to allow running Task SDK  tests separately.
ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
Task SDK tests were running as part of "DB tests" with other Core files & providers, this PR changes it so they run separately.
It also adds separate mypy checks for it.

This commit fixes several other issues to allow running Task SDK  tests separately.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dev-tools area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK
Development

Successfully merging this pull request may close these issues.

4 participants