Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport 3043 (remote register launch plan name) to v1.14 + 3024 #3045

Merged
merged 3 commits into from
Jan 10, 2025

Conversation

wild-endeavor
Copy link
Contributor

@wild-endeavor wild-endeavor commented Jan 8, 2025

Cherry-pick a06c4dae7

also https://github.com/flyteorg/flytekit/pull/3024/files

Summary by Bito

This PR backports changes from PR #3043 to version 1.14, enhancing workflow handling and remote execution capabilities. It implements proper _lhs attribute initialization in PythonFunctionWorkflow and improves remote launch plan registration. Additionally, it adds version constraint to apache-airflow-providers-google package in flytekit-airflow plugin, limiting it to versions below 12.0.0.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 1

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 8, 2025

Code Review Agent Run #5383bb

Actionable Suggestions - 2
  • flytekit/remote/remote.py - 1
  • tests/flytekit/unit/core/test_workflows.py - 1
    • Consider expanding workflow lhs test coverage · Line 360-362
Review Details
  • Files reviewed - 4 · Commit Range: 1973798..1973798
    • flytekit/core/workflow.py
    • flytekit/remote/remote.py
    • tests/flytekit/integration/remote/test_remote.py
    • tests/flytekit/unit/core/test_workflows.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

@wild-endeavor wild-endeavor changed the title cp a06c4dae7 Backport 3043 to v1.14 Jan 8, 2025
@wild-endeavor wild-endeavor changed the title Backport 3043 to v1.14 Backport 3043 (remote register launch plan name) to v1.14 Jan 8, 2025
pingsutw
pingsutw previously approved these changes Jan 8, 2025
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 8, 2025

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Feature Improvement - Enhanced Workflow and Launch Plan Handling

workflow.py - Added _lhs attribute initialization for workflow function name

remote.py - Fixed workflow module extraction in remote execution

test_remote.py - Added test for launch plan registration functionality

test_workflows.py - Added test to verify workflow _lhs attribute

Other Improvements - Dependency Management Updates

setup.py - Added version constraint for apache-airflow-providers-google package

@@ -1264,7 +1264,7 @@ def register_launch_plan(
:return:
"""
if serialization_settings is None:
_, _, _, module_file = extract_task_module(entity)
_, _, _, module_file = extract_task_module(entity.workflow)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential attribute error in module extraction

Consider checking if entity.workflow exists before accessing it. The extract_task_module() function is being called with entity.workflow which may not exist on all entities.

Code suggestion
Check the AI-generated fix before applying
Suggested change
_, _, _, module_file = extract_task_module(entity.workflow)
if not hasattr(entity, 'workflow'):
raise ValueError('Entity does not have a workflow attribute')
_, _, _, module_file = extract_task_module(entity.workflow)

Code Review Run #5383bb


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Comment on lines +360 to +362
def test_workflow_lhs():
assert my_wf_example._lhs == "my_wf_example"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider expanding workflow lhs test coverage

Consider adding more test cases to verify _lhs behavior in different scenarios like nested workflows or when workflow name is changed using decorators.

Code suggestion
Check the AI-generated fix before applying
Suggested change
def test_workflow_lhs():
assert my_wf_example._lhs == "my_wf_example"
def test_workflow_lhs():
assert my_wf_example._lhs == "my_wf_example"
@workflow
def nested_wf():
return my_wf_example(a=5)
assert nested_wf._lhs == "nested_wf"
renamed_wf = workflow(name="new_name")(my_wf_example)
assert renamed_wf._lhs == "new_name"

Code Review Run #5383bb


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Copy link

codecov bot commented Jan 8, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.22%. Comparing base (0b4a60a) to head (df4447c).
Report is 4 commits behind head on release-v1.14.

Additional details and impacted files
@@                Coverage Diff                 @@
##           release-v1.14    #3045       +/-   ##
==================================================
+ Coverage          81.40%   93.22%   +11.82%     
==================================================
  Files                200       23      -177     
  Lines              21012     1580    -19432     
  Branches            2707        0     -2707     
==================================================
- Hits               17104     1473    -15631     
+ Misses              3151      107     -3044     
+ Partials             757        0      -757     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 8, 2025

Code Review Agent Run #f4d751

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: 1973798..df4447c
    • tests/flytekit/integration/remote/test_remote.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
@wild-endeavor wild-endeavor changed the title Backport 3043 (remote register launch plan name) to v1.14 Backport 3043 (remote register launch plan name) to v1.14 + 3024 Jan 9, 2025
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 10, 2025

Code Review Agent Run #d3c15a

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: df4447c..0926ba0
    • plugins/flytekit-airflow/setup.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

@wild-endeavor wild-endeavor merged commit adf3869 into release-v1.14 Jan 10, 2025
103 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants