Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic reconciliation between executor RunState and kubernetes #2604

Merged
merged 3 commits into from
Jun 23, 2023

Commits on Jun 22, 2023

  1. Tidy pod_issue_handler

    This is largely a noop with a few minor changes
    
     - Move logging to happen before action is taken
     - Add logging for when issues self resolve
     - Don't break out of detectPodIssues - so we can detect more than 1 issue per round
     -
    JamesMurkin committed Jun 22, 2023
    Configuration menu
    Copy the full SHA
    cbb35d5 View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2023

  1. Add basic reconciliation between executor RunState and kubernetes

    We can get into the state where the executor RunState thinks there is an active run - but there is no pod backing the run
    
    The result of this is that the run can never finish (as there is no pod) and will stay in Pending/Running forever
    
    This PR just adds some basic reconciliation so if there is no pod backing the RunState then an action will be taken
    JamesMurkin committed Jun 23, 2023
    Configuration menu
    Copy the full SHA
    9ace09a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d4ae9c7 View commit details
    Browse the repository at this point in the history