Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support high-availability setups for the interactive tools proxy #18481

Merged
merged 24 commits into from
Sep 4, 2024

Commits on Jul 2, 2024

  1. Support using a table in any database supported by SQLAlchemy as mapp…

    …ing for interactive tools
    
    Replace the class `InteractiveToolSqlite` in lib/galaxy/managers/interactivetool.py with a new class `InteractiveToolPropagatorSQLAlchemy`. The new class implements a SQLAlchemy "propagator" for `InteractiveToolManager` (on the same file). This propagator writes the mappings to the table named after the value of `DATABASE_TABLE_NAME`, in the database specified by the SQLAlchemy database url passed to its constructor. Change the constructor of `InteractiveToolManager` so that it uses `InteractiveToolPropagatorSQLAlchemy`.
    
    Change the method `_process_config` of `galaxy.config.GalaxyAppConfiguration` so that it converts the value of `interactivetools_map` to a SQLAlchemy database url if it is a path.
    
    Update documentation to reflect these changes.
    kysrpex committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    b54bf35 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. Configuration menu
    Copy the full SHA
    bf6b6ed View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    967b591 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Configuration menu
    Copy the full SHA
    d85e1b5 View commit details
    Browse the repository at this point in the history
  2. Configure SQLAlchemy interactive tool maps via new `interactivetools_…

    …map_sqlalchemy` setting
    
    Define a new configuration property `interactivetools_map_sqlalchemy` to store interactivetool proxy mappings on a RDBMS supported by SQLAlchemy.
    
    If set, it takes precedence over `interactivetools_map`. When unset, a SQLite mapping file located at `interactivetools_map` is used (as usual).
    kysrpex committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    ce6d0f7 View commit details
    Browse the repository at this point in the history
  3. Revert changes to description for setting sessions in galaxy.yml.sa…

    …mple
    
    This text is managed by Gravity
    kysrpex committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    647fd4c View commit details
    Browse the repository at this point in the history
  4. Forbid setting interactivetools_map_sqlalchemy from matching `datab…

    …ase_connection` or `install_database_connection`
    kysrpex committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    678bea1 View commit details
    Browse the repository at this point in the history
  5. Add note in interactive tools docs page informing about `interactivet…

    …ools_map_sqlalchemy` setting
    
    Refer to SQLite docs page describing situations where a client/server RDBMS works better and provide an example. Link to the `interactivetools_map_sqlalchemy` on the Galaxy configuration page.
    kysrpex committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    dddf5ba View commit details
    Browse the repository at this point in the history
  6. workaround mypy error on statement using sqlalchemy.select

    ```
    lib/galaxy/managers/interactivetool.py:65: error: No overload variant of
    "select" matches argument type "ReadOnlyColumnCollection[str, Column[Any]]"
    ```
    kysrpex committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    6be9b60 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Configuration menu
    Copy the full SHA
    e7dbe42 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c1fd2b5 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Add sqlalchemy to config package requirements

    `lib.galaxy.config.GalaxyAppConfiguration._process_config` requires `sqlalchemy.engine.make_url` to forbid `interactivetools_map_sqlalchemy` from matching `database_connection` or `install_database_connection` (678bea1)
    kysrpex committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    b6ad695 View commit details
    Browse the repository at this point in the history
  2. Test that Galaxy raises ConfigurationError when setting `interactiv…

    …etools_map_sqlalchemy` matches `database_connection` or `install_database_connection` (678bea1)
    kysrpex committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    5e8ca55 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Set interactivetools_map to None when `interactivetools_map_sqlal…

    …chemy` is defined
    
    Given that `interactivetools_map` is ignored if `interactivetools_map_sqlalchemy` is set, first check `interactivetools_map_sqlalchemy`. If it's not set, then build the sqlite url; but if it is set, then set `interactivetools_map` to `None` for clarity (and to avoid potential bugs in the future)
    
    Co-authored-by: John Davis <jdavcs@gmail.com>
    kysrpex and jdavcs committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    26fd210 View commit details
    Browse the repository at this point in the history
  2. Fix typos in descriptions of interactivetools_map and `interactivet…

    …ools_map_sqlalchemy`
    
    Authored-by: John Davis <jdavcs@gmail.com>
    kysrpex committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    ffa8ea4 View commit details
    Browse the repository at this point in the history
  3. Mention that interactivetools_map_sqlalchemy overrides `interactive…

    …tools_map` in the latter's description
    
    Co-authored-by: John Davis <jdavcs@gmail.com>
    kysrpex and jdavcs committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    018e541 View commit details
    Browse the repository at this point in the history
  4. Refactor variable name query to stmt for calls to select(), `in…

    …sert()` or `delete()`
    kysrpex committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    e040750 View commit details
    Browse the repository at this point in the history
  5. Access individual table columns via __getattr__ rather than `__geti…

    …tem__`
    
    Match @jdavcs' coding style.
    kysrpex committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    8305016 View commit details
    Browse the repository at this point in the history
  6. Restore original error handling for `InteractiveToolPropagatorSQLAlch…

    …emy.get()`
    
    Emit a warning when `get()` returns no results.
    kysrpex committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    6fc0d46 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    4e6688c View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Configuration menu
    Copy the full SHA
    0b6ad17 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9ee6636 View commit details
    Browse the repository at this point in the history
  3. Use urlparse to compare interactivetoolsproxy_map with `database_…

    …connection` and `install_database_connection`
    
    Using `urlparse` rather than `sqlalchemy.engine.make_url` decreases the precision of the comparison, but removes the need to declare `sqlalchemy` as a dependency of the config package.
    
    This is a tradeoff between convenience (preventing mistakes from the user) and complexity (number of dependencies of the config package).
    kysrpex committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    5c0b5d6 View commit details
    Browse the repository at this point in the history
  4. Revert "Add sqlalchemy to config package requirements"

    This reverts commit b6ad695.
    kysrpex committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    7510372 View commit details
    Browse the repository at this point in the history