Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix deploy scripts #531

Merged
merged 7 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 43 additions & 5 deletions docs/developer/general/how-to/deploy-a-release.rst
Original file line number Diff line number Diff line change
@@ -1,25 +1,63 @@
Deploy a New Release
-----------------------
====================

**Remember to discuss any new deployments with the appropriate beamline scientist.**

The ``/deploy/deploy_mxbluesky.py`` script will deploy the latest Hyperion version to a specified beamline. Deployments live in ``/dls_sw/ixx/software/bluesky/mx-bluesky_vXXX``. To do a new deployment you should run the deploy script from your mx-bluesky dev environment with e.g.

``python ./deploy/deploy_mxbluesky.py --beamline i24``
General deployment
--------------------

The ``utility_scripts/deploy/deploy_mxbluesky.py`` script will deploy the latest mx-bluesky version to a specified beamline. Deployments live in ``/dls_sw/ixx/software/bluesky/mx-bluesky_vX.X.X``. To do a new deployment you should run the deploy script from your mx-bluesky dev environment with e.g.

.. code:: console

python ./utility_scripts/deploy/deploy_mxbluesky.py --beamline i24


If you want to test the script you can run:

``python ./deploy/deploy_mxbluesky.py --dev-path /your-path/``
.. code:: console

python ./deploy/deploy_mxbluesky.py --dev-path /your-path/

and a released version will be put in ``/your-path/mxbluesky_release_test``.

If you need a specific beamline test deployment you can also run:

``python ./deploy/deploy_mxbluesky.py --beamline i24 --dev-path /your-path/``

.. code:: console

python ./deploy/deploy_mxbluesky.py --beamline i24 --dev-path /your-path/

which will create the beamline deployment (eg. I24) in the specified test directory ``/your-path/mxbluesky_release_test``.


**Note:** When deploying on I24, the edm screens for serial crystallography will be deployed automatically along with the mx-bluesky release.
When running a ``dev`` deployment instead, `this script <https://github.com/DiamondLightSource/mx-bluesky/wiki/Serial-Crystallography-on-I24#deploying-a-local-version-of-the-edm-screens>`_ will also need to be run to get the latest version of the screens.


Hyperion deployment
-------------------

The ``utility_scripts/deploy/deploy_hyperion.py`` script will deploy the latest mx-bluesky/Hyperion version to a specified beamline. Deployments live in ``/dls_sw/ixx/software/bluesky/mx-bluesky_vX.X.X``.

If you have just created a release as above, you may need to run git fetch --tags to get the newest release.

To do a new deployment you should run the deploy script from your Hyperion dev environment with e.g.

.. code:: console

python ./utility_scripts/deploy/deploy_hyperion.py i03


If you want to test the script you can run:


.. code:: console

python ./utility_scripts/deploy/deploy_hyperion.py dev


and a released version will be put in ``/scratch/30day_tmp/hyperion_release_test``.

For building and deploying a Docker image please see :doc:`../../hyperion/deploying-hyperion`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a note for deploying on i03:

post-installation step:
On i03 installation will succeed with error messages due to RedHat 7 versions of a dependency being unavailable. However the installation will be incomplete.

To fix this, on a RH 8 workstation

. ./.venv/bin/activate
pip install confluent-kafka

Then on the control machine

. ./.venv/bin/activate
pip install -e.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then

pip install -e ../dodal

2 changes: 1 addition & 1 deletion utility_scripts/deploy/deploy_hyperion.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ def _create_environment_from_control_machine(

def main(options: Options):
release_area = options.release_dir
this_repo_top = os.path.abspath(os.path.join(os.path.dirname(__file__), "../.."))
this_repo_top = os.path.abspath(os.path.join(os.path.dirname(__file__), "../../.."))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we actually need this change - ../.. will be the project directory which contains the .git folder, mx-bluesky needs it because it was ../.git rather than ../../.git
I think I only ran into this issue bc I initially ran the wrong script


if not options.quiet:
print(f"Repo top is {this_repo_top}")
Expand Down
26 changes: 13 additions & 13 deletions utility_scripts/deploy/deploy_mxbluesky.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from git import Repo
from packaging.version import Version

recognised_beamlines = ["i24"]
recognised_beamlines = ["i04", "i24"]

help_message = """
To deploy mx_bluesky on a specific beamline, pass only the --beamline argument.
Expand Down Expand Up @@ -74,8 +74,8 @@ def get_permission_groups(beamline: str | None = None) -> list:

# Get the release directory based off the beamline and the latest mx_bluesky version
def get_beamline_and_release_dir_from_args(repo: repo) -> tuple[str | None, str]:
if repo.name != "mx_bluesky":
raise ValueError("This function should only be used with the mx_bluesky repo")
if repo.name != "mx-bluesky":
raise ValueError("This function should only be used with the mx-bluesky repo")

parser = argparse.ArgumentParser(
formatter_class=argparse.RawTextHelpFormatter,
Expand Down Expand Up @@ -126,22 +126,22 @@ def run_process_and_print_output(proc_to_run):

if __name__ == "__main__":
mx_repo = repo(
name="mx_bluesky",
repo_args=os.path.join(os.path.dirname(__file__), "../.git"),
name="mx-bluesky",
repo_args=os.path.join(os.path.dirname(__file__), "../../.git"),
)

# Gives path to /bluesky
beamline, release_area = get_beamline_and_release_dir_from_args(mx_repo)

release_area_version = os.path.join(
release_area, f"mx_bluesky_{mx_repo.latest_version_str}"
release_area, f"mx-bluesky_{mx_repo.latest_version_str}"
)

print(f"Putting releases into {release_area_version}")

dodal_repo = repo(
name="dodal",
repo_args=os.path.join(os.path.dirname(__file__), "../../dodal/.git"),
repo_args=os.path.join(os.path.dirname(__file__), "../../../dodal/.git"),
)

dodal_repo.set_deploy_location(release_area_version)
Expand All @@ -151,7 +151,7 @@ def run_process_and_print_output(proc_to_run):
mx_repo.deploy(mx_repo.origin.url, beamline)

# Get version of dodal that latest mx_bluesky version uses
with open(f"{release_area_version}/mx_bluesky/pyproject.toml") as setup_file:
with open(f"{release_area_version}/mx-bluesky/pyproject.toml") as setup_file:
dodal_url = [
line
for line in setup_file
Expand All @@ -165,8 +165,8 @@ def run_process_and_print_output(proc_to_run):
os.chdir(mx_repo.deploy_location)
print(f"Setting up environment in {mx_repo.deploy_location}")

if mx_repo.name == "mx_bluesky":
run_process_and_print_output("./utility_scripts/dls_dev_setup.sh")
if mx_repo.name == "mx-bluesky":
run_process_and_print_output("./utility_scripts/dls_dev_env.sh")

# If on beamline I24 also deploy the screens to run ssx collections
if beamline == "i24":
Expand All @@ -175,15 +175,15 @@ def run_process_and_print_output(proc_to_run):

move_symlink = input(
"""Move symlink (y/n)? WARNING: this will affect the running version!
Only do so if you have informed the beamline scientist and you're sure mx_bluesky is not running.
Only do so if you have informed the beamline scientist and you're sure mx-bluesky is not running.
"""
)
# Creates symlink: software/bluesky/mx_bluesky_version -> software/bluesky/mx_bluesky
if move_symlink == "y":
live_location = os.path.join(release_area, "mx_bluesky")
live_location = os.path.join(release_area, "mx-bluesky")
new_tmp_location = os.path.join(release_area, "tmp_art")
os.symlink(mx_repo.deploy_location, new_tmp_location)
os.rename(new_tmp_location, live_location)
print(f"New version moved to {live_location}")
else:
print("Quiting without latest version being updated")
print("Quitting without latest version being updated")