From c041bcdde90d1330f9c0f4e6efa6dd8b3dbd0b8c Mon Sep 17 00:00:00 2001 From: Jeremy Prevost Date: Tue, 26 Mar 2024 12:57:00 -0400 Subject: [PATCH] Add heroku scaling commands to publication process Why are these changes being introduced: * the preservation job uses a lot of memory. Scaling the worker job where preservation is done to a dyno with additional memory will help give us a bit more headroom Relevant ticket(s): * https://mitlibraries.atlassian.net/browse/ETD-598 How does this address that need: * provides instructions on how and why to scale the worker dyno before and after processing the output queue for publishing jobs Document any side effects to this change: * This doesn't really solve the problem, but masks it for our common thesis sizes. To fully address this problem we'll need to revisit how we create the preservation zip files --- README.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index da2f29c2..11979fbc 100644 --- a/README.md +++ b/README.md @@ -365,14 +365,21 @@ Publication Review - Publish) [DSS repo](https://github.com/MITLibraries/dspace-submission-service#run-stage)) 4. ETD processes output queue to update records and send email to stakeholders with summary data and list of any error records. As of now this is a manual process, but can be triggered via rake task using the following - Heroku run command: + sequence of heroku-cli commands: ```shell + # scale the worker dyno to ensure we have enough memory + heroku ps:scale worker=1:standard-2x --app TARGET-HEROKU-APP + + # run the output queue processing job heroku run -s standard-2x rails dss:process_output_queue --app TARGET-HEROKU-APP + + # wait for all ETD emails to be received (the preservation email is the final one to look for) + # scale the worker back down so we do not pay for more CPU/memory than we need + heroku ps:scale worker=1:standard-1x --app TARGET-HEROKU-APP ``` -Note the `-s` option, which sets the dyno size for the run command. We are scaling to the larger '2X' dyno because -this job is very memory-intensive. +Note the `-s` option on the second command, which sets the dyno size for the run command. We are scaling to the larger '2X' dyno because this job is very memory-intensive. We also first scale the worker dyno to 2x and then set it back to 1x when we are done for the same reason (preservation takes a lot of memory). ### Publishing a single thesis