Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create cupid workflow #176

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

Create cupid workflow #176

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
298c95c
Create cupid workflow
Jul 19, 2024
4834b4c
Get rid of code that loads conda and add code that will ensure the co…
Jul 24, 2024
59b6785
Merge tag 'ccs_config_cesm1.0.6' into add_cupid_workflow
mnlevy1981 Sep 7, 2024
95ae795
Move generate_cupid_config_file into CUPiD
mnlevy1981 Sep 7, 2024
cb6b713
Update template.cupid call to generate config
mnlevy1981 Sep 8, 2024
bbfb133
Changed API for CUPiD script
mnlevy1981 Sep 9, 2024
737fb64
Merge tag 'ccs_config_cesm1.0.8' into add_cupid_workflow
mnlevy1981 Nov 20, 2024
48d78fb
Updates for CUPiD API changes
mnlevy1981 Dec 18, 2024
4b2ac3b
Add a few local variables to template.cupid
mnlevy1981 Dec 19, 2024
d3cdbd0
Call cupid scripts directly
mnlevy1981 Dec 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions machines/casper/config_machines.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<OS>LINUX</OS>
<COMPILERS>nvhpc,intel</COMPILERS>
<MPILIBS>openmpi</MPILIBS>
<CIME_OUTPUT_ROOT>/glade/scratch/$USER</CIME_OUTPUT_ROOT>
<CIME_OUTPUT_ROOT>/glade/derecho/scratch/$USER</CIME_OUTPUT_ROOT>
<DIN_LOC_ROOT>$ENV{CESMDATAROOT}/inputdata</DIN_LOC_ROOT>
<DIN_LOC_ROOT_CLMFORC>/glade/p/cgd/tss/CTSM_datm_forcing_data</DIN_LOC_ROOT_CLMFORC>
<DOUT_S_ROOT>$CIME_OUTPUT_ROOT/archive/$CASE</DOUT_S_ROOT>
Expand Down Expand Up @@ -108,7 +108,7 @@
<environment_variables>
<env name="MODULEPATH">/glade/u/apps/dav/modulefiles/default/compilers:/glade/u/apps/dav/modulefiles/default/idep</env>
<env name="OMP_STACKSIZE">256M</env>
<env name="TMPDIR">/glade/scratch/$USER</env>
<env name="TMPDIR">/glade/derecho/scratch/$USER</env>
<env name="CESMDATAROOT">/glade/p/cesmdata/cseg</env>
<env name="NETCDF_PATH">$ENV{NETCDF}</env>
</environment_variables>
Expand Down
13 changes: 13 additions & 0 deletions machines/config_workflow.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,19 @@
</job>
</workflow_jobs>

<workflow_jobs id="case.cupid" prepend="default">
<job name="case.cupid">
<template>template.cupid</template>
<dependency>case.st_archive</dependency>
<prereq>1</prereq>
<runtime_parameters MACH="derecho">
<task_count>1</task_count>
<tasks_per_node>1</tasks_per_node>
<walltime>0:20:00</walltime>
</runtime_parameters>
</job>
</workflow_jobs>

<!-- THIS CREATES THE JOB FOR THE LAUNCHING OF THE SMARTSIM ORCHESTRATOR -->
<workflow_jobs id="smartsim_db" prepend="default">
<job name="smartsim_db"> <!-- name here corresponds with scripts -->
Expand Down
2 changes: 2 additions & 0 deletions machines/derecho/config_machines.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@
<command name="load">cesmdev/1.0</command>
<command name="load">ncarenv/23.09</command>
<command name="purge"/>
<command name="load">conda/latest</command>
<command name="load">nco</command>
<command name="load">craype</command>
</modules>
<modules compiler="intel">
Expand Down
11 changes: 11 additions & 0 deletions machines/template.cupid
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

# Batch system directives
{{ batchdirectives }}

source .env_mach_specific.sh
conda activate cupid-dev
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cupid-dev is now cupid-infrastructure.

We should then add in generating timeseries files, creating an ADF config file, and running ADF.

We will also want to use cupid-diagnostics (the updated name for cupid-run) below (line 11).

After cupid-diagnostics, we can add cupid-webpage to build the webpage.

{{ srcroot }}/tools/CUPiD/helper_scripts/generate_cupid_config_for_cesm_case.py \
--cesm-root {{ SRCROOT }}

cupid-run