Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@thurber
thurber authored Oct 15, 2021
1 parent 44b9db8 commit 7f72ec2
Showing 1 changed file with 20 additions and 21 deletions.
41 changes: 20 additions & 21 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,15 @@ TBD
5. USDA 2012 Census of Agriculture. (2013 and 2008). Table 36. Estimated Quantity of Water Applied and Primary Method of Distribution by Selected Crops Harvested in the Open [Online]. Available at https://www.nass.usda.gov/Publications/AgCensus/2012/Online_Resources/Farm_and_Ranch_Irrigation_Survey/fris13_2_036_036.pdf (accessed 2021-04-12; verified 2021-04-12). USDA-NASS, Washington, DC.
6. USDA Economic Research Service using data from USDA’s Agricultural Resource Management Survey (ARMS) and other sources. (1995-2017). Published Commodity Costs and Returns [Online]. Available at https://www.ers.usda.gov/data-products/commodity-costs-and-returns/commodity-costs-and-returns (accessed 2021-05-12; verified 2021-05-12). USDA-NASS, Washington, DC
7. USDA National Agricultural Statistics Service Cropland Data Layer. (2008-2018). Published crop-specific data layer [Online]. Available at https://nassgeodata.gmu.edu/CropScape/ (accessed 2021-04-05; verified 2021-04-05). USDA-NASS, Washington, DC.
8. Voisin, N. et al. (2018). MOSART-WM-ABM Input Data [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4836886.
8. Voisin, N. et al. (2018). MOSART-WM-ABM Input Data. [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4836886.

### Output data
9. Yoon, Jim. (2021). TBD the `all_nldas_cdl_data_v3.txt` [Data set]. Zenodo. TBD DOI
10. Yoon, Jim. (2021). TBD other output data [Data set]. Zenodo. TBD DOI
9. Yoon, Jim et al. (2021). Supporting data for Yoon et al. 2021 - Nature Sustainability. [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5573041.

## Code reference
11. Yoon, Jim. (2021, TBD). IMMM-SFA/wm-abm_pmp: Water Management Agent Based Model (v1.0.0). Zenodo. http://doi.org/10.5281/zenodo.5570864.
12. Yoon, Jim, & Thurber, Travis. (2021, May 5). IMMM-SFA/iwmm: MOSART-WM-ABM (Version v1.1.2.abm). Zenodo. http://doi.org/10.5281/zenodo.4739516.
13. Yoon, Jim. (2021, TBD). IMMM-SFA/wm-abm_data_processing: PMP calibration and simulation for MOSART-WM-ABM (v1.0.0). Zenodo. http://doi.org/10.5281/zenodo.5570501.
10. Yoon, Jim. (2021, TBD). IMMM-SFA/wm-abm_pmp: Water Management Agent Based Model (v1.0.0). Zenodo. http://doi.org/10.5281/zenodo.5570864.
11. Yoon, Jim, & Thurber, Travis. (2021, May 5). IMMM-SFA/iwmm: MOSART-WM-ABM (Version v1.1.2.abm). Zenodo. http://doi.org/10.5281/zenodo.4739516.
12. Yoon, Jim. (2021, TBD). IMMM-SFA/wm-abm_data_processing: PMP calibration and simulation for MOSART-WM-ABM (v1.0.0). Zenodo. http://doi.org/10.5281/zenodo.5570501.

## Contributing modeling software
| Model | Version | Repository Link | DOI |
Expand All @@ -56,28 +55,28 @@ __0a.__ Preprocess the USDA data to produce these files:

__0b.__ Download the Cropland Data Layer (CDL) files for 2008-2018 from __[7]__ and preprocess:
* For each year, sum the pixels for each CDL Crop category within each 1/8 degree North American Land Data Assimilation System (NLDAS) grid cell using ArcMap or similar tool; producing files like `cdl_{year}_clean.csv`
* Run the script `cdl_processing.py` from __[11]__ to combine the yearly files into a single file in the expected format: `all_nldas_cdl_data_v3.txt`
* Run the script `cdl_processing.py` from __[10]__ to combine the yearly files into a single file in the expected format: `all_nldas_cdl_data_v3.txt`

__0c.__ Preprocess the output from steps __0a__ and __0b__ for use in MOSART-WM-ABM:
* `crop_ids_by_farm.p` - Pickled dictionary mapping farm ID to the list of crop IDs on the farm; developed as part of this experiment.
* `NLDAS_Grid_Reference.csv` - Maps NLDAS IDs to latitude/longitude coordinates; developed as part of this experiment by spatially joining an NLDAS shapefile with a U.S. counties shapefile in GIS.
* `nldas.txt` - Subset of columns from `NLDAS_Grid_Reference.csv`.
* `nldas_ids.p` - Pickled list of NLDAS IDs to be included in the optimization; developed as part of this experiment.
* Run the script `wmabm_data_process.py` from __[11]__ to generate the following files:
* Run the script `wmabm_data_process.py` from __[10]__ to generate the following files:
* `max_land_constr_20201102_protocol2.p`
* `MOSART_WM_PMP_inputs_20201028.xlsx`
* Run the script `hist_water_availability_abm.py` from __[13]__ which ingests some output files from a historical MOSART-WM simulation to generate the following files:
* Run the script `hist_water_availability_abm.py` from __[12]__ which ingests some output files from a historical MOSART-WM simulation to generate the following files:
* `hist_dependent_storage.csv`
* `hist_avail_bias_correction_20201102.csv`
* Run the script `MOSART_WM_PMP_stage1_noloop.py` from __[13]__ to generate the following files:
* Run the script `MOSART_WM_PMP_stage1_noloop.py` from __[12]__ to generate the following files:
* `gammas_new_20201102_protocol2.p`
* `net_prices_new_20201102_protocol2.p`

__1.__ The results of step __0a__ are available as part of the code repository __[11]__; the results of step __0b__ can be downloaded from __[9]__; the results of step __0c__ are available as part of the code repository __[12]__.
__1.__ The results of step __0a__ are available as part of the code repository __[10]__; the results of step __0b__ can be downloaded from __[9]__; the results of step __0c__ are available as part of the code repository __[11]__.

__2.__ Download and unpack the MOSART-WM-ABM input data from __[8]__ into a supercomputing environment.

__3.__ Clone the MOSART-WM-ABM repository __[12]__ into a supercomputing environment; modifications to the source code may be necessary to run on unsupported machines.
__3.__ Clone the MOSART-WM-ABM repository __[11]__ into a supercomputing environment; modifications to the source code may be necessary to run on unsupported machines.

__4.__ From the repository root directory, setup the model with the following shell commands (replacing sections surrounded by curly braces with the specifics of your environment):
* `./cime/script/create_newcase --case {absolute path to a directory to create for your run} --res NLDAS_NLDAS --compset mosart_runoff_driven --project {cost center for your use of supercomputer}`
Expand All @@ -101,25 +100,25 @@ __6.__ Build the model with the shell command: `./case.build`

__7.__ Submit the model run to the job queue with the shell command: `./case.submit`

__8.__ If all is well, the model will eventually run, generating output and log files into your directory within the supercomputer's scratch file system, including the data found in __[10]__.
__8.__ If all is well, the model will eventually run, generating output and log files into your directory within the supercomputer's scratch file system, including the output data found in __[9]__.

## Reproduce my figures
Reproduce my figures:

The figures for the manuscript are generated based on post-processing of MOSART-WM-ABM output files in Python, Tableau, QGIS, and Inkscape. The WM-ABM working directory contains two general types of output files: 1) monthly netCDF output files with the MOSART-WM results (e.g., river flows, water demand, water shortage, etc.) and, 2) csv files containing annual ABM results (crop types, crop areas, etc.). The general steps are laid out below:

__1.__ The monthly netCDF files are post-processed into a consolidated csv file using the "WM_output_PIC_ncdf.py" script. The script reads in monthly the MOSART-WM netCDF files, calculates various annual summary statistics (e.g., average supply for each cell by year), saves the summary statistics into a pandas dataframe, and finally outputs the pandas dataframe as a csv (e.g., "wm_summary_results.csv").
__1.__ The monthly netCDF files are post-processed into a consolidated csv file using the "WM_output_PIC_ncdf.py" script. The script reads in monthly the MOSART-WM netCDF files from __[9]__, calculates various annual summary statistics (e.g., average supply for each cell by year), saves the summary statistics into a pandas dataframe, and finally outputs the pandas dataframe as a csv (e.g., "wm_summary_results.csv").

__2.__ The "abm_output_processing.py" script is used to further post-process output, taking the annual abm output csv files and the wm summary file generated in __[1]__ above as inputs.
__2.__ The "abm_output_processing.py" script is used to further post-process output, taking the annual abm output csv files and the wm summary file generated in step 1 as inputs.

__3.__ Figure 1a is generated based upon taking post-processed output from __[2]__ and developing the final visualizations in Tableau. Final touch ups (e.g., drought labels) are added in Inkscape.
__3.__ Figure 1a is generated based upon taking post-processed output from step 2 and developing the final visualizations in Tableau. Final touch ups (e.g., drought labels) are added in Inkscape.

__4.__ Figure 1b is generated based upon taking post-processed output from __[2]__ and generating the final maps in QGIS. A Stamen Terrain Background map is used for the visualization using QuickMapServices in QGIS. Final touch ups (e.g., modification of label locations) are conducted in Inkscape.
__4.__ Figure 1b is generated based upon taking post-processed output from step 2 and generating the final maps in QGIS. A Stamen Terrain Background map is used for the visualization using QuickMapServices in QGIS. Final touch ups (e.g., modification of label locations) are conducted in Inkscape.

__5.__ The crop area bump charts (left side) of Figure 2 are generated based upon taking post-processed output from __[2]__ and developing the final visualizations in Tableau. See guide here: http://www.datatableauandme.com/2017/08/how-to-area-bump-chart-in-tableau.html
__5.__ The crop area bump charts (left side) of Figure 2 are generated based upon taking post-processed output from step 2 and developing the final visualizations in Tableau. See guide here: http://www.datatableauandme.com/2017/08/how-to-area-bump-chart-in-tableau.html.

__6.__ The shortage and adaptivity maps (right side) of Figure 2 are generated taking post-processed output from __[2]__ and developing the final visualizations in QGIS with final touch ups conducted in Inkscape.
__6.__ The shortage and adaptivity maps (right side) of Figure 2 are generated taking post-processed output from step 2 and developing the final visualizations in QGIS with final touch ups conducted in Inkscape.

__7.__ Figure 3 is generated based upon taking post-processed output from __[2]__ and developing the final visualizations in Tableau. Final touch ups (e.g., drought labels) are added in Inkscape.
__7.__ Figure 3 is generated based upon taking post-processed output from step 2 and developing the final visualizations in Tableau. Final touch ups (e.g., drought labels) are added in Inkscape.

__8.__ Figure 4 is developed in Powerpoint and does not rely on any data inputs
__8.__ Figure 4 is developed in Powerpoint and does not rely on any data inputs.

0 comments on commit 7f72ec2

Please sign in to comment.