From e01086e6e468f4afc068be645364c0fd91c51df8 Mon Sep 17 00:00:00 2001 From: erexer <13180883+erexer@users.noreply.github.com> Date: Fri, 5 Apr 2024 11:05:03 -0700 Subject: [PATCH 1/3] update quickstarter with dagworks platform instructions --- docs/source/quickstarter.rst | 53 ++++++++++++++++++++++++++++++----- docs/source/release_notes.rst | 6 ++++ 2 files changed, 52 insertions(+), 7 deletions(-) diff --git a/docs/source/quickstarter.rst b/docs/source/quickstarter.rst index ad7391a10d..becafb5898 100644 --- a/docs/source/quickstarter.rst +++ b/docs/source/quickstarter.rst @@ -3,15 +3,15 @@ Quickstarter ============ -Data input +Input data ---------- The only input data required for `naturf` is a shapefile with building footprints and height data. There should be a field with a unique ID for each building the shapefile, and it should be in a projected coordinate system such as Alber Equal Area Conic. For input to the Weather Research and Forecasting model (WRF), the computed parameters for each building will be projected into WGS 84. Either check out our interactive `quickstarter `_, or continue below to run `naturf` using a python file. -Install `naturf` ------------------ +1. Install `naturf` +------------------- In a clean virtual or Conda environment, install `naturf`. NOTE: For Conda environments using Python 3.12, the `setuptools` package does not work as intended. One workaround is to create a Conda environment in Python 3.11. @@ -19,8 +19,8 @@ In a clean virtual or Conda environment, install `naturf`. NOTE: For Conda envir pip install naturf -Edit config variables and create run script -------------------------------------------- +2. Edit config variables and create run script +---------------------------------------------- The `config` module in `naturf` sets the default names for variables used in the `naturf` workflow. The two variables below need to be modified to reflect the ID field and the building height field of the input shapefile. Instructions on changing field names will be given further below. @@ -62,10 +62,49 @@ To run data other than the example data, create the `run.py` below. df = model.execute() model.graph() -Run `naturf` ------------- +3. Run `naturf` +--------------- This will run all functions required to create the output specified in the `run.py` `output_columns` variable. Currently `write_binary` and `write_index`. The `path` variable should point towards the input shapefile. .. code:: bash python run.py + + +Run using the `DAGWorks Platform `_ +---------------------------------------------------- + +Import os and the DAGWorks Tracker: + +.. code:: python3 + + import os + from dagworks import adapters + +Add this to `run.py` at the top of main: + +.. code:: python3 + + tracker = adapters.DAGWorksTracker( + project_id=, + api_key=os.environ["DAGWORKS_API_KEY"], + username="", + dag_name="", + tags={"environment": "DEV", "team": "MY_TEAM", "version": "X"} + ) + +Set the API Key as an environment variable: + +.. code:: bash + + $ export DAGWORKS_API_KEY="" + +Add `tracker` in the `hamilton_adaptors` list: + +.. code:: python3 + + hamilton_adapters = [ + base.SimplePythonDataFrameGraphAdapter(), + h_tqdm.ProgressBar("Naturf DAG"), + tracker, + ] diff --git a/docs/source/release_notes.rst b/docs/source/release_notes.rst index c8dcfc5eb1..3b95ea3c4a 100644 --- a/docs/source/release_notes.rst +++ b/docs/source/release_notes.rst @@ -4,6 +4,12 @@ Release notes This is the list of changes to **naturf** between each release. For full details, see the `commit logs `_. +Version 1.0.2 +_____________ + +- See the `v1.0.2 release notes `_. + + Version 1.0.1 _____________ From 6c1ffe2033bf2ac71ab46d0dbf9b20168cee1fc3 Mon Sep 17 00:00:00 2001 From: erexer <13180883+erexer@users.noreply.github.com> Date: Tue, 9 Apr 2024 07:29:55 -0700 Subject: [PATCH 2/3] update quickstarter with graphviz instructions, clearer wording --- docs/source/quickstarter.rst | 35 ++++++++++++++++++++++++----------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/docs/source/quickstarter.rst b/docs/source/quickstarter.rst index becafb5898..f0567837b5 100644 --- a/docs/source/quickstarter.rst +++ b/docs/source/quickstarter.rst @@ -8,7 +8,7 @@ Input data The only input data required for `naturf` is a shapefile with building footprints and height data. There should be a field with a unique ID for each building the shapefile, and it should be in a projected coordinate system such as Alber Equal Area Conic. For input to the Weather Research and Forecasting model (WRF), the computed parameters for each building will be projected into WGS 84. -Either check out our interactive `quickstarter `_, or continue below to run `naturf` using a python file. +Either check out our interactive `quickstarter `_, or continue below to run `naturf` using a python file. Users need to have `graphviz `_ installed to visualize the Directed Acyclic Graph (DAG). 1. Install `naturf` ------------------- @@ -17,7 +17,7 @@ In a clean virtual or Conda environment, install `naturf`. NOTE: For Conda envir .. code:: bash - pip install naturf + $ pip install naturf 2. Edit config variables and create run script ---------------------------------------------- @@ -74,14 +74,22 @@ This will run all functions required to create the output specified in the `run. Run using the `DAGWorks Platform `_ ---------------------------------------------------- -Import os and the DAGWorks Tracker: +Set the DAGWorks API Key as an environment variable: + +.. code:: bash + + $ export DAGWORKS_API_KEY="" + + +Start with the `run.py` file from above (using either the example data or your own data) and add the following. Import os and DAGWorks adapters, which contains the tracker: .. code:: python3 import os from dagworks import adapters -Add this to `run.py` at the top of main: + +Initialize the DAGWorks tracker: .. code:: python3 @@ -90,16 +98,11 @@ Add this to `run.py` at the top of main: api_key=os.environ["DAGWORKS_API_KEY"], username="", dag_name="", - tags={"environment": "DEV", "team": "MY_TEAM", "version": "X"} + tags={"environment": "DEV", "team": "MY_TEAM", "version": "X"}, ) -Set the API Key as an environment variable: - -.. code:: bash - - $ export DAGWORKS_API_KEY="" -Add `tracker` in the `hamilton_adaptors` list: +Add `tracker` to the `hamilton_adaptors` list: .. code:: python3 @@ -108,3 +111,13 @@ Add `tracker` in the `hamilton_adaptors` list: h_tqdm.ProgressBar("Naturf DAG"), tracker, ] + + +Run the python file! + +.. code:: bash + + $ python run.py + + +You should see a tracked run on the `DAGWorks Platform `_! From 37b73dbf6cbdff4a31616e98b8dbecc848786b75 Mon Sep 17 00:00:00 2001 From: erexer <13180883+erexer@users.noreply.github.com> Date: Tue, 9 Apr 2024 08:06:49 -0700 Subject: [PATCH 3/3] add codecov token, update codecov version --- .github/workflows/build.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 56dcd2469a..aefe130b00 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -41,7 +41,9 @@ jobs: pip install pytest-cov pytest --cov=naturf/ --cov-report=xml - name: Upload coverage to Codecov - uses: codecov/codecov-action@v3 + uses: codecov/codecov-action@v4 + env: + CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} with: file: ./coverage.xml fail_ci_if_error: true