From b9490e425c291b75cc13ecd877a4b34ca96ab68c Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Tue, 7 May 2024 16:56:32 +1000 Subject: [PATCH 01/31] First draft of the ADR for watt time v3 changes First draft of the ADR for watt time v3 changes. Looking at path mappings and parameters. Still plenty to work on. --- .../decisions/0016-watt-time-v3.md | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 docs/architecture/decisions/0016-watt-time-v3.md diff --git a/docs/architecture/decisions/0016-watt-time-v3.md b/docs/architecture/decisions/0016-watt-time-v3.md new file mode 100644 index 000000000..1e266a14b --- /dev/null +++ b/docs/architecture/decisions/0016-watt-time-v3.md @@ -0,0 +1,62 @@ + +# 0015. WattTime v3 Changes + +## Status + +Proposed + +## Context +As part of the update to Watt Time v3 we are proposing the changes to the underlying API calls. This needs to be tracked so we understand the impacts, and if multiple options are available, which option was selected and why. + +This wll impact the `CarbonAware.DataSources.WattTime` project primarily. + +## Decision + +The proposal is for the outlined WattTime API mapping and changes. + +## WattTime v2, v3 Mapping + +The following document and guidelines was used to understand the impact to the Carbon Aware SDK for the WattTime v3 updates. https://docs.watttime.org/#tag/Transitioning-from-APIv2-to-APIv3 + +### Base URL +The base URL will need to change. +> TODO: Add where this is configured + +|Base URL (v2) | Based URL (v3) | +|---|---| +| /v2 | /v3 | + + +### Paths +The paths will also need to change. + +The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/Paths.cs` + +| API Endpoint | Description | Path (v2) | Path (v3) | Notes | +|--------------|-------------|-----------|-----------|---| +| Data | Get data | /data | /historical | Parameter changes outlined below. Start and End are now mandatory. +| Forecast | Get forecast| /forecast | /forecast | Forecast parameters have undergone broader changes, and it can no longer be used for historical data +| Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** +| BalancingAuthorityFromLocation | Get balancing authority from location | /ba-from-loc | | +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | No other changes + +### Query Strings + +#### Signal Type +Everything call takes an optional `signal_type` parameter that defaults to `co2_moer`. + +The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` + +| Query String (v2) | Query String (v3) | Description | +|------------------------------------|----------------------------------|------------------------------| +| ba | region | Balancing Authority / Region | +| starttime | start | Start Time | +| endtime | end | End Time | +| latitude | | Latitude | +| longitude | | Longitude | +| username | | Username | + +## Green Impact + +Neutral + From b991bac14c9864f331e56b739398b9675f23e37c Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Thu, 9 May 2024 08:22:09 +1000 Subject: [PATCH 02/31] Moved ADR to correct location Moved ADR to correct location --- .../docs}/architecture/decisions/0016-watt-time-v3.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename {docs => casdk-docs/docs}/architecture/decisions/0016-watt-time-v3.md (100%) diff --git a/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md similarity index 100% rename from docs/architecture/decisions/0016-watt-time-v3.md rename to casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md From f4bbf5205e42e59b0ffcade30d43d5d7b1ffb462 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Thu, 9 May 2024 08:46:30 +1000 Subject: [PATCH 03/31] Further updates for the watt time v2 to v3 upgrade Signed-off-by: Vaughan Knight --- .../decisions/0016-watt-time-v3.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md index 1e266a14b..06b1b4b2a 100644 --- a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md +++ b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md @@ -34,27 +34,27 @@ The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/ | API Endpoint | Description | Path (v2) | Path (v3) | Notes | |--------------|-------------|-----------|-----------|---| -| Data | Get data | /data | /historical | Parameter changes outlined below. Start and End are now mandatory. -| Forecast | Get forecast| /forecast | /forecast | Forecast parameters have undergone broader changes, and it can no longer be used for historical data +| Data | Get data | /data | /historical | _Request_
  • `starttime` is now `start` and mandatory
  • `endtime` is now `end` and mandatory
  • `ba` is now `region`
  • `signal_type` added
    _Response_
  • `signal_type` added +| Forecast | Get forecast| /forecast | /forecast | **TODO: CHECK IMPACT**
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added | Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** -| BalancingAuthorityFromLocation | Get balancing authority from location | /ba-from-loc | | -| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | No other changes +| Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being unique from the API version.

    **TODO: CHECK HOW BASE URL IS DEFINED AS THIS WILL NOW HAVE DIFFERENT VALUES** ### Query Strings #### Signal Type Everything call takes an optional `signal_type` parameter that defaults to `co2_moer`. -The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` +The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` and the changes are consistent with the discussion above. -| Query String (v2) | Query String (v3) | Description | +| Query String (v2) | Query String if Changed (v3) | Description | |------------------------------------|----------------------------------|------------------------------| -| ba | region | Balancing Authority / Region | -| starttime | start | Start Time | -| endtime | end | End Time | -| latitude | | Latitude | -| longitude | | Longitude | -| username | | Username | +| `ba` | `region` | Balancing Authority / Region | +| `starttime` | `start` | Start Time | +| `endtime` | `end` | End Time | +| `latitude` | - | Latitude | +| `longitude` | - | Longitude | +| `username` | - | Username | ## Green Impact From d6ad67c5b07e00786207cf381775b944cee62512 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Thu, 9 May 2024 08:46:30 +1000 Subject: [PATCH 04/31] Further updates for the watt time v2 to v3 upgrade Signed-off-by: Vaughan Knight --- .../decisions/0016-watt-time-v3.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md index 1e266a14b..06b1b4b2a 100644 --- a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md +++ b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md @@ -34,27 +34,27 @@ The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/ | API Endpoint | Description | Path (v2) | Path (v3) | Notes | |--------------|-------------|-----------|-----------|---| -| Data | Get data | /data | /historical | Parameter changes outlined below. Start and End are now mandatory. -| Forecast | Get forecast| /forecast | /forecast | Forecast parameters have undergone broader changes, and it can no longer be used for historical data +| Data | Get data | /data | /historical | _Request_
  • `starttime` is now `start` and mandatory
  • `endtime` is now `end` and mandatory
  • `ba` is now `region`
  • `signal_type` added
    _Response_
  • `signal_type` added +| Forecast | Get forecast| /forecast | /forecast | **TODO: CHECK IMPACT**
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added | Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** -| BalancingAuthorityFromLocation | Get balancing authority from location | /ba-from-loc | | -| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | No other changes +| Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being unique from the API version.

    **TODO: CHECK HOW BASE URL IS DEFINED AS THIS WILL NOW HAVE DIFFERENT VALUES** ### Query Strings #### Signal Type Everything call takes an optional `signal_type` parameter that defaults to `co2_moer`. -The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` +The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` and the changes are consistent with the discussion above. -| Query String (v2) | Query String (v3) | Description | +| Query String (v2) | Query String if Changed (v3) | Description | |------------------------------------|----------------------------------|------------------------------| -| ba | region | Balancing Authority / Region | -| starttime | start | Start Time | -| endtime | end | End Time | -| latitude | | Latitude | -| longitude | | Longitude | -| username | | Username | +| `ba` | `region` | Balancing Authority / Region | +| `starttime` | `start` | Start Time | +| `endtime` | `end` | End Time | +| `latitude` | - | Latitude | +| `longitude` | - | Longitude | +| `username` | - | Username | ## Green Impact From cb468778c8907cffc35c95aeb8a3b72866ab89b5 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Wed, 15 May 2024 17:42:29 +1000 Subject: [PATCH 05/31] Update 0016-watt-time-v3.md updated notes for BA Signed-off-by: Vaughan Knight --- casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md index 06b1b4b2a..0e726d313 100644 --- a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md +++ b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md @@ -37,7 +37,7 @@ The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/ | Data | Get data | /data | /historical | _Request_
  • `starttime` is now `start` and mandatory
  • `endtime` is now `end` and mandatory
  • `ba` is now `region`
  • `signal_type` added
    _Response_
  • `signal_type` added | Forecast | Get forecast| /forecast | /forecast | **TODO: CHECK IMPACT**
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added | Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** -| Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | +| Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | Check if the CA SDK uses BA at all

    _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | | Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being unique from the API version.

    **TODO: CHECK HOW BASE URL IS DEFINED AS THIS WILL NOW HAVE DIFFERENT VALUES** ### Query Strings From e324f365a3086830f3ac2d0e5ad4d37112fa5c6a Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Tue, 7 May 2024 16:56:32 +1000 Subject: [PATCH 06/31] First draft of the ADR for watt time v3 changes First draft of the ADR for watt time v3 changes. Looking at path mappings and parameters. Still plenty to work on. --- .../decisions/0016-watt-time-v3.md | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 docs/architecture/decisions/0016-watt-time-v3.md diff --git a/docs/architecture/decisions/0016-watt-time-v3.md b/docs/architecture/decisions/0016-watt-time-v3.md new file mode 100644 index 000000000..1e266a14b --- /dev/null +++ b/docs/architecture/decisions/0016-watt-time-v3.md @@ -0,0 +1,62 @@ + +# 0015. WattTime v3 Changes + +## Status + +Proposed + +## Context +As part of the update to Watt Time v3 we are proposing the changes to the underlying API calls. This needs to be tracked so we understand the impacts, and if multiple options are available, which option was selected and why. + +This wll impact the `CarbonAware.DataSources.WattTime` project primarily. + +## Decision + +The proposal is for the outlined WattTime API mapping and changes. + +## WattTime v2, v3 Mapping + +The following document and guidelines was used to understand the impact to the Carbon Aware SDK for the WattTime v3 updates. https://docs.watttime.org/#tag/Transitioning-from-APIv2-to-APIv3 + +### Base URL +The base URL will need to change. +> TODO: Add where this is configured + +|Base URL (v2) | Based URL (v3) | +|---|---| +| /v2 | /v3 | + + +### Paths +The paths will also need to change. + +The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/Paths.cs` + +| API Endpoint | Description | Path (v2) | Path (v3) | Notes | +|--------------|-------------|-----------|-----------|---| +| Data | Get data | /data | /historical | Parameter changes outlined below. Start and End are now mandatory. +| Forecast | Get forecast| /forecast | /forecast | Forecast parameters have undergone broader changes, and it can no longer be used for historical data +| Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** +| BalancingAuthorityFromLocation | Get balancing authority from location | /ba-from-loc | | +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | No other changes + +### Query Strings + +#### Signal Type +Everything call takes an optional `signal_type` parameter that defaults to `co2_moer`. + +The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` + +| Query String (v2) | Query String (v3) | Description | +|------------------------------------|----------------------------------|------------------------------| +| ba | region | Balancing Authority / Region | +| starttime | start | Start Time | +| endtime | end | End Time | +| latitude | | Latitude | +| longitude | | Longitude | +| username | | Username | + +## Green Impact + +Neutral + From be6663cba48dba0af2ace3db90519377101d2d4d Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Thu, 9 May 2024 08:22:09 +1000 Subject: [PATCH 07/31] Moved ADR to correct location Moved ADR to correct location --- .../docs}/architecture/decisions/0016-watt-time-v3.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename {docs => casdk-docs/docs}/architecture/decisions/0016-watt-time-v3.md (100%) diff --git a/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md similarity index 100% rename from docs/architecture/decisions/0016-watt-time-v3.md rename to casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md From 17210edfda8f0063dab046252de2498788615dec Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Thu, 9 May 2024 08:46:30 +1000 Subject: [PATCH 08/31] Further updates for the watt time v2 to v3 upgrade Signed-off-by: Vaughan Knight --- .../decisions/0016-watt-time-v3.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md index 1e266a14b..06b1b4b2a 100644 --- a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md +++ b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md @@ -34,27 +34,27 @@ The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/ | API Endpoint | Description | Path (v2) | Path (v3) | Notes | |--------------|-------------|-----------|-----------|---| -| Data | Get data | /data | /historical | Parameter changes outlined below. Start and End are now mandatory. -| Forecast | Get forecast| /forecast | /forecast | Forecast parameters have undergone broader changes, and it can no longer be used for historical data +| Data | Get data | /data | /historical | _Request_
  • `starttime` is now `start` and mandatory
  • `endtime` is now `end` and mandatory
  • `ba` is now `region`
  • `signal_type` added
    _Response_
  • `signal_type` added +| Forecast | Get forecast| /forecast | /forecast | **TODO: CHECK IMPACT**
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added | Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** -| BalancingAuthorityFromLocation | Get balancing authority from location | /ba-from-loc | | -| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | No other changes +| Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being unique from the API version.

    **TODO: CHECK HOW BASE URL IS DEFINED AS THIS WILL NOW HAVE DIFFERENT VALUES** ### Query Strings #### Signal Type Everything call takes an optional `signal_type` parameter that defaults to `co2_moer`. -The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` +The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` and the changes are consistent with the discussion above. -| Query String (v2) | Query String (v3) | Description | +| Query String (v2) | Query String if Changed (v3) | Description | |------------------------------------|----------------------------------|------------------------------| -| ba | region | Balancing Authority / Region | -| starttime | start | Start Time | -| endtime | end | End Time | -| latitude | | Latitude | -| longitude | | Longitude | -| username | | Username | +| `ba` | `region` | Balancing Authority / Region | +| `starttime` | `start` | Start Time | +| `endtime` | `end` | End Time | +| `latitude` | - | Latitude | +| `longitude` | - | Longitude | +| `username` | - | Username | ## Green Impact From 7920786bb329034efc4850a7f08709e75d4bc9ff Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Wed, 15 May 2024 17:42:29 +1000 Subject: [PATCH 09/31] Update 0016-watt-time-v3.md updated notes for BA Signed-off-by: Vaughan Knight --- casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md index 06b1b4b2a..0e726d313 100644 --- a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md +++ b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md @@ -37,7 +37,7 @@ The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/ | Data | Get data | /data | /historical | _Request_
  • `starttime` is now `start` and mandatory
  • `endtime` is now `end` and mandatory
  • `ba` is now `region`
  • `signal_type` added
    _Response_
  • `signal_type` added | Forecast | Get forecast| /forecast | /forecast | **TODO: CHECK IMPACT**
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added | Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** -| Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | +| Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | Check if the CA SDK uses BA at all

    _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | | Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being unique from the API version.

    **TODO: CHECK HOW BASE URL IS DEFINED AS THIS WILL NOW HAVE DIFFERENT VALUES** ### Query Strings From e4f14944901e0ebecf4e4a7c8b2c543a87b37a7d Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Fri, 31 May 2024 14:50:27 +0100 Subject: [PATCH 10/31] Create 0016-watt-time-v3.md More updates. --- .../decisions/0016-watt-time-v3.md | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 docs/architecture/decisions/0016-watt-time-v3.md diff --git a/docs/architecture/decisions/0016-watt-time-v3.md b/docs/architecture/decisions/0016-watt-time-v3.md new file mode 100644 index 000000000..661f38262 --- /dev/null +++ b/docs/architecture/decisions/0016-watt-time-v3.md @@ -0,0 +1,62 @@ + +# 0015. WattTime v3 Changes + +## Status + +Proposed + +## Context +As part of the update to Watt Time v3 we are proposing the changes to the underlying API calls. This needs to be tracked so we understand the impacts, and if multiple options are available, which option was selected and why. + +This wll impact the `CarbonAware.DataSources.WattTime` project primarily. + +## Decision + +The proposal is for the outlined WattTime API mapping and changes. + +## WattTime v2, v3 Mapping + +The following document and guidelines was used to understand the impact to the Carbon Aware SDK for the WattTime v3 updates. https://docs.watttime.org/#tag/Transitioning-from-APIv2-to-APIv3 + +### Base URL +The base URL will need to change. +> TODO: Add where this is configured + +|Base URL (v2) | Based URL (v3) | +|---|---| +| /v2 | /v3 | + + +### Paths +The paths will also need to change. + +The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/Paths.cs` + +| API Endpoint | Description | Path (v2) | Path (v3) | Notes | +|--------------|-------------|-----------|-----------|---| +| Data | Get data | /data | /historical | Parameter changes outlined below. Start and End are now mandatory. | +| Forecast | Get forecast| /forecast | /forecast | Forecast parameters have undergone broader changes, and it can no longer be used for historical data +| Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** +| BalancingAuthorityFromLocation | Get balancing authority from location | /ba-from-loc | /region-from-loc | Parameter changes are only `signal_type` | +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | No other changes, but note **the path change is not in the same API path for `login` and other paths** | + +### Query Strings + +#### Signal Type +Everything call takes an optional `signal_type` parameter that defaults to `co2_moer`. + +The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` + +| Query String (v2) | Query String (v3) | Description | +|------------------------------------|----------------------------------|------------------------------| +| ba | region | Balancing Authority / Region | +| starttime | start | Start Time | +| endtime | end | End Time | +| latitude | | Latitude | +| longitude | | Longitude | +| username | | Username | + +## Green Impact + +Neutral + From b443e9eaa3cdcedd7cfab435e64f8a9999ca44d8 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Fri, 31 May 2024 16:49:01 +0100 Subject: [PATCH 11/31] Added base url to the configuration with validation Added AuthenticationBaseUrl to the configuration and updated Authentication to leverage the v3 authenication path - note the API is not updated and will require further updates. --- .../docs/architecture/decisions/0016-watt-time-v3.md | 2 +- .../src/Client/IWattTimeClient.cs | 1 + .../src/Client/WattTimeClient.cs | 11 ++++++++++- .../src/Configuration/WattTimeClientConfiguration.cs | 11 +++++++++++ 4 files changed, 23 insertions(+), 2 deletions(-) diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md index 0e726d313..a2b199c64 100644 --- a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md +++ b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md @@ -38,7 +38,7 @@ The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/ | Forecast | Get forecast| /forecast | /forecast | **TODO: CHECK IMPACT**
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added | Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** | Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | Check if the CA SDK uses BA at all

    _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | -| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being unique from the API version.

    **TODO: CHECK HOW BASE URL IS DEFINED AS THIS WILL NOW HAVE DIFFERENT VALUES** +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being no longer related to the API version.

    **TODO: CHECK HOW BASE URL IS DEFINED AS THIS WILL NOW HAVE DIFFERENT VALUES** ### Query Strings diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs index fbe1fdef1..4cdf21f18 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs @@ -8,6 +8,7 @@ namespace CarbonAware.DataSources.WattTime.Client; internal interface IWattTimeClient { public const string NamedClient = "WattTimeClient"; + public const string NamedAuthenticationClient = "WattTimeAuthenticationClient" /// /// Async method to get observed emission data for a given balancing authority and time period. diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index ed4e10e5d..9b17882ae 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -25,6 +25,7 @@ internal class WattTimeClient : IWattTimeClient }; private HttpClient _client; + private HttpClient _authenticationClient; private IOptionsMonitor _configurationMonitor { get; } @@ -37,12 +38,19 @@ internal class WattTimeClient : IWattTimeClient public WattTimeClient(IHttpClientFactory factory, IOptionsMonitor configurationMonitor, ILogger log, IMemoryCache memoryCache) { _client = factory.CreateClient(IWattTimeClient.NamedClient); + _authenticationClient = factory.CreateClient(IWattTimeClient.NamedAuthenticationClient); + _configurationMonitor = configurationMonitor; _log = log; _configuration.Validate(); _client.BaseAddress = new Uri(this._configuration.BaseUrl); _client.DefaultRequestHeaders.Accept.Clear(); _client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(MediaTypeNames.Application.Json)); + + _authenticationClient.BaseAddress = new Uri(this._configuration.AuthenticationBaseUrl); + _authenticationClient.DefaultRequestHeaders.Accept.Clear(); + _authenticationClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(MediaTypeNames.Application.Json)); + _memoryCache = memoryCache; } @@ -215,7 +223,7 @@ private async Task UpdateAuthTokenAsync() _log.LogInformation("Attempting to log in user {username}", this._configuration.Username); this.SetBasicAuthenticationHeader(); - HttpResponseMessage response = await this._client.GetAsync(Paths.Login); + HttpResponseMessage response = await this._authenticationClient.GetAsync(Paths.Login); LoginResult? data = null; @@ -239,6 +247,7 @@ private void SetBasicAuthenticationHeader() { var authToken = Encoding.UTF8.GetBytes($"{this._configuration.Username}:{this._configuration.Password}"); this._client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(AuthenticationHeaderTypes.Basic, Convert.ToBase64String(authToken)); + this._authenticationClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(AuthenticationHeaderTypes.Basic, Convert.ToBase64String(authToken)); } internal void SetBearerAuthenticationHeader(string token) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs index 746cd2b4c..104aeb759 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs @@ -25,6 +25,12 @@ internal class WattTimeClientConfiguration /// public string BaseUrl { get; set; } = "https://api2.watttime.org/v2/"; + /// + /// Authentication base url. This changed between v2 and v3 + /// to be different to the API base url. + /// + public string AuthenticationBaseUrl { get; set; } = "https://api.watttime.org/"; + /// /// Gets or sets the cached expiration time (in seconds) for a BalancingAuthority instance. /// It defaults to 86400 secs. @@ -51,6 +57,11 @@ public void Validate() throw new ConfigurationException($"{Key}:{nameof(this.BaseUrl)} is not a valid absolute url."); } + if (!Uri.IsWellFormedUriString(this.AuthenticationBaseUrl, UriKind.Absolute)) + { + throw new ConfigurationException($"{Key}:{nameof(this.AuthenticationBaseUrl)} is not a valid absolute url."); + } + // Validate credential encoding/decoding with UTF8 if (!Encoding.UTF8.GetString(Encoding.UTF8.GetBytes(this.Username)).Equals(this.Username)) { From ab1205d699f99887e9a7e59f2bf8b0a692690df6 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Fri, 31 May 2024 16:55:18 +0100 Subject: [PATCH 12/31] Updated start and end configuration Updated start and end configuration to new values --- .../src/Constants/QueryStrings.cs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs index 4aad4aa6f..c9ea685e7 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs @@ -3,8 +3,8 @@ internal class QueryStrings { public const string BalancingAuthorityAbbreviation = "ba"; - public const string StartTime = "starttime"; - public const string EndTime = "endtime"; + public const string StartTime = "start"; + public const string EndTime = "end"; public const string Latitude = "latitude"; public const string Longitude = "longitude"; public const string Username = "username"; From 7c115fa1213cbc042ffbe6e72f208a190b5d7584 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Fri, 31 May 2024 17:22:03 +0100 Subject: [PATCH 13/31] Balancing Authority Parameter Renamed to Region Balancing Authority Renamed to Region. Does not include updates to API, just the Query String parameter. --- .../src/Client/WattTimeClient.cs | 18 +++++++++--------- .../src/Constants/QueryStrings.cs | 2 +- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index 9b17882ae..b87dda6f4 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -46,11 +46,11 @@ public WattTimeClient(IHttpClientFactory factory, IOptionsMonitor> GetDataAsync(string balanc var parameters = new Dictionary() { - { QueryStrings.BalancingAuthorityAbbreviation, balancingAuthorityAbbreviation }, + { QueryStrings.Region, balancingAuthorityAbbreviation }, { QueryStrings.StartTime, startTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) }, { QueryStrings.EndTime, endTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) } }; var tags = new Dictionary() { - { QueryStrings.BalancingAuthorityAbbreviation, balancingAuthorityAbbreviation } + { QueryStrings.Region, balancingAuthorityAbbreviation } }; using (var result = await this.MakeRequestGetStreamAsync(Paths.Data, parameters, tags)) @@ -91,12 +91,12 @@ public async Task GetCurrentForecastAsync(string balancingAuthorityAbb var parameters = new Dictionary() { - { QueryStrings.BalancingAuthorityAbbreviation, balancingAuthorityAbbreviation } + { QueryStrings.Region, balancingAuthorityAbbreviation } }; var tags = new Dictionary() { - { QueryStrings.BalancingAuthorityAbbreviation, balancingAuthorityAbbreviation } + { QueryStrings.Region, balancingAuthorityAbbreviation } }; var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters, tags); @@ -119,14 +119,14 @@ public Task GetCurrentForecastAsync(BalancingAuthority balancingAuthor var parameters = new Dictionary() { - { QueryStrings.BalancingAuthorityAbbreviation, balancingAuthorityAbbreviation }, + { QueryStrings.Region, balancingAuthorityAbbreviation }, { QueryStrings.StartTime, requestedAt.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) }, { QueryStrings.EndTime, requestedAt.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) } }; var tags = new Dictionary() { - { QueryStrings.BalancingAuthorityAbbreviation, balancingAuthorityAbbreviation } + { QueryStrings.Region, balancingAuthorityAbbreviation } }; using (var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters, tags)) { @@ -161,7 +161,7 @@ public async Task GetHistoricalDataAsync(string balancingAuthorityAbbrev var parameters = new Dictionary() { - { QueryStrings.BalancingAuthorityAbbreviation, balancingAuthorityAbbreviation } + { QueryStrings.Region, balancingAuthorityAbbreviation } }; var url = BuildUrlWithQueryString(Paths.Historical, parameters); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs index c9ea685e7..33f15837f 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs @@ -2,7 +2,7 @@ internal class QueryStrings { - public const string BalancingAuthorityAbbreviation = "ba"; + public const string Region = "region"; public const string StartTime = "start"; public const string EndTime = "end"; public const string Latitude = "latitude"; From e047c9a2711ef5aa2e5cb2cbb094fa21914eb8d5 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Mon, 17 Jun 2024 10:05:04 +1000 Subject: [PATCH 14/31] Updates for historical data API Updates for historical data API --- .../decisions/0016-watt-time-v3.md | 2 +- .../mock/WattTimeDataSourceMocker.cs | 20 +++---- .../src/Client/IWattTimeClient.cs | 14 ++--- .../src/Client/WattTimeClient.cs | 55 +++++++++++-------- .../WattTimeClientConfiguration.cs | 2 +- .../src/Constants/Paths.cs | 4 +- .../src/Constants/QueryStrings.cs | 1 + .../src/Constants/SignalTypes.cs | 6 ++ .../src/Model/GridEmissionDataPoint.cs | 12 ---- .../src/Model/GridEmissionsDataResponse.cs | 14 +++++ .../src/Model/GridEmissionsMetaData.cs | 34 ++++++++++++ .../src/Model/GridEmissionsModelData.cs | 16 ++++++ ...alancingAuthority.cs => RegionResponse.cs} | 18 +++--- .../src/WattTimeDataSource.cs | 55 +++++++++---------- .../test/Client/WattTimeClientTests.cs | 44 +++++++-------- .../test/WattTimeDataSourceTests.cs | 16 +++--- .../src/CarbonAware.WebApi.csproj | 11 +++- 17 files changed, 198 insertions(+), 126 deletions(-) create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/SignalTypes.cs create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsMetaData.cs create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsModelData.cs rename src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/{BalancingAuthority.cs => RegionResponse.cs} (51%) diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md index a2b199c64..3ec13fbea 100644 --- a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md +++ b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md @@ -38,7 +38,7 @@ The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/ | Forecast | Get forecast| /forecast | /forecast | **TODO: CHECK IMPACT**
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added | Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** | Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | Check if the CA SDK uses BA at all

    _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | -| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being no longer related to the API version.

    **TODO: CHECK HOW BASE URL IS DEFINED AS THIS WILL NOW HAVE DIFFERENT VALUES** +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being no longer related to the API version.

    NOTE: Updated in wattTime client to now have 2 HTTP clients to decouple versions from the login. ### Query Strings diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs index 21ce7f66e..69191dfb9 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs @@ -13,11 +13,11 @@ internal class WattTimeDataSourceMocker : IDataSourceMocker { protected WireMockServer _server; - private static readonly BalancingAuthority defaultBalancingAuthority = new() + private static readonly RegionResponse defaultBalancingAuthority = new() { Id = 12345, - Abbreviation = "TEST_BA", - Name = "Test Balancing Authority" + Region = "TEST_BA", + RegionFullName = "Test Balancing Authority" }; private static readonly LoginResult defaultLoginResult = new() { Token = "myDefaultToken123" }; @@ -39,11 +39,11 @@ public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string locat { var newDataPoint = new GridEmissionDataPoint() { - BalancingAuthorityAbbreviation = defaultBalancingAuthority.Abbreviation, + Region = defaultBalancingAuthority.Region, PointTime = pointTime, Value = 999.99F, Version = "1.0", - Datatype = "dt", + SignalType = "dt", Frequency = 300, Market = "mkt", }; @@ -70,8 +70,8 @@ public void SetupForecastMock() { var newForecastPoint = new GridEmissionDataPoint() { - BalancingAuthorityAbbreviation = defaultBalancingAuthority.Abbreviation, - Datatype = "dt", + Region = defaultBalancingAuthority.Region, + SignalType = "dt", Frequency = 300, Market = "mkt", PointTime = start, @@ -104,8 +104,8 @@ public void SetupBatchForecastMock() { var newForecastPoint = new GridEmissionDataPoint() { - BalancingAuthorityAbbreviation = defaultBalancingAuthority.Abbreviation, - Datatype = "dt", + Region = defaultBalancingAuthority.Region, + SignalType = "dt", Frequency = 300, Market = "mkt", PointTime = start, @@ -156,7 +156,7 @@ private void SetupResponseGivenGetRequest(string path, string body) .WithBody(body) ); } - private void SetupBaMock(BalancingAuthority? content = null) => + private void SetupBaMock(RegionResponse? content = null) => SetupResponseGivenGetRequest(Paths.BalancingAuthorityFromLocation, JsonSerializer.Serialize(content ?? defaultBalancingAuthority)); private void SetupLoginMock(LoginResult? content = null) => diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs index 4cdf21f18..d20c8d978 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs @@ -8,7 +8,7 @@ namespace CarbonAware.DataSources.WattTime.Client; internal interface IWattTimeClient { public const string NamedClient = "WattTimeClient"; - public const string NamedAuthenticationClient = "WattTimeAuthenticationClient" + public const string NamedAuthenticationClient = "WattTimeAuthenticationClient"; /// /// Async method to get observed emission data for a given balancing authority and time period. @@ -18,7 +18,7 @@ internal interface IWattTimeClient /// End time of the time period /// An which contains all emissions data points in a period. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task> GetDataAsync(string balancingAuthorityAbbreviation, DateTimeOffset startTime, DateTimeOffset endTime); + Task GetDataAsync(string balancingAuthorityAbbreviation, DateTimeOffset startTime, DateTimeOffset endTime); /// /// Async method to get observed emission data for a given balancing authority and time period. @@ -28,7 +28,7 @@ internal interface IWattTimeClient /// End time of the time period /// An which contains all emissions data points in a period. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task> GetDataAsync(BalancingAuthority balancingAuthority, DateTimeOffset startTime, DateTimeOffset endTime); + Task GetDataAsync(RegionResponse balancingAuthority, DateTimeOffset startTime, DateTimeOffset endTime); /// /// Async method to get the most recent 24 hour forecasted emission data for a given balancing authority. @@ -44,7 +44,7 @@ internal interface IWattTimeClient /// Balancing authority /// An which contains forecasted emissions data points. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetCurrentForecastAsync(BalancingAuthority balancingAuthority); + Task GetCurrentForecastAsync(RegionResponse balancingAuthority); /// /// Async method to get generated forecast at requested time and balancing authority. @@ -62,7 +62,7 @@ internal interface IWattTimeClient /// The historical time used to fetch the most recent forecast generated as of that time. /// An which contains forecasted emissions data points or null if no Forecast generated at the requested time. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetForecastOnDateAsync(BalancingAuthority balancingAuthority, DateTimeOffset requestedAt); + Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt); /// /// Async method to get the balancing authority for a given location. @@ -71,7 +71,7 @@ internal interface IWattTimeClient /// Longitude of the location /// An which contains the balancing authority details. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetBalancingAuthorityAsync(string latitude, string longitude); + Task GetBalancingAuthorityAsync(string latitude, string longitude); /// /// Async method to get the balancing authority abbreviation for a given location. @@ -96,5 +96,5 @@ internal interface IWattTimeClient /// Balancing authority /// An which contains the data Stream of the .zip file. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetHistoricalDataAsync(BalancingAuthority balancingAuthority); + Task GetHistoricalDataAsync(RegionResponse balancingAuthority); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index b87dda6f4..f52955abc 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -55,61 +55,66 @@ public WattTimeClient(IHttpClientFactory factory, IOptionsMonitor - public async Task> GetDataAsync(string balancingAuthorityAbbreviation, DateTimeOffset startTime, DateTimeOffset endTime) + public async Task GetDataAsync(string regionAbbreviation, DateTimeOffset startTime, DateTimeOffset endTime) { _log.LogInformation("Requesting grid emission data using start time {startTime} and endTime {endTime}", startTime, endTime); var parameters = new Dictionary() { - { QueryStrings.Region, balancingAuthorityAbbreviation }, + { QueryStrings.Region, regionAbbreviation }, { QueryStrings.StartTime, startTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) }, - { QueryStrings.EndTime, endTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) } + { QueryStrings.EndTime, endTime.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) }, + { QueryStrings.SignalType, SignalTypes.co2_moer}, }; var tags = new Dictionary() { - { QueryStrings.Region, balancingAuthorityAbbreviation } + { QueryStrings.Region, regionAbbreviation } }; using (var result = await this.MakeRequestGetStreamAsync(Paths.Data, parameters, tags)) { - return await JsonSerializer.DeserializeAsync>(result, _options) ?? throw new WattTimeClientException($"Error getting forecasts for {balancingAuthorityAbbreviation}"); + return await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting forecasts for {regionAbbreviation}"); } } /// - public Task> GetDataAsync(BalancingAuthority balancingAuthority, DateTimeOffset startTime, DateTimeOffset endTime) + public Task GetDataAsync(RegionResponse region, DateTimeOffset startTime, DateTimeOffset endTime) { - return this.GetDataAsync(balancingAuthority.Abbreviation, startTime, endTime); + return this.GetDataAsync(region.Region, startTime, endTime); } /// - public async Task GetCurrentForecastAsync(string balancingAuthorityAbbreviation) + public async Task GetCurrentForecastAsync(string region) { - _log.LogInformation("Requesting current forecast from balancing authority {balancingAuthority}", balancingAuthorityAbbreviation); + _log.LogInformation("Requesting current forecast from balancing authority {balancingAuthority}", region); var parameters = new Dictionary() { - { QueryStrings.Region, balancingAuthorityAbbreviation } + { QueryStrings.Region, region }, + { QueryStrings.SignalType, SignalTypes.co2_moer } }; var tags = new Dictionary() { - { QueryStrings.Region, balancingAuthorityAbbreviation } + { QueryStrings.Region, region } }; var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters, tags); - var forecast = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting forecast for {balancingAuthorityAbbreviation}"); + var sr = new StreamReader(result); + var s = sr.ReadToEnd(); + + var forecast = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting forecast for {region}"); return forecast; } /// - public Task GetCurrentForecastAsync(BalancingAuthority balancingAuthority) + public Task GetCurrentForecastAsync(RegionResponse balancingAuthority) { - return this.GetCurrentForecastAsync(balancingAuthority.Abbreviation); + return this.GetCurrentForecastAsync(balancingAuthority.Region); } /// @@ -136,13 +141,13 @@ public Task GetCurrentForecastAsync(BalancingAuthority balancingAuthor } /// - public Task GetForecastOnDateAsync(BalancingAuthority balancingAuthority, DateTimeOffset requestedAt) + public Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt) { - return this.GetForecastOnDateAsync(balancingAuthority.Abbreviation, requestedAt); + return this.GetForecastOnDateAsync(balancingAuthority.Region, requestedAt); } /// - public async Task GetBalancingAuthorityAsync(string latitude, string longitude) + public async Task GetBalancingAuthorityAsync(string latitude, string longitude) { _log.LogInformation("Requesting balancing authority for lattitude {lattitude} and longitude {longitude}", latitude, longitude); return await GetBalancingAuthorityFromCacheAsync(latitude, longitude); @@ -151,7 +156,7 @@ public async Task GetBalancingAuthorityAsync(string latitude /// public async Task GetBalancingAuthorityAbbreviationAsync(string latitude, string longitude) { - return (await this.GetBalancingAuthorityAsync(latitude, longitude))?.Abbreviation; + return (await this.GetBalancingAuthorityAsync(latitude, longitude))?.Region; } /// @@ -176,9 +181,9 @@ public async Task GetHistoricalDataAsync(string balancingAuthorityAbbrev } /// - public Task GetHistoricalDataAsync(BalancingAuthority balancingAuthority) + public Task GetHistoricalDataAsync(RegionResponse balancingAuthority) { - return this.GetHistoricalDataAsync(balancingAuthority.Abbreviation); + return this.GetHistoricalDataAsync(balancingAuthority.Region); } private async Task GetAsyncWithAuthRetry(string uriPath) @@ -290,7 +295,7 @@ private string BuildUrlWithQueryString(string url, IDictionary q return result; } - private async Task GetBalancingAuthorityFromCacheAsync(string latitude, string longitude) + private async Task GetBalancingAuthorityFromCacheAsync(string latitude, string longitude) { var key = new Tuple(latitude, longitude); var balancingAuthority = await this._memoryCache.GetOrCreateAsync(key, async entry => @@ -298,16 +303,18 @@ private async Task GetBalancingAuthorityFromCacheAsync(strin var parameters = new Dictionary() { { QueryStrings.Latitude, latitude }, - { QueryStrings.Longitude, longitude } + { QueryStrings.Longitude, longitude }, + { QueryStrings.SignalType, SignalTypes.co2_moer} }; var tags = new Dictionary() { { QueryStrings.Latitude, latitude }, - { QueryStrings.Longitude, longitude } + { QueryStrings.Longitude, longitude }, + { QueryStrings.SignalType, SignalTypes.co2_moer } }; var result = await this.MakeRequestGetStreamAsync(Paths.BalancingAuthorityFromLocation, parameters, tags); - var baValue = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting Balancing Authority for latitude {latitude} and longitude {longitude}"); + var baValue = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting Balancing Authority for latitude {latitude} and longitude {longitude}"); entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(_configuration.BalancingAuthorityCacheTTL); entry.Value = baValue; return baValue; diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs index 104aeb759..8a8314baf 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs @@ -23,7 +23,7 @@ internal class WattTimeClientConfiguration /// /// Gets or sets the base url to use when connecting to WattTime /// - public string BaseUrl { get; set; } = "https://api2.watttime.org/v2/"; + public string BaseUrl { get; set; } = "https://api.watttime.org/v3/"; /// /// Authentication base url. This changed between v2 and v3 diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs index d564a9c26..61c3092f9 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs @@ -2,9 +2,9 @@ internal class Paths { - public const string Data = "data"; + public const string Data = "historical"; public const string Forecast = "forecast"; - public const string BalancingAuthorityFromLocation = "ba-from-loc"; + public const string BalancingAuthorityFromLocation = "region-from-loc"; public const string Login = "login"; public const string Historical = "historical"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs index 33f15837f..93d32d473 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs @@ -8,4 +8,5 @@ internal class QueryStrings public const string Latitude = "latitude"; public const string Longitude = "longitude"; public const string Username = "username"; + public const string SignalType = "signal_type"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/SignalTypes.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/SignalTypes.cs new file mode 100644 index 000000000..3ff905da5 --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/SignalTypes.cs @@ -0,0 +1,6 @@ +namespace CarbonAware.DataSources.WattTime.Constants; + +internal class SignalTypes +{ + public const string co2_moer = "co2_moer"; +} diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs index 7a9debb83..97eed6b89 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs @@ -8,18 +8,6 @@ namespace CarbonAware.DataSources.WattTime.Model; [Serializable] internal record GridEmissionDataPoint { - /// - /// Balancing authority abbreviation - /// - [JsonPropertyName("ba")] - public string BalancingAuthorityAbbreviation { get; set; } = string.Empty; - - /// - /// Type of data. eg MOER - /// - [JsonPropertyName("datatype")] - public string? Datatype { get; set; } - /// /// Duration in seconds for which the data is valid from point_time. /// diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs new file mode 100644 index 000000000..f95918d83 --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs @@ -0,0 +1,14 @@ +using System.Text.Json.Serialization; + +namespace CarbonAware.DataSources.WattTime.Model; + +[Serializable] +internal record GridEmissionsDataResponse +{ + [JsonPropertyName("data")] + public List Data { get; set; } = new List(); + + + [JsonPropertyName("meta")] + public GridEmissionsMetaData Meta { get; set; } +} \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsMetaData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsMetaData.cs new file mode 100644 index 000000000..7f4768e3a --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsMetaData.cs @@ -0,0 +1,34 @@ +using System.Text.Json.Serialization; + +namespace CarbonAware.DataSources.WattTime.Model; + +[Serializable] +internal record GridEmissionsMetaData +{ + [JsonPropertyName("data_point_periods_second")] + public int DataPointPeriodSeconds { get; set; } + + /// + /// Region (abbreviation) + /// + [JsonPropertyName("region")] + public string Region { get; set; } = string.Empty; + + /// + /// Signal Type. eg MOER + /// + [JsonPropertyName("signal_type")] + public string? SignalType { get; set; } + + [JsonPropertyName("model")] + public GridEmissionsModelData? Model { get; set; } + + [JsonPropertyName("units")] + public string? Units { get; set; } + + [JsonPropertyName("generated_at_period_seconds")] + public int? GeneratedAtPeriodSeconds { get; set; } + + [JsonPropertyName("generated_at")] + public DateTimeOffset? GeneratedAt { get; set; } +} \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsModelData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsModelData.cs new file mode 100644 index 000000000..7830079a7 --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsModelData.cs @@ -0,0 +1,16 @@ +using System.Text.Json.Serialization; + +namespace CarbonAware.DataSources.WattTime.Model; + +/// +/// Data type used to capture the model as part of an EmissionsDataResponse +/// +[Serializable] +internal record GridEmissionsModelData +{ + [JsonPropertyName("date")] + public DateTime Date { get; set; } + + [JsonPropertyName("type")] + public string? Type { get; set; } = String.Empty; +} \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/BalancingAuthority.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/RegionResponse.cs similarity index 51% rename from src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/BalancingAuthority.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/RegionResponse.cs index 7560e0653..19632061d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/BalancingAuthority.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/RegionResponse.cs @@ -6,24 +6,24 @@ namespace CarbonAware.DataSources.WattTime.Model; /// The details of the balancing authority (BA) serving a particular location. /// [Serializable] -internal record BalancingAuthority +internal record RegionResponse { /// - /// Balancing authority abbreviation. + /// Region abbreviation. /// - [JsonPropertyName("abbrev")] - public string Abbreviation { get; set; } = string.Empty; + [JsonPropertyName("region")] + public string Region { get; set; } = string.Empty; /// - /// Unique WattTime id for the region. + /// Signal Type /// - [JsonPropertyName("id")] - public int Id { get; set; } + [JsonPropertyName("signal_type")] + public string SignalType { get; set; } = string.Empty; /// /// Human readable name/description for the region. /// - [JsonPropertyName("name")] - public string Name { get; set; } = string.Empty; + [JsonPropertyName("region_full_name")] + public string RegionFullName { get; set; } = string.Empty; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs index b639bd2e7..cd7d77c82 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs @@ -4,7 +4,6 @@ using CarbonAware.Interfaces; using CarbonAware.Model; using Microsoft.Extensions.Logging; -using System.Diagnostics; namespace CarbonAware.DataSources.WattTime; @@ -48,7 +47,7 @@ public WattTimeDataSource(ILogger logger, IWattTimeClient cl public async Task> GetCarbonIntensityAsync(IEnumerable locations, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) { this.Logger.LogInformation("Getting carbon intensity for locations {locations} for period {periodStartTime} to {periodEndTime}.", locations, periodStartTime, periodEndTime); - List result = new (); + List result = new(); foreach (var location in locations) { IEnumerable interimResult = await GetCarbonIntensityAsync(location, periodStartTime, periodEndTime); @@ -63,12 +62,12 @@ public async Task> GetCarbonIntensityAsync(Location l Logger.LogInformation($"Getting carbon intensity for location {location} for period {periodStartTime} to {periodEndTime}."); var balancingAuthority = await this.GetBalancingAuthority(location); var (newStartTime, newEndTime) = IntervalHelper.ExtendTimeByWindow(periodStartTime, periodEndTime, MinSamplingWindow); - var data = await this.WattTimeClient.GetDataAsync(balancingAuthority, newStartTime, newEndTime); + var historialResponse = await this.WattTimeClient.GetDataAsync(balancingAuthority, newStartTime, newEndTime); if (Logger.IsEnabled(LogLevel.Debug)) { - Logger.LogDebug($"Found {data.Count()} total forecasts for location {location} for period {periodStartTime} to {periodEndTime}."); + Logger.LogDebug($"Found {historialResponse.Data.Count()} total forecasts for location {location} for period {periodStartTime} to {periodEndTime}."); } - var windowData = ConvertToEmissionsData(data); + var windowData = ConvertToEmissionsData(historialResponse); var filteredData = IntervalHelper.FilterByDuration(windowData, periodStartTime, periodEndTime); if (!filteredData.Any()) @@ -83,7 +82,7 @@ public async Task GetCurrentCarbonIntensityForecastAsync(Loca { this.Logger.LogInformation($"Getting carbon intensity forecast for location {location}"); var balancingAuthority = await this.GetBalancingAuthority(location); - var forecast = await this.WattTimeClient.GetCurrentForecastAsync(balancingAuthority); + var forecast = await this.WattTimeClient.GetCurrentForecastAsync(balancingAuthority); return ForecastToEmissionsForecast(forecast, location, DateTimeOffset.UtcNow); } @@ -101,15 +100,15 @@ public async Task GetCarbonIntensityForecastAsync(Location lo throw ex; } // keep input from the user. - return ForecastToEmissionsForecast(forecast, location, requestedAt); + return ForecastToEmissionsForecast(forecast, location, requestedAt); } - private EmissionsForecast ForecastToEmissionsForecast(Forecast forecast, Location location, DateTimeOffset requestedAt) + private EmissionsForecast ForecastToEmissionsForecast(Forecast forecast, Location location, DateTimeOffset requestedAt) { var duration = GetDurationFromGridEmissionDataPoints(forecast.ForecastData); var forecastData = forecast.ForecastData.Select(e => new EmissionsData() { - Location = e.BalancingAuthorityAbbreviation, + Location = "", //e.Region, // TODO: VAUGHAN Rating = ConvertMoerToGramsPerKilowattHour(e.Value), Time = e.PointTime, Duration = duration @@ -128,26 +127,26 @@ internal double ConvertMoerToGramsPerKilowattHour(double value) return value * LBS_TO_GRAMS_CONVERSION_FACTOR / MWH_TO_KWH_CONVERSION_FACTOR; } - private IEnumerable ConvertToEmissionsData(IEnumerable gridEmissionDataPoints) + private IEnumerable ConvertToEmissionsData(GridEmissionsDataResponse gridEmissionDataPoints) { - var defaultDuration = GetDurationFromGridEmissionDataPointsOrDefault(gridEmissionDataPoints, TimeSpan.Zero); - + var defaultDuration = GetDurationFromGridEmissionDataPointsOrDefault(gridEmissionDataPoints.Data, TimeSpan.Zero); + // Linq statement to convert WattTime forecast data into EmissionsData for the CarbonAware SDK. - return gridEmissionDataPoints.Select(e => new EmissionsData() - { - Location = e.BalancingAuthorityAbbreviation, - Rating = ConvertMoerToGramsPerKilowattHour(e.Value), - Time = e.PointTime, - Duration = FrequencyToTimeSpanOrDefault(e.Frequency, defaultDuration) - }); + return gridEmissionDataPoints.Data.Select(e => new EmissionsData() + { + Location = gridEmissionDataPoints.Meta.Region, + Rating = ConvertMoerToGramsPerKilowattHour(e.Value), + Time = e.PointTime, + Duration = FrequencyToTimeSpanOrDefault(e.Frequency, defaultDuration) + }); } private TimeSpan GetDurationFromGridEmissionDataPoints(IEnumerable gridEmissionDataPoints) { - var firstPoint = gridEmissionDataPoints.FirstOrDefault(); + var firstPoint = gridEmissionDataPoints.FirstOrDefault(); var secondPoint = gridEmissionDataPoints.Skip(1)?.FirstOrDefault(); - var first = firstPoint ?? throw new WattTimeClientException("Too few data points returned"); + var first = firstPoint ?? throw new WattTimeClientException("Too few data points returned"); var second = secondPoint ?? throw new WattTimeClientException("Too few data points returned"); // Handle chronological and reverse-chronological data by using `.Duration()` to get @@ -157,13 +156,13 @@ private TimeSpan GetDurationFromGridEmissionDataPoints(IEnumerable gridEmissionDataPoints, TimeSpan defaultValue) { - try + try { return GetDurationFromGridEmissionDataPoints(gridEmissionDataPoints); } - catch (WattTimeClientException) + catch (WattTimeClientException) { - return defaultValue; + return defaultValue; } } @@ -172,21 +171,21 @@ private TimeSpan FrequencyToTimeSpanOrDefault(int? frequency, TimeSpan defaultVa return (frequency != null) ? TimeSpan.FromSeconds((double)frequency) : defaultValue; } - private async Task GetBalancingAuthority(Location location) + private async Task GetBalancingAuthority(Location location) { - BalancingAuthority balancingAuthority; + RegionResponse balancingAuthority; try { var geolocation = await this.LocationSource.ToGeopositionLocationAsync(location); balancingAuthority = await WattTimeClient.GetBalancingAuthorityAsync(geolocation.LatitudeAsCultureInvariantString(), geolocation.LongitudeAsCultureInvariantString()); } - catch(Exception ex) when (ex is LocationConversionException || ex is WattTimeClientHttpException) + catch (Exception ex) when (ex is LocationConversionException || ex is WattTimeClientHttpException) { Logger.LogError(ex, "Failed to convert the location {location} into a Balancing Authority.", location); throw; } - Logger.LogDebug("Converted location {location} to balancing authority {balancingAuthorityAbbreviation}", location, balancingAuthority.Abbreviation); + Logger.LogDebug("Converted location {location} to balancing authority {balancingAuthorityAbbreviation}", location, balancingAuthority.Region); return balancingAuthority; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index eff67195e..fbdbc8ca7 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -85,14 +85,14 @@ public void AllPublicMethods_ThrowClientException_WhenNull() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new BalancingAuthority() { Abbreviation = "balauth" }; + var ba = new RegionResponse() { Region = "balauth" }; Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAsync("lat", "long")); - Assert.ThrowsAsync(async () => await client.GetDataAsync(ba.Abbreviation, new DateTimeOffset(), new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetDataAsync(ba.Region, new DateTimeOffset(), new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetDataAsync(ba, new DateTimeOffset(), new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba.Abbreviation)); + Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba.Region)); Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba)); - Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba.Abbreviation, new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba.Region, new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba, new DateTimeOffset())); } @@ -103,14 +103,14 @@ public void AllPublicMethods_ThrowJsonException_WhenBadJsonIsReturned() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new BalancingAuthority() { Abbreviation = "balauth" }; + var ba = new RegionResponse() { Region = "balauth" }; Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAsync("lat", "long")); - Assert.ThrowsAsync(async () => await client.GetDataAsync(ba.Abbreviation, new DateTimeOffset(), new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetDataAsync(ba.Region, new DateTimeOffset(), new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetDataAsync(ba, new DateTimeOffset(), new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba.Abbreviation)); + Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba.Region)); Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba)); - Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba.Abbreviation, new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba.Region, new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba, new DateTimeOffset())); } @@ -130,8 +130,8 @@ public async Task GetDataAsync_DeserializesExpectedResponse() Assert.IsTrue(data.Count() > 0); var gridDataPoint = data.ToList().First(); - Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); - Assert.AreEqual("dt", gridDataPoint.Datatype); + Assert.AreEqual("ba", gridDataPoint.Region); + Assert.AreEqual("dt", gridDataPoint.SignalType); Assert.AreEqual(300, gridDataPoint.Frequency); Assert.AreEqual("mkt", gridDataPoint.Market); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), gridDataPoint.PointTime); @@ -151,7 +151,7 @@ public async Task GetDataAsync_RefreshesTokenWhenExpired() Assert.IsTrue(data.Count() > 0); var gridDataPoint = data.ToList().First(); - Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); + Assert.AreEqual("ba", gridDataPoint.Region); } [Test] @@ -165,7 +165,7 @@ public async Task GetDataAsync_RefreshesTokenWhenNoneSet() Assert.IsTrue(data.Count() > 0); var gridDataPoint = data.ToList().First(); - Assert.AreEqual("ba", gridDataPoint.BalancingAuthorityAbbreviation); + Assert.AreEqual("ba", gridDataPoint.Region); } [Test] @@ -180,9 +180,9 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new BalancingAuthority() { Abbreviation = "balauth" }; + var ba = new RegionResponse() { Region = "balauth" }; - var forecast = await client.GetCurrentForecastAsync(ba.Abbreviation); + var forecast = await client.GetCurrentForecastAsync(ba.Region); var overloadedForecast = await client.GetCurrentForecastAsync(ba); Assert.AreEqual(forecast.GeneratedAt, overloadedForecast.GeneratedAt); @@ -191,7 +191,7 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() Assert.IsNotNull(forecast); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast?.GeneratedAt); var forecastDataPoint = forecast?.ForecastData.First(); - Assert.AreEqual("ba", forecastDataPoint?.BalancingAuthorityAbbreviation); + Assert.AreEqual("ba", forecastDataPoint?.Region); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint?.PointTime); Assert.AreEqual("999.99", forecastDataPoint?.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", forecastDataPoint?.Version); @@ -210,7 +210,7 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenExpired() Assert.IsNotNull(forecast); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast?.GeneratedAt); var forecastDataPoint = forecast?.ForecastData.First(); - Assert.AreEqual("ba", forecastDataPoint?.BalancingAuthorityAbbreviation); + Assert.AreEqual("ba", forecastDataPoint?.Region); } [Test] @@ -233,7 +233,7 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenNoneSet() Assert.IsNotNull(forecast); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast?.GeneratedAt); var forecastDataPoint = forecast?.ForecastData.First(); - Assert.AreEqual("ba", forecastDataPoint?.BalancingAuthorityAbbreviation); + Assert.AreEqual("ba", forecastDataPoint?.Region); } [Test] @@ -247,9 +247,9 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new BalancingAuthority() { Abbreviation = "balauth" }; + var ba = new RegionResponse() { Region = "balauth" }; - var forecast = await client.GetForecastOnDateAsync(ba.Abbreviation, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); + var forecast = await client.GetForecastOnDateAsync(ba.Region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); var overloadedForecast = await client.GetForecastOnDateAsync(ba, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); Assert.AreEqual(forecast!.GeneratedAt, overloadedForecast!.GeneratedAt); @@ -257,7 +257,7 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast.GeneratedAt); var forecastDataPoint = forecast.ForecastData.ToList().First(); - Assert.AreEqual("ba", forecastDataPoint.BalancingAuthorityAbbreviation); + Assert.AreEqual("ba", forecastDataPoint.Region); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint.PointTime); Assert.AreEqual("999.99", forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", forecastDataPoint.Version); @@ -312,8 +312,8 @@ public async Task GetBalancingAuthorityAsync_DeserializesExpectedResponse() Assert.IsNotNull(ba); Assert.AreEqual(12345, ba?.Id); - Assert.AreEqual("TEST_BA", ba?.Abbreviation); - Assert.AreEqual("Test Balancing Authority", ba?.Name); + Assert.AreEqual("TEST_BA", ba?.Region); + Assert.AreEqual("Test Balancing Authority", ba?.RegionFullName); } [Test] diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs index 9d721cf56..967507b22 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs @@ -25,7 +25,7 @@ class WattTimeDataSourceTests private Mock LocationSource { get; set; } private Location DefaultLocation { get; set; } - private BalancingAuthority DefaultBalancingAuthority { get; set; } + private RegionResponse DefaultBalancingAuthority { get; set; } private DateTimeOffset DefaultDataStartTime { get; set; } // Magic floating point tolerance to allow for minuscule differences in floating point arithmetic. @@ -46,7 +46,7 @@ public void Setup() this.DataSource = new WattTimeDataSource(this.Logger.Object, this.WattTimeClient.Object, this.LocationSource.Object); this.DefaultLocation = new Location() { Name = "eastus" }; - this.DefaultBalancingAuthority = new BalancingAuthority() { Abbreviation = "BA" }; + this.DefaultBalancingAuthority = new RegionResponse() { Region = "BA" }; this.DefaultDataStartTime = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); MockBalancingAuthorityLocationMapping(); } @@ -76,7 +76,7 @@ public async Task GetCarbonIntensity_ReturnsResultsWhenRecordsFound() var first = result.First(); Assert.IsNotNull(first); Assert.AreEqual(gPerKwhEmissions, first.Rating); - Assert.AreEqual(this.DefaultBalancingAuthority.Abbreviation, first.Location); + Assert.AreEqual(this.DefaultBalancingAuthority.Region, first.Location); Assert.AreEqual(startDate, first.Time); this.LocationSource.Verify(r => r.ToGeopositionLocationAsync(this.DefaultLocation)); @@ -158,13 +158,13 @@ public async Task GetCarbonIntensityForecastAsync_ReturnsResultsWhenRecordsFound var lastDataPoint = result.ForecastData.Last(); Assert.IsNotNull(firstDataPoint); Assert.AreEqual(gPerKwhEmissions, firstDataPoint.Rating); - Assert.AreEqual(this.DefaultBalancingAuthority.Abbreviation, firstDataPoint.Location); + Assert.AreEqual(this.DefaultBalancingAuthority.Region, firstDataPoint.Location); Assert.AreEqual(startDate, firstDataPoint.Time); Assert.AreEqual(expectedDuration, firstDataPoint.Duration); Assert.IsNotNull(lastDataPoint); Assert.AreEqual(gPerKwhEmissions, lastDataPoint.Rating); - Assert.AreEqual(this.DefaultBalancingAuthority.Abbreviation, lastDataPoint.Location); + Assert.AreEqual(this.DefaultBalancingAuthority.Region, lastDataPoint.Location); Assert.AreEqual(startDate + expectedDuration, lastDataPoint.Time); Assert.AreEqual(expectedDuration, lastDataPoint.Duration); @@ -235,7 +235,7 @@ public async Task GetCarbonIntensityForecastAsync_RequiredAtRounded(string reque // Assert Assert.IsNotNull(result); this.WattTimeClient.Verify(w => w.GetForecastOnDateAsync( - It.IsAny(), It.Is(date => date.Equals(expectedAt))), Times.Once); + It.IsAny(), It.Is(date => date.Equals(expectedAt))), Times.Once); } [DatapointSource] @@ -274,7 +274,7 @@ public async Task GetCarbonIntensity_CalculatesDurationBasedOnFrequency(double[] List expectedDurationList = durationValues.ToList(); this.WattTimeClient.Setup(w => w.GetDataAsync( - It.IsAny(), + It.IsAny(), It.IsAny(), It.IsAny()) ).ReturnsAsync(() => emissionData); @@ -307,7 +307,7 @@ private List GenerateDataPoints(int numberOfDatapoints, f { var dataPoint = new GridEmissionDataPoint() { - BalancingAuthorityAbbreviation = this.DefaultBalancingAuthority.Abbreviation, + Region = this.DefaultBalancingAuthority.Region, PointTime = pointTime, Value = value, Frequency = defaultFrequency diff --git a/src/CarbonAware.WebApi/src/CarbonAware.WebApi.csproj b/src/CarbonAware.WebApi/src/CarbonAware.WebApi.csproj index de8904092..5bf105b49 100644 --- a/src/CarbonAware.WebApi/src/CarbonAware.WebApi.csproj +++ b/src/CarbonAware.WebApi/src/CarbonAware.WebApi.csproj @@ -14,11 +14,14 @@ 1591 + + + + - + @@ -47,4 +50,8 @@ + + + + From aa81382704ccec7a8da108ef5c293845a9468c8f Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Mon, 17 Jun 2024 10:10:25 +1000 Subject: [PATCH 15/31] Removed accidental file Removed accidental file --- .../decisions/0016-watt-time-v3.md | 62 ------------------- 1 file changed, 62 deletions(-) delete mode 100644 docs/architecture/decisions/0016-watt-time-v3.md diff --git a/docs/architecture/decisions/0016-watt-time-v3.md b/docs/architecture/decisions/0016-watt-time-v3.md deleted file mode 100644 index 661f38262..000000000 --- a/docs/architecture/decisions/0016-watt-time-v3.md +++ /dev/null @@ -1,62 +0,0 @@ - -# 0015. WattTime v3 Changes - -## Status - -Proposed - -## Context -As part of the update to Watt Time v3 we are proposing the changes to the underlying API calls. This needs to be tracked so we understand the impacts, and if multiple options are available, which option was selected and why. - -This wll impact the `CarbonAware.DataSources.WattTime` project primarily. - -## Decision - -The proposal is for the outlined WattTime API mapping and changes. - -## WattTime v2, v3 Mapping - -The following document and guidelines was used to understand the impact to the Carbon Aware SDK for the WattTime v3 updates. https://docs.watttime.org/#tag/Transitioning-from-APIv2-to-APIv3 - -### Base URL -The base URL will need to change. -> TODO: Add where this is configured - -|Base URL (v2) | Based URL (v3) | -|---|---| -| /v2 | /v3 | - - -### Paths -The paths will also need to change. - -The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/Paths.cs` - -| API Endpoint | Description | Path (v2) | Path (v3) | Notes | -|--------------|-------------|-----------|-----------|---| -| Data | Get data | /data | /historical | Parameter changes outlined below. Start and End are now mandatory. | -| Forecast | Get forecast| /forecast | /forecast | Forecast parameters have undergone broader changes, and it can no longer be used for historical data -| Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** -| BalancingAuthorityFromLocation | Get balancing authority from location | /ba-from-loc | /region-from-loc | Parameter changes are only `signal_type` | -| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | No other changes, but note **the path change is not in the same API path for `login` and other paths** | - -### Query Strings - -#### Signal Type -Everything call takes an optional `signal_type` parameter that defaults to `co2_moer`. - -The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` - -| Query String (v2) | Query String (v3) | Description | -|------------------------------------|----------------------------------|------------------------------| -| ba | region | Balancing Authority / Region | -| starttime | start | Start Time | -| endtime | end | End Time | -| latitude | | Latitude | -| longitude | | Longitude | -| username | | Username | - -## Green Impact - -Neutral - From 8640c8cf242b1872701fa46b5ba338d28c09a512 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Mon, 17 Jun 2024 10:54:04 +1000 Subject: [PATCH 16/31] Lots of test updates Lots of test updates, need to do some fixes. --- .../mock/WattTimeDataSourceMocker.cs | 75 +++++++---- .../src/Client/IWattTimeClient.cs | 10 +- .../src/Client/WattTimeClient.cs | 16 +-- .../src/Model/Forecast.cs | 22 ---- .../Model/ForecastEmissionsDataResponse.cs | 16 +++ .../src/Model/GridEmissionsDataResponse.cs | 4 +- .../src/Model/GridEmissionsMetaData.cs | 2 +- .../src/WattTimeDataSource.cs | 12 +- .../test/Client/WattTimeClientTests.cs | 114 ++++++++-------- .../ServiceCollectionExtensionTests.cs | 2 +- .../test/WattTimeDataSourceTests.cs | 122 +++++++++++------- 11 files changed, 219 insertions(+), 176 deletions(-) delete mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/Forecast.cs create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/ForecastEmissionsDataResponse.cs diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs index 69191dfb9..3c95e5350 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs @@ -1,6 +1,6 @@ -using CarbonAware.Interfaces; -using CarbonAware.DataSources.WattTime.Constants; +using CarbonAware.DataSources.WattTime.Constants; using CarbonAware.DataSources.WattTime.Model; +using CarbonAware.Interfaces; using System.Net; using System.Net.Mime; using System.Text.Json; @@ -13,11 +13,11 @@ internal class WattTimeDataSourceMocker : IDataSourceMocker { protected WireMockServer _server; - private static readonly RegionResponse defaultBalancingAuthority = new() + private static readonly RegionResponse defaultRegion = new() { - Id = 12345, - Region = "TEST_BA", - RegionFullName = "Test Balancing Authority" + Region = "TEST_REGION", + RegionFullName = "Test Region Full Name", + SignalType = SignalTypes.co2_moer }; private static readonly LoginResult defaultLoginResult = new() { Token = "myDefaultToken123" }; @@ -39,20 +39,31 @@ public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string locat { var newDataPoint = new GridEmissionDataPoint() { - Region = defaultBalancingAuthority.Region, PointTime = pointTime, Value = 999.99F, Version = "1.0", - SignalType = "dt", Frequency = 300, Market = "mkt", }; + data.Add(newDataPoint); pointTime = newDataPoint.PointTime + duration; } - SetupResponseGivenGetRequest(Paths.Data, JsonSerializer.Serialize(data)); + var meta = new GridEmissionsMetaData() + { + Region = defaultRegion.Region, + SignalType = SignalTypes.co2_moer + }; + + var gridEmissionsResponse = new GridEmissionsDataResponse() + { + Data = data, + Meta = meta + }; + + SetupResponseGivenGetRequest(Paths.Data, JsonSerializer.Serialize(gridEmissionsResponse)); } public void SetupForecastMock() @@ -63,15 +74,13 @@ public void SetupForecastMock() var start = new DateTimeOffset(((curr.Ticks + d.Ticks - 1) / d.Ticks) * d.Ticks, TimeSpan.Zero); var end = start + TimeSpan.FromDays(1.0); var pointTime = start; - var ForecastData = new List(); + var forecastData = new List(); var currValue = 200.0F; while (pointTime < end) { var newForecastPoint = new GridEmissionDataPoint() { - Region = defaultBalancingAuthority.Region, - SignalType = "dt", Frequency = 300, Market = "mkt", PointTime = start, @@ -80,17 +89,26 @@ public void SetupForecastMock() }; newForecastPoint.PointTime = pointTime; newForecastPoint.Value = currValue; - ForecastData.Add(newForecastPoint); + forecastData.Add(newForecastPoint); pointTime = pointTime + TimeSpan.FromMinutes(5); currValue = currValue + 5.0F; } - var forecast = new Forecast() + var meta = new GridEmissionsMetaData() { - ForecastData = ForecastData, + Region = defaultRegion.Region, + SignalType = SignalTypes.co2_moer, GeneratedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero) }; - SetupResponseGivenGetRequest(Paths.Forecast, JsonSerializer.Serialize(forecast)); + + var forecastResponse = new ForecastEmissionsDataResponse() + { + Data = forecastData, + Meta = meta + }; + + + SetupResponseGivenGetRequest(Paths.Forecast, JsonSerializer.Serialize(forecastResponse)); } public void SetupBatchForecastMock() @@ -98,14 +116,12 @@ public void SetupBatchForecastMock() var start = new DateTimeOffset(2021, 9, 1, 8, 30, 0, TimeSpan.Zero); var end = start + TimeSpan.FromDays(1.0); var pointTime = start; - var ForecastData = new List(); + var forecastData = new List(); var currValue = 200.0F; while (pointTime < end) { var newForecastPoint = new GridEmissionDataPoint() { - Region = defaultBalancingAuthority.Region, - SignalType = "dt", Frequency = 300, Market = "mkt", PointTime = start, @@ -114,19 +130,26 @@ public void SetupBatchForecastMock() }; newForecastPoint.PointTime = pointTime; newForecastPoint.Value = currValue; - ForecastData.Add(newForecastPoint); + forecastData.Add(newForecastPoint); pointTime = pointTime + TimeSpan.FromMinutes(5); currValue = currValue + 5.0F; } - var forecastData = new List { - new Forecast() + var meta = new GridEmissionsMetaData() + { + Region = defaultRegion.Region, + SignalType = SignalTypes.co2_moer, + GeneratedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero) + }; + + var forecastBatchData = new List { + new ForecastEmissionsDataResponse() { - ForecastData = ForecastData, - GeneratedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero) + Data = forecastData, + Meta = meta } }; - SetupResponseGivenGetRequest(Paths.Forecast, JsonSerializer.Serialize(forecastData)); + SetupResponseGivenGetRequest(Paths.Forecast, JsonSerializer.Serialize(forecastBatchData)); } public void Initialize() @@ -157,7 +180,7 @@ private void SetupResponseGivenGetRequest(string path, string body) ); } private void SetupBaMock(RegionResponse? content = null) => - SetupResponseGivenGetRequest(Paths.BalancingAuthorityFromLocation, JsonSerializer.Serialize(content ?? defaultBalancingAuthority)); + SetupResponseGivenGetRequest(Paths.BalancingAuthorityFromLocation, JsonSerializer.Serialize(content ?? defaultRegion)); private void SetupLoginMock(LoginResult? content = null) => SetupResponseGivenGetRequest(Paths.Login, JsonSerializer.Serialize(content ?? defaultLoginResult)); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs index d20c8d978..b07f96cd8 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs @@ -36,7 +36,7 @@ internal interface IWattTimeClient /// Balancing authority abbreviation /// An which contains forecasted emissions data points. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetCurrentForecastAsync(string balancingAuthorityAbbreviation); + Task GetCurrentForecastAsync(string balancingAuthorityAbbreviation); /// /// Async method to get the most recent 24 hour forecasted emission data for a given balancing authority. @@ -44,7 +44,7 @@ internal interface IWattTimeClient /// Balancing authority /// An which contains forecasted emissions data points. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetCurrentForecastAsync(RegionResponse balancingAuthority); + Task GetCurrentForecastAsync(RegionResponse balancingAuthority); /// /// Async method to get generated forecast at requested time and balancing authority. @@ -53,7 +53,7 @@ internal interface IWattTimeClient /// The historical time used to fetch the most recent forecast generated as of that time. /// An which contains forecasted emissions data points or null if no Forecast generated at the requested time. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetForecastOnDateAsync(string balancingAuthorityAbbreviation, DateTimeOffset requestedAt); + Task GetForecastOnDateAsync(string balancingAuthorityAbbreviation, DateTimeOffset requestedAt); /// /// Async method to get generated forecast at requested time and balancing authority. @@ -62,7 +62,7 @@ internal interface IWattTimeClient /// The historical time used to fetch the most recent forecast generated as of that time. /// An which contains forecasted emissions data points or null if no Forecast generated at the requested time. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt); + Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt); /// /// Async method to get the balancing authority for a given location. @@ -71,7 +71,7 @@ internal interface IWattTimeClient /// Longitude of the location /// An which contains the balancing authority details. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetBalancingAuthorityAsync(string latitude, string longitude); + Task GetRegionAsync(string latitude, string longitude); /// /// Async method to get the balancing authority abbreviation for a given location. diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index f52955abc..8e02c7349 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -85,7 +85,7 @@ public Task GetDataAsync(RegionResponse region, DateT } /// - public async Task GetCurrentForecastAsync(string region) + public async Task GetCurrentForecastAsync(string region) { _log.LogInformation("Requesting current forecast from balancing authority {balancingAuthority}", region); @@ -106,19 +106,19 @@ public async Task GetCurrentForecastAsync(string region) var sr = new StreamReader(result); var s = sr.ReadToEnd(); - var forecast = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting forecast for {region}"); + var forecast = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting forecast for {region}"); return forecast; } /// - public Task GetCurrentForecastAsync(RegionResponse balancingAuthority) + public Task GetCurrentForecastAsync(RegionResponse balancingAuthority) { return this.GetCurrentForecastAsync(balancingAuthority.Region); } /// - public async Task GetForecastOnDateAsync(string balancingAuthorityAbbreviation, DateTimeOffset requestedAt) + public async Task GetForecastOnDateAsync(string balancingAuthorityAbbreviation, DateTimeOffset requestedAt) { _log.LogInformation($"Requesting forecast from balancingAuthority {balancingAuthorityAbbreviation} generated at {requestedAt}."); @@ -135,19 +135,19 @@ public Task GetCurrentForecastAsync(RegionResponse balancingAuthority) }; using (var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters, tags)) { - var forecasts = await JsonSerializer.DeserializeAsync>(result, _options) ?? throw new WattTimeClientException($"Error getting forecasts for {balancingAuthorityAbbreviation}"); + var forecasts = await JsonSerializer.DeserializeAsync>(result, _options) ?? throw new WattTimeClientException($"Error getting forecasts for {balancingAuthorityAbbreviation}"); return forecasts.FirstOrDefault(); } } /// - public Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt) + public Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt) { return this.GetForecastOnDateAsync(balancingAuthority.Region, requestedAt); } /// - public async Task GetBalancingAuthorityAsync(string latitude, string longitude) + public async Task GetRegionAsync(string latitude, string longitude) { _log.LogInformation("Requesting balancing authority for lattitude {lattitude} and longitude {longitude}", latitude, longitude); return await GetBalancingAuthorityFromCacheAsync(latitude, longitude); @@ -156,7 +156,7 @@ public async Task GetBalancingAuthorityAsync(string latitude, st /// public async Task GetBalancingAuthorityAbbreviationAsync(string latitude, string longitude) { - return (await this.GetBalancingAuthorityAsync(latitude, longitude))?.Region; + return (await this.GetRegionAsync(latitude, longitude))?.Region; } /// diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/Forecast.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/Forecast.cs deleted file mode 100644 index e7c5209e6..000000000 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/Forecast.cs +++ /dev/null @@ -1,22 +0,0 @@ -using System.Text.Json.Serialization; - -namespace CarbonAware.DataSources.WattTime.Model; - -/// -/// An emissions forecast for a given time period. -/// -[Serializable] -internal record Forecast -{ - /// - /// DateTime indicating when the forecast was generated. - /// - [JsonPropertyName("generated_at")] - public DateTimeOffset GeneratedAt { get; set; } - - /// - /// List of GridEmissionDataPoints representing the predicted values for those points in time. - /// - [JsonPropertyName("forecast")] - public List ForecastData { get; set; } = new List(); -} diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/ForecastEmissionsDataResponse.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/ForecastEmissionsDataResponse.cs new file mode 100644 index 000000000..9b4bd6597 --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/ForecastEmissionsDataResponse.cs @@ -0,0 +1,16 @@ +using System.Text.Json.Serialization; + +namespace CarbonAware.DataSources.WattTime.Model; + +[Serializable] +internal record ForecastEmissionsDataResponse +{ + [JsonPropertyName("data")] + public List Data { get; set; } = new List(); + + + [JsonPropertyName("meta")] + public GridEmissionsMetaData Meta { get; set; } +} + + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs index f95918d83..e50eccb80 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsDataResponse.cs @@ -11,4 +11,6 @@ internal record GridEmissionsDataResponse [JsonPropertyName("meta")] public GridEmissionsMetaData Meta { get; set; } -} \ No newline at end of file +} + + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsMetaData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsMetaData.cs index 7f4768e3a..d5abb4117 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsMetaData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionsMetaData.cs @@ -30,5 +30,5 @@ internal record GridEmissionsMetaData public int? GeneratedAtPeriodSeconds { get; set; } [JsonPropertyName("generated_at")] - public DateTimeOffset? GeneratedAt { get; set; } + public DateTimeOffset GeneratedAt { get; set; } = DateTimeOffset.MinValue; } \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs index cd7d77c82..c879df636 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs @@ -103,19 +103,19 @@ public async Task GetCarbonIntensityForecastAsync(Location lo return ForecastToEmissionsForecast(forecast, location, requestedAt); } - private EmissionsForecast ForecastToEmissionsForecast(Forecast forecast, Location location, DateTimeOffset requestedAt) + private EmissionsForecast ForecastToEmissionsForecast(ForecastEmissionsDataResponse forecast, Location location, DateTimeOffset requestedAt) { - var duration = GetDurationFromGridEmissionDataPoints(forecast.ForecastData); - var forecastData = forecast.ForecastData.Select(e => new EmissionsData() + var duration = GetDurationFromGridEmissionDataPoints(forecast.Data); + var forecastData = forecast.Data.Select(e => new EmissionsData() { - Location = "", //e.Region, // TODO: VAUGHAN + Location = forecast.Meta.Region, Rating = ConvertMoerToGramsPerKilowattHour(e.Value), Time = e.PointTime, Duration = duration }); var emissionsForecast = new EmissionsForecast() { - GeneratedAt = forecast.GeneratedAt, + GeneratedAt = forecast.Meta.GeneratedAt, Location = location, ForecastData = forecastData }; @@ -177,7 +177,7 @@ private async Task GetBalancingAuthority(Location location) try { var geolocation = await this.LocationSource.ToGeopositionLocationAsync(location); - balancingAuthority = await WattTimeClient.GetBalancingAuthorityAsync(geolocation.LatitudeAsCultureInvariantString(), geolocation.LongitudeAsCultureInvariantString()); + balancingAuthority = await WattTimeClient.GetRegionAsync(geolocation.LatitudeAsCultureInvariantString(), geolocation.LongitudeAsCultureInvariantString()); } catch (Exception ex) when (ex is LocationConversionException || ex is WattTimeClientHttpException) { diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index fbdbc8ca7..a79ca8b56 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -1,4 +1,5 @@ using CarbonAware.DataSources.WattTime.Configuration; +using CarbonAware.DataSources.WattTime.Constants; using CarbonAware.DataSources.WattTime.Model; using Microsoft.Extensions.Caching.Memory; using Microsoft.Extensions.Logging; @@ -73,7 +74,7 @@ public void AllPublicMethods_ThrowsWhenInvalidLogin() Assert.ThrowsAsync(async () => await client.GetDataAsync("ba", new DateTimeOffset(), new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync("ba")); Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync("ba", new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAsync("lat", "long")); + Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAbbreviationAsync("lat", "long")); Assert.ThrowsAsync(async () => await client.GetHistoricalDataAsync("ba")); } @@ -87,7 +88,7 @@ public void AllPublicMethods_ThrowClientException_WhenNull() client.SetBearerAuthenticationHeader(this.DefaultTokenValue); var ba = new RegionResponse() { Region = "balauth" }; - Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAsync("lat", "long")); + Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); Assert.ThrowsAsync(async () => await client.GetDataAsync(ba.Region, new DateTimeOffset(), new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetDataAsync(ba, new DateTimeOffset(), new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba.Region)); @@ -105,7 +106,7 @@ public void AllPublicMethods_ThrowJsonException_WhenBadJsonIsReturned() client.SetBearerAuthenticationHeader(this.DefaultTokenValue); var ba = new RegionResponse() { Region = "balauth" }; - Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAsync("lat", "long")); + Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); Assert.ThrowsAsync(async () => await client.GetDataAsync(ba.Region, new DateTimeOffset(), new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetDataAsync(ba, new DateTimeOffset(), new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba.Region)); @@ -126,12 +127,13 @@ public async Task GetDataAsync_DeserializesExpectedResponse() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var data = await client.GetDataAsync("balauth", new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); + var emissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); - Assert.IsTrue(data.Count() > 0); - var gridDataPoint = data.ToList().First(); - Assert.AreEqual("ba", gridDataPoint.Region); - Assert.AreEqual("dt", gridDataPoint.SignalType); + Assert.IsTrue(emissionsResponse.Data.Count() > 0); + var meta = emissionsResponse.Meta; + Assert.AreEqual("region", meta.Region); + Assert.AreEqual("signal_type", meta.SignalType); + var gridDataPoint = emissionsResponse.Data.ToList().First(); Assert.AreEqual(300, gridDataPoint.Frequency); Assert.AreEqual("mkt", gridDataPoint.Market); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), gridDataPoint.PointTime); @@ -147,11 +149,10 @@ public async Task GetDataAsync_RefreshesTokenWhenExpired() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var data = await client.GetDataAsync("balauth", new DateTimeOffset(), new DateTimeOffset()); + var emissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(), new DateTimeOffset()); - Assert.IsTrue(data.Count() > 0); - var gridDataPoint = data.ToList().First(); - Assert.AreEqual("ba", gridDataPoint.Region); + Assert.IsTrue(emissionsResponse.Data.Count() > 0); + Assert.AreEqual("region", emissionsResponse.Meta.Region); } [Test] @@ -161,11 +162,10 @@ public async Task GetDataAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var data = await client.GetDataAsync("balauth", new DateTimeOffset(), new DateTimeOffset()); + var gridEmissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(), new DateTimeOffset()); - Assert.IsTrue(data.Count() > 0); - var gridDataPoint = data.ToList().First(); - Assert.AreEqual("ba", gridDataPoint.Region); + Assert.IsTrue(gridEmissionsResponse.Data.Count() > 0); + Assert.AreEqual("region", gridEmissionsResponse.Meta.Region); } [Test] @@ -180,18 +180,19 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new RegionResponse() { Region = "balauth" }; + var regionResponse = new RegionResponse() { Region = "region" }; + + var forecastResponse = await client.GetCurrentForecastAsync(regionResponse.Region); + var overloadedForecast = await client.GetCurrentForecastAsync(regionResponse); - var forecast = await client.GetCurrentForecastAsync(ba.Region); - var overloadedForecast = await client.GetCurrentForecastAsync(ba); + Assert.AreEqual(forecastResponse.Meta.GeneratedAt, overloadedForecast.Meta.GeneratedAt); + Assert.AreEqual(forecastResponse.Data.First(), overloadedForecast.Data.First()); - Assert.AreEqual(forecast.GeneratedAt, overloadedForecast.GeneratedAt); - Assert.AreEqual(forecast.ForecastData.First(), overloadedForecast.ForecastData.First()); + Assert.IsNotNull(forecastResponse); + Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); + Assert.AreEqual("region", forecastResponse?.Meta.Region); + var forecastDataPoint = forecastResponse?.Data.First(); - Assert.IsNotNull(forecast); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast?.GeneratedAt); - var forecastDataPoint = forecast?.ForecastData.First(); - Assert.AreEqual("ba", forecastDataPoint?.Region); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint?.PointTime); Assert.AreEqual("999.99", forecastDataPoint?.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", forecastDataPoint?.Version); @@ -205,12 +206,11 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenExpired() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var forecast = await client.GetCurrentForecastAsync("balauth"); + var forecastResponse = await client.GetCurrentForecastAsync("region"); - Assert.IsNotNull(forecast); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast?.GeneratedAt); - var forecastDataPoint = forecast?.ForecastData.First(); - Assert.AreEqual("ba", forecastDataPoint?.Region); + Assert.IsNotNull(forecastResponse); + Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); + Assert.AreEqual("region", forecastResponse?.Meta.Region); } [Test] @@ -228,12 +228,11 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var forecast = await client.GetCurrentForecastAsync("balauth"); + var forecastResponse = await client.GetCurrentForecastAsync("region"); - Assert.IsNotNull(forecast); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast?.GeneratedAt); - var forecastDataPoint = forecast?.ForecastData.First(); - Assert.AreEqual("ba", forecastDataPoint?.Region); + Assert.IsNotNull(forecastResponse); + Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); + Assert.AreEqual("region", forecastResponse?.Meta.Region); } [Test] @@ -247,17 +246,18 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new RegionResponse() { Region = "balauth" }; + var ba = new RegionResponse() { Region = "region" }; - var forecast = await client.GetForecastOnDateAsync(ba.Region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); + var forecastResponse = await client.GetForecastOnDateAsync(ba.Region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); var overloadedForecast = await client.GetForecastOnDateAsync(ba, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); - Assert.AreEqual(forecast!.GeneratedAt, overloadedForecast!.GeneratedAt); - Assert.AreEqual(forecast.ForecastData.First(), overloadedForecast.ForecastData.First()); + Assert.AreEqual(forecastResponse!.Meta.GeneratedAt, overloadedForecast!.Meta.GeneratedAt); + Assert.AreEqual(forecastResponse.Data.First(), overloadedForecast.Data.First()); + + Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse.Meta.GeneratedAt); + Assert.AreEqual("region", forecastResponse.Meta.Region); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast.GeneratedAt); - var forecastDataPoint = forecast.ForecastData.ToList().First(); - Assert.AreEqual("ba", forecastDataPoint.Region); + var forecastDataPoint = forecastResponse.Data.ToList().First(); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint.PointTime); Assert.AreEqual("999.99", forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", forecastDataPoint.Version); @@ -271,8 +271,8 @@ public async Task GetForecastOnDateAsync_RefreshesTokenWhenExpired() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var forecast = await client.GetForecastOnDateAsync("balauth", new DateTimeOffset()); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast!.GeneratedAt); + var forecastResponse = await client.GetForecastOnDateAsync("region", new DateTimeOffset()); + Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse!.Meta.GeneratedAt); } [Test] @@ -291,9 +291,9 @@ public async Task GetForecastOnDateAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var forecast = await client.GetForecastOnDateAsync("balauth", new DateTimeOffset()); + var forecastResponse = await client.GetForecastOnDateAsync("region", new DateTimeOffset()); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecast!.GeneratedAt); + Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse!.Meta.GeneratedAt); } [Test] @@ -308,12 +308,12 @@ public async Task GetBalancingAuthorityAsync_DeserializesExpectedResponse() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = await client.GetBalancingAuthorityAsync("lat", "long"); + var regionResponse = await client.GetRegionAsync("lat", "long"); - Assert.IsNotNull(ba); - Assert.AreEqual(12345, ba?.Id); - Assert.AreEqual("TEST_BA", ba?.Region); - Assert.AreEqual("Test Balancing Authority", ba?.RegionFullName); + Assert.IsNotNull(regionResponse); + Assert.AreEqual("TEST_BA", regionResponse?.Region); + Assert.AreEqual("Test Balancing Authority", regionResponse?.RegionFullName); + Assert.AreEqual(SignalTypes.co2_moer, regionResponse?.SignalType); } [Test] @@ -324,10 +324,10 @@ public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenExpired() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = await client.GetBalancingAuthorityAsync("lat", "long"); + var regionResponse = await client.GetRegionAsync("lat", "long"); - Assert.IsNotNull(ba); - Assert.AreEqual(12345, ba?.Id); + Assert.IsNotNull(regionResponse); + Assert.AreEqual(SignalTypes.co2_moer, regionResponse?.SignalType); } [Test] @@ -346,10 +346,10 @@ public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var ba = await client.GetBalancingAuthorityAsync("lat", "long"); + var regionResponse = await client.GetRegionAsync("lat", "long"); - Assert.IsNotNull(ba); - Assert.AreEqual(12345, ba?.Id); + Assert.IsNotNull(regionResponse); + Assert.AreEqual(SignalTypes.co2_moer, regionResponse?.SignalType); } [Test] diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs index b3f845e3a..653b2f18e 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs @@ -49,7 +49,7 @@ public void ClientProxyTest_With_Invalid_ProxyURL_ThrowsException() var client = serviceProvider.GetRequiredService(); // Act & Assert - Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAsync("lat", "long")); + Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); } [Test] diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs index 967507b22..0ddf84dcb 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs @@ -1,4 +1,5 @@ using CarbonAware.DataSources.WattTime.Client; +using CarbonAware.DataSources.WattTime.Constants; using CarbonAware.DataSources.WattTime.Model; using CarbonAware.Exceptions; using CarbonAware.Interfaces; @@ -25,7 +26,7 @@ class WattTimeDataSourceTests private Mock LocationSource { get; set; } private Location DefaultLocation { get; set; } - private RegionResponse DefaultBalancingAuthority { get; set; } + private RegionResponse DefaultRegion { get; set; } private DateTimeOffset DefaultDataStartTime { get; set; } // Magic floating point tolerance to allow for minuscule differences in floating point arithmetic. @@ -46,7 +47,7 @@ public void Setup() this.DataSource = new WattTimeDataSource(this.Logger.Object, this.WattTimeClient.Object, this.LocationSource.Object); this.DefaultLocation = new Location() { Name = "eastus" }; - this.DefaultBalancingAuthority = new RegionResponse() { Region = "BA" }; + this.DefaultRegion = new RegionResponse() { Region = "BA" }; this.DefaultDataStartTime = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); MockBalancingAuthorityLocationMapping(); } @@ -60,13 +61,13 @@ public async Task GetCarbonIntensity_ReturnsResultsWhenRecordsFound() var lbsPerMwhEmissions = 10; var gPerKwhEmissions = this.DataSource.ConvertMoerToGramsPerKilowattHour(lbsPerMwhEmissions); - var emissionData = GenerateDataPoints(1, value: lbsPerMwhEmissions); + var emissionDataResponse = GenerateGridEmissionsResponse(1, value: lbsPerMwhEmissions); this.WattTimeClient.Setup(w => w.GetDataAsync( - this.DefaultBalancingAuthority, + this.DefaultRegion, It.IsAny(), It.IsAny()) - ).ReturnsAsync(() => emissionData); + ).ReturnsAsync(() => emissionDataResponse); var result = await this.DataSource.GetCarbonIntensityAsync(new List() { this.DefaultLocation }, startDate, endDate); @@ -76,7 +77,7 @@ public async Task GetCarbonIntensity_ReturnsResultsWhenRecordsFound() var first = result.First(); Assert.IsNotNull(first); Assert.AreEqual(gPerKwhEmissions, first.Rating); - Assert.AreEqual(this.DefaultBalancingAuthority.Region, first.Location); + Assert.AreEqual(this.DefaultRegion.Region, first.Location); Assert.AreEqual(startDate, first.Time); this.LocationSource.Verify(r => r.ToGeopositionLocationAsync(this.DefaultLocation)); @@ -89,10 +90,10 @@ public async Task GetCarbonIntensity_ReturnsEmptyListWhenNoRecordsFound() var endDate = startDate.AddMinutes(1); this.WattTimeClient.Setup(w => w.GetDataAsync( - this.DefaultBalancingAuthority, + this.DefaultRegion, startDate, endDate) - ).ReturnsAsync(() => new List()); + ).ReturnsAsync(() => new GridEmissionsDataResponse()); var result = await this.DataSource.GetCarbonIntensityAsync(new List() { this.DefaultLocation }, startDate, endDate); @@ -123,27 +124,23 @@ public async Task GetCarbonIntensityForecastAsync_ReturnsResultsWhenRecordsFound var gPerKwhEmissions = this.DataSource.ConvertMoerToGramsPerKilowattHour(lbsPerMwhEmissions); var expectedDuration = TimeSpan.FromMinutes(5); - var emissionData = GenerateDataPoints(2, value: lbsPerMwhEmissions); - var forecast = new Forecast() - { - GeneratedAt = generatedAt, - ForecastData = emissionData - }; + var forecastResponse = GenerateForecastResponse(2, value: lbsPerMwhEmissions); + forecastResponse.Meta.GeneratedAt = generatedAt; EmissionsForecast result; if (getCurrentForecast) { - this.WattTimeClient.Setup(w => w.GetCurrentForecastAsync(this.DefaultBalancingAuthority) - ).ReturnsAsync(() => forecast); + this.WattTimeClient.Setup(w => w.GetCurrentForecastAsync(this.DefaultRegion) + ).ReturnsAsync(() => forecastResponse); // Act result = await this.DataSource.GetCurrentCarbonIntensityForecastAsync(this.DefaultLocation); } else { - this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultBalancingAuthority, generatedAt) - ).ReturnsAsync(() => forecast); + this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultRegion, generatedAt) + ).ReturnsAsync(() => forecastResponse); // Act result = await this.DataSource.GetCarbonIntensityForecastAsync(this.DefaultLocation, generatedAt); @@ -158,13 +155,13 @@ public async Task GetCarbonIntensityForecastAsync_ReturnsResultsWhenRecordsFound var lastDataPoint = result.ForecastData.Last(); Assert.IsNotNull(firstDataPoint); Assert.AreEqual(gPerKwhEmissions, firstDataPoint.Rating); - Assert.AreEqual(this.DefaultBalancingAuthority.Region, firstDataPoint.Location); + Assert.AreEqual(this.DefaultRegion.Region, firstDataPoint.Location); Assert.AreEqual(startDate, firstDataPoint.Time); Assert.AreEqual(expectedDuration, firstDataPoint.Duration); Assert.IsNotNull(lastDataPoint); Assert.AreEqual(gPerKwhEmissions, lastDataPoint.Rating); - Assert.AreEqual(this.DefaultBalancingAuthority.Region, lastDataPoint.Location); + Assert.AreEqual(this.DefaultRegion.Region, lastDataPoint.Location); Assert.AreEqual(startDate + expectedDuration, lastDataPoint.Time); Assert.AreEqual(expectedDuration, lastDataPoint.Duration); @@ -185,7 +182,7 @@ public void GetCarbonIntensityForecastAsync_ThrowsWhenNoForecastFoundForReuqeste { var generatedAt = new DateTimeOffset(); - this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultBalancingAuthority, generatedAt)).Returns(Task.FromResult(null)); + this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultRegion, generatedAt)).Returns(Task.FromResult(null)); // The datasource throws an exception if no forecasts are found at the requested generatedAt time. Assert.ThrowsAsync(async () => await this.DataSource.GetCarbonIntensityForecastAsync(this.DefaultLocation, generatedAt)); @@ -196,16 +193,11 @@ public void GetCarbonIntensityForecastAsync_ThrowsWhenNoForecastFoundForReuqeste public void GetCurrentCarbonIntensityForecastAsync_ThrowsWhenTooFewDatapointsReturned(int numDataPoints) { // Arrange - var emissionData = GenerateDataPoints(numDataPoints); - - var forecast = new Forecast() - { - GeneratedAt = DateTimeOffset.Now, - ForecastData = emissionData - }; + var forecastResponse = GenerateForecastResponse(numDataPoints); + forecastResponse.Meta.GeneratedAt = DateTimeOffset.Now; - this.WattTimeClient.Setup(w => w.GetCurrentForecastAsync(this.DefaultBalancingAuthority) - ).ReturnsAsync(() => forecast); + this.WattTimeClient.Setup(w => w.GetCurrentForecastAsync(this.DefaultRegion) + ).ReturnsAsync(() => forecastResponse); Assert.ThrowsAsync(async () => await this.DataSource.GetCurrentCarbonIntensityForecastAsync(this.DefaultLocation)); } @@ -219,19 +211,16 @@ public async Task GetCarbonIntensityForecastAsync_RequiredAtRounded(string reque var requestedAt = DateTimeOffset.Parse(requested); var expectedAt = DateTimeOffset.Parse(expected); - var emissionData = GenerateDataPoints(2, startTime: requestedAt); - var forecast = new Forecast() - { - GeneratedAt = expectedAt, - ForecastData = emissionData - }; + var forecastResponse = GenerateForecastResponse(2, startTime: requestedAt); + forecastResponse.Meta.GeneratedAt = expectedAt; + - this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultBalancingAuthority, expectedAt) - ).ReturnsAsync(() => forecast); + this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultRegion, expectedAt) + ).ReturnsAsync(() => forecastResponse); // Act var result = await this.DataSource.GetCarbonIntensityForecastAsync(this.DefaultLocation, requestedAt); - + // Assert Assert.IsNotNull(result); this.WattTimeClient.Verify(w => w.GetForecastOnDateAsync( @@ -239,7 +228,7 @@ public async Task GetCarbonIntensityForecastAsync_RequiredAtRounded(string reque } [DatapointSource] - public float[] moerValues = new float[] { 0.0F, 10.0F, 100.0F, 1000.0F, 596.1367F}; + public float[] moerValues = new float[] { 0.0F, 10.0F, 100.0F, 1000.0F, 596.1367F }; [Theory] public void GetCarbonIntensity_ConvertsMoerToGramsPerKwh(float lbsPerMwhEmissions) @@ -265,26 +254,26 @@ public async Task GetCarbonIntensity_CalculatesDurationBasedOnFrequency(double[] // Arrange var startDate = this.DefaultDataStartTime; var endDate = startDate.AddMinutes(10); - var emissionData = GenerateDataPoints(frequencyValues.Length); - for( int i = 0; i < frequencyValues.Length; i++) + var emissionResponse = GenerateGridEmissionsResponse(frequencyValues.Length); + for (int i = 0; i < frequencyValues.Length; i++) { - emissionData[i].Frequency = frequencyValues[i]; + emissionResponse.Data[i].Frequency = frequencyValues[i]; } - + List expectedDurationList = durationValues.ToList(); this.WattTimeClient.Setup(w => w.GetDataAsync( It.IsAny(), It.IsAny(), It.IsAny()) - ).ReturnsAsync(() => emissionData); + ).ReturnsAsync(() => emissionResponse); // Act var result = await this.DataSource.GetCarbonIntensityAsync(new List() { this.DefaultLocation }, startDate, endDate); // Assert List actualDurationList = result.Select(e => e.Duration.TotalSeconds).ToList(); - + CollectionAssert.AreEqual(expectedDurationList, actualDurationList); } @@ -294,8 +283,44 @@ private void MockBalancingAuthorityLocationMapping() var latitude = this.DefaultLocation.Latitude.ToString(); var longitude = this.DefaultLocation.Longitude.ToString(); - this.WattTimeClient.Setup(w => w.GetBalancingAuthorityAsync(latitude!, longitude!) - ).ReturnsAsync(() => this.DefaultBalancingAuthority); + this.WattTimeClient.Setup(w => w.GetRegionAsync(latitude!, longitude!) + ).ReturnsAsync(() => this.DefaultRegion); + } + + private GridEmissionsDataResponse GenerateGridEmissionsResponse(int numberOfDatapoints, float value = 10, DateTimeOffset startTime = default) + { + var data = GenerateDataPoints(numberOfDatapoints, value, startTime); + var meta = new GridEmissionsMetaData() + { + Region = this.DefaultRegion.Region, + SignalType = SignalTypes.co2_moer + }; + + var response = new GridEmissionsDataResponse() + { + Data = data, + Meta = meta + }; + + return response; + } + + private ForecastEmissionsDataResponse GenerateForecastResponse(int numberOfDatapoints, float value = 10, DateTimeOffset startTime = default) + { + var data = GenerateDataPoints(numberOfDatapoints, value, startTime); + var meta = new GridEmissionsMetaData() + { + Region = this.DefaultRegion.Region, + SignalType = SignalTypes.co2_moer + }; + + var response = new ForecastEmissionsDataResponse() + { + Data = data, + Meta = meta + }; + + return response; } private List GenerateDataPoints(int numberOfDatapoints, float value = 10, DateTimeOffset startTime = default) @@ -307,7 +332,6 @@ private List GenerateDataPoints(int numberOfDatapoints, f { var dataPoint = new GridEmissionDataPoint() { - Region = this.DefaultBalancingAuthority.Region, PointTime = pointTime, Value = value, Frequency = defaultFrequency From 880fcf78c1e29342b17086bce6375d31a3dbb224 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Mon, 17 Jun 2024 13:22:09 +1000 Subject: [PATCH 17/31] Historical forecasts updated Historical forecasts updated --- .../src/ElectricityMapsDataSource.cs | 2 +- .../src/Client/IWattTimeClient.cs | 6 +-- .../src/Client/WattTimeClient.cs | 22 +++++----- .../src/Constants/Paths.cs | 2 + .../src/Model/HistoricalEmissionsData.cs | 15 +++++++ ...HistoricalForecastEmissionsDataResponse.cs | 16 +++++++ .../src/WattTimeDataSource.cs | 23 +++++++++- .../test/Client/WattTimeClientTests.cs | 4 +- .../test/WattTimeDataSourceTests.cs | 44 +++++++++++++++---- .../src/Interfaces/IForecastDataSource.cs | 2 +- src/CarbonAware/src/NullForecastDataSource.cs | 2 +- .../src/Handlers/ForecastHandler.cs | 2 +- .../test/Handlers/ForecastHandlerTests.cs | 4 +- 13 files changed, 111 insertions(+), 33 deletions(-) create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/HistoricalEmissionsData.cs create mode 100644 src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/HistoricalForecastEmissionsDataResponse.cs diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs index 01e494bbd..acfc1339e 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs @@ -72,7 +72,7 @@ private static EmissionsForecast ToEmissionsForecast(Location location, Forecast } /// - public async Task GetCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt) + public async Task GetHistoricalCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt) { await Task.Run(() => true); throw new NotImplementedException(); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs index b07f96cd8..73d432861 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs @@ -49,11 +49,11 @@ internal interface IWattTimeClient /// /// Async method to get generated forecast at requested time and balancing authority. /// - /// Balancing authority abbreviation + /// Balancing authority abbreviation /// The historical time used to fetch the most recent forecast generated as of that time. /// An which contains forecasted emissions data points or null if no Forecast generated at the requested time. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetForecastOnDateAsync(string balancingAuthorityAbbreviation, DateTimeOffset requestedAt); + Task GetForecastOnDateAsync(string region, DateTimeOffset requestedAt); /// /// Async method to get generated forecast at requested time and balancing authority. @@ -62,7 +62,7 @@ internal interface IWattTimeClient /// The historical time used to fetch the most recent forecast generated as of that time. /// An which contains forecasted emissions data points or null if no Forecast generated at the requested time. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt); + Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt); /// /// Async method to get the balancing authority for a given location. diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index 8e02c7349..2d0b50951 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -103,9 +103,6 @@ public async Task GetCurrentForecastAsync(string var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters, tags); - var sr = new StreamReader(result); - var s = sr.ReadToEnd(); - var forecast = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting forecast for {region}"); return forecast; @@ -118,32 +115,33 @@ public Task GetCurrentForecastAsync(RegionRespons } /// - public async Task GetForecastOnDateAsync(string balancingAuthorityAbbreviation, DateTimeOffset requestedAt) + public async Task GetForecastOnDateAsync(string region, DateTimeOffset requestedAt) { - _log.LogInformation($"Requesting forecast from balancingAuthority {balancingAuthorityAbbreviation} generated at {requestedAt}."); + _log.LogInformation($"Requesting forecast from balancingAuthority {region} generated at {requestedAt}."); var parameters = new Dictionary() { - { QueryStrings.Region, balancingAuthorityAbbreviation }, + { QueryStrings.Region, region }, + { QueryStrings.SignalType, SignalTypes.co2_moer }, { QueryStrings.StartTime, requestedAt.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) }, { QueryStrings.EndTime, requestedAt.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) } }; var tags = new Dictionary() { - { QueryStrings.Region, balancingAuthorityAbbreviation } + { QueryStrings.Region, region } }; - using (var result = await this.MakeRequestGetStreamAsync(Paths.Forecast, parameters, tags)) + using (var result = await this.MakeRequestGetStreamAsync(Paths.ForecastHistorical, parameters, tags)) { - var forecasts = await JsonSerializer.DeserializeAsync>(result, _options) ?? throw new WattTimeClientException($"Error getting forecasts for {balancingAuthorityAbbreviation}"); - return forecasts.FirstOrDefault(); + var historicalForecastResponse = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting forecasts for {region}"); + return historicalForecastResponse; } } /// - public Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt) + public Task GetForecastOnDateAsync(RegionResponse region, DateTimeOffset requestedAt) { - return this.GetForecastOnDateAsync(balancingAuthority.Region, requestedAt); + return this.GetForecastOnDateAsync(region.Region, requestedAt); } /// diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs index 61c3092f9..8704d8bfb 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs @@ -4,6 +4,8 @@ internal class Paths { public const string Data = "historical"; public const string Forecast = "forecast"; + + public const string ForecastHistorical = "forecast/historical"; public const string BalancingAuthorityFromLocation = "region-from-loc"; public const string Login = "login"; public const string Historical = "historical"; diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/HistoricalEmissionsData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/HistoricalEmissionsData.cs new file mode 100644 index 000000000..5c82f01a6 --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/HistoricalEmissionsData.cs @@ -0,0 +1,15 @@ +using System.Text.Json.Serialization; + +namespace CarbonAware.DataSources.WattTime.Model; + +[Serializable] +internal class HistoricalEmissionsData +{ + [JsonPropertyName("generated_at")] + public DateTimeOffset GeneratedAt { get; set; } = DateTimeOffset.MinValue; + + [JsonPropertyName("forecast")] + public List Forecast { get; set; } = new List(); + +} + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/HistoricalForecastEmissionsDataResponse.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/HistoricalForecastEmissionsDataResponse.cs new file mode 100644 index 000000000..212e8170a --- /dev/null +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/HistoricalForecastEmissionsDataResponse.cs @@ -0,0 +1,16 @@ +using System.Text.Json.Serialization; + +namespace CarbonAware.DataSources.WattTime.Model; + +[Serializable] +internal record HistoricalForecastEmissionsDataResponse +{ + [JsonPropertyName("data")] + public List Data { get; set; } = new List(); + + + [JsonPropertyName("meta")] + public GridEmissionsMetaData Meta { get; set; } = new GridEmissionsMetaData(); +} + + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs index c879df636..39a114777 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs @@ -87,7 +87,7 @@ public async Task GetCurrentCarbonIntensityForecastAsync(Loca } /// - public async Task GetCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt) + public async Task GetHistoricalCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt) { this.Logger.LogInformation($"Getting carbon intensity forecast for location {location} requested at {requestedAt}"); var balancingAuthority = await this.GetBalancingAuthority(location); @@ -100,7 +100,26 @@ public async Task GetCarbonIntensityForecastAsync(Location lo throw ex; } // keep input from the user. - return ForecastToEmissionsForecast(forecast, location, requestedAt); + return HistoricalForecastToEmissionsForecast(forecast, location, requestedAt); + } + + private EmissionsForecast HistoricalForecastToEmissionsForecast(HistoricalForecastEmissionsDataResponse historicalForecast, Location location, DateTimeOffset requestedAt) + { + var duration = GetDurationFromGridEmissionDataPoints(historicalForecast.Data[0].Forecast); + var forecastData = historicalForecast.Data[0].Forecast.Select(e => new EmissionsData() + { + Location = historicalForecast.Meta.Region, + Rating = ConvertMoerToGramsPerKilowattHour(e.Value), + Time = e.PointTime, + Duration = duration + }); + var emissionsForecast = new EmissionsForecast() + { + GeneratedAt = historicalForecast.Data[0].GeneratedAt, + Location = location, + ForecastData = forecastData + }; + return emissionsForecast; } private EmissionsForecast ForecastToEmissionsForecast(ForecastEmissionsDataResponse forecast, Location location, DateTimeOffset requestedAt) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index a79ca8b56..52d244a9f 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -252,12 +252,12 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() var overloadedForecast = await client.GetForecastOnDateAsync(ba, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); Assert.AreEqual(forecastResponse!.Meta.GeneratedAt, overloadedForecast!.Meta.GeneratedAt); - Assert.AreEqual(forecastResponse.Data.First(), overloadedForecast.Data.First()); + Assert.AreEqual(forecastResponse.Data[0].Forecast.First(), overloadedForecast.Data[0].Forecast.First()); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse.Meta.GeneratedAt); Assert.AreEqual("region", forecastResponse.Meta.Region); - var forecastDataPoint = forecastResponse.Data.ToList().First(); + var forecastDataPoint = forecastResponse.Data[0].Forecast.ToList().First(); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint.PointTime); Assert.AreEqual("999.99", forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", forecastDataPoint.Version); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs index 0ddf84dcb..4e9942efb 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs @@ -127,6 +127,9 @@ public async Task GetCarbonIntensityForecastAsync_ReturnsResultsWhenRecordsFound var forecastResponse = GenerateForecastResponse(2, value: lbsPerMwhEmissions); forecastResponse.Meta.GeneratedAt = generatedAt; + var historicalForecastResponse = GenerateHistoricalForecastResponse(2, value: lbsPerMwhEmissions); + historicalForecastResponse.Meta.GeneratedAt = generatedAt; + EmissionsForecast result; if (getCurrentForecast) @@ -140,10 +143,10 @@ public async Task GetCarbonIntensityForecastAsync_ReturnsResultsWhenRecordsFound else { this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultRegion, generatedAt) - ).ReturnsAsync(() => forecastResponse); + ).ReturnsAsync(() => historicalForecastResponse); // Act - result = await this.DataSource.GetCarbonIntensityForecastAsync(this.DefaultLocation, generatedAt); + result = await this.DataSource.GetHistoricalCarbonIntensityForecastAsync(this.DefaultLocation, generatedAt); } // Assert @@ -174,18 +177,18 @@ public void GetCarbonIntensityForecastAsync_ThrowsWhenRegionNotFound() this.LocationSource.Setup(l => l.ToGeopositionLocationAsync(this.DefaultLocation)).Throws(); Assert.ThrowsAsync(async () => await this.DataSource.GetCurrentCarbonIntensityForecastAsync(this.DefaultLocation)); - Assert.ThrowsAsync(async () => await this.DataSource.GetCarbonIntensityForecastAsync(this.DefaultLocation, new DateTimeOffset())); + Assert.ThrowsAsync(async () => await this.DataSource.GetHistoricalCarbonIntensityForecastAsync(this.DefaultLocation, new DateTimeOffset())); } [Test] - public void GetCarbonIntensityForecastAsync_ThrowsWhenNoForecastFoundForReuqestedTime() + public void GetHistoricalCarbonIntensityForecastAsync_ThrowsWhenNoForecastFoundForReuqestedTime() { var generatedAt = new DateTimeOffset(); - this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultRegion, generatedAt)).Returns(Task.FromResult(null)); + this.WattTimeClient.Setup(w => w.GetForecastOnDateAsync(this.DefaultRegion, generatedAt)).Returns(Task.FromResult(null)); // The datasource throws an exception if no forecasts are found at the requested generatedAt time. - Assert.ThrowsAsync(async () => await this.DataSource.GetCarbonIntensityForecastAsync(this.DefaultLocation, generatedAt)); + Assert.ThrowsAsync(async () => await this.DataSource.GetHistoricalCarbonIntensityForecastAsync(this.DefaultLocation, generatedAt)); } [TestCase(0, TestName = "GetCurrentCarbonIntensityForecastAsync throws for: No datapoints")] @@ -211,7 +214,7 @@ public async Task GetCarbonIntensityForecastAsync_RequiredAtRounded(string reque var requestedAt = DateTimeOffset.Parse(requested); var expectedAt = DateTimeOffset.Parse(expected); - var forecastResponse = GenerateForecastResponse(2, startTime: requestedAt); + var forecastResponse = GenerateHistoricalForecastResponse(2, startTime: requestedAt); forecastResponse.Meta.GeneratedAt = expectedAt; @@ -219,7 +222,7 @@ public async Task GetCarbonIntensityForecastAsync_RequiredAtRounded(string reque ).ReturnsAsync(() => forecastResponse); // Act - var result = await this.DataSource.GetCarbonIntensityForecastAsync(this.DefaultLocation, requestedAt); + var result = await this.DataSource.GetHistoricalCarbonIntensityForecastAsync(this.DefaultLocation, requestedAt); // Assert Assert.IsNotNull(result); @@ -305,6 +308,31 @@ private GridEmissionsDataResponse GenerateGridEmissionsResponse(int numberOfData return response; } + private HistoricalForecastEmissionsDataResponse GenerateHistoricalForecastResponse(int numberOfDatapoints, float value = 10, DateTimeOffset startTime = default) + { + var data = GenerateDataPoints(numberOfDatapoints, value, startTime); + var meta = new GridEmissionsMetaData() + { + Region = this.DefaultRegion.Region, + SignalType = SignalTypes.co2_moer + }; + + var response = new HistoricalForecastEmissionsDataResponse() + { + Data = new List() + { + new HistoricalEmissionsData() + { + Forecast = data, + GeneratedAt = DateTimeOffset.Now + } + }, + Meta = meta + }; + + return response; + } + private ForecastEmissionsDataResponse GenerateForecastResponse(int numberOfDatapoints, float value = 10, DateTimeOffset startTime = default) { var data = GenerateDataPoints(numberOfDatapoints, value, startTime); diff --git a/src/CarbonAware/src/Interfaces/IForecastDataSource.cs b/src/CarbonAware/src/Interfaces/IForecastDataSource.cs index cd8d06c08..964b0e18e 100644 --- a/src/CarbonAware/src/Interfaces/IForecastDataSource.cs +++ b/src/CarbonAware/src/Interfaces/IForecastDataSource.cs @@ -14,5 +14,5 @@ internal interface IForecastDataSource /// The location that should be used for getting the forecast. /// The historical time used to fetch the most recent forecast generated as of that time. /// A forecasted emissions object for the given location generated at the given time. - Task GetCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt); + Task GetHistoricalCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt); } \ No newline at end of file diff --git a/src/CarbonAware/src/NullForecastDataSource.cs b/src/CarbonAware/src/NullForecastDataSource.cs index 7b4288dbd..9492e3d31 100644 --- a/src/CarbonAware/src/NullForecastDataSource.cs +++ b/src/CarbonAware/src/NullForecastDataSource.cs @@ -4,7 +4,7 @@ namespace CarbonAware; internal class NullForecastDataSource : IForecastDataSource { - public Task GetCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt) + public Task GetHistoricalCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt) { throw new ArgumentException("ForecastDataSource is not configured"); } diff --git a/src/GSF.CarbonAware/src/Handlers/ForecastHandler.cs b/src/GSF.CarbonAware/src/Handlers/ForecastHandler.cs index adc40f4de..fa8b413ab 100644 --- a/src/GSF.CarbonAware/src/Handlers/ForecastHandler.cs +++ b/src/GSF.CarbonAware/src/Handlers/ForecastHandler.cs @@ -73,7 +73,7 @@ public async Task GetForecastByDateAsync(string location, Dat { parameters.SetRequiredProperties(PropertyName.SingleLocation, PropertyName.Requested); parameters.Validate(); - var forecast = await _forecastDataSource.GetCarbonIntensityForecastAsync(parameters.SingleLocation, parameters.Requested); + var forecast = await _forecastDataSource.GetHistoricalCarbonIntensityForecastAsync(parameters.SingleLocation, parameters.Requested); var emissionsForecast = ProcessAndValidateForecast(forecast, parameters); return emissionsForecast; } diff --git a/src/GSF.CarbonAware/test/Handlers/ForecastHandlerTests.cs b/src/GSF.CarbonAware/test/Handlers/ForecastHandlerTests.cs index d0057fe48..d87733311 100644 --- a/src/GSF.CarbonAware/test/Handlers/ForecastHandlerTests.cs +++ b/src/GSF.CarbonAware/test/Handlers/ForecastHandlerTests.cs @@ -207,7 +207,7 @@ private static Mock CreateForecastByDateDataSource(global:: { var datasource = new Mock(); datasource - .Setup(x => x.GetCarbonIntensityForecastAsync(It.IsAny(), requested)) + .Setup(x => x.GetHistoricalCarbonIntensityForecastAsync(It.IsAny(), requested)) .ReturnsAsync(data); return datasource; @@ -221,7 +221,7 @@ private static Mock SetupMockDataSourceThatThrows() .ThrowsAsync(new CarbonAware.Exceptions.CarbonAwareException("")); datasource - .Setup(x => x.GetCarbonIntensityForecastAsync(It.IsAny(), It.IsAny())) + .Setup(x => x.GetHistoricalCarbonIntensityForecastAsync(It.IsAny(), It.IsAny())) .ThrowsAsync(new CarbonAware.Exceptions.CarbonAwareException("", It.IsAny())); return datasource; From 92f8dbc968da9f709cd232f64c6ebf110a586b1a Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Mon, 17 Jun 2024 16:46:04 +1000 Subject: [PATCH 18/31] DCO Remediation Commit for Vaughan Knight I, Vaughan Knight , hereby add my Signed-off-by to this commit: b9490e425c291b75cc13ecd877a4b34ca96ab68c I, Vaughan Knight , hereby add my Signed-off-by to this commit: b991bac14c9864f331e56b739398b9675f23e37c I, Vaughan Knight , hereby add my Signed-off-by to this commit: e4f14944901e0ebecf4e4a7c8b2c543a87b37a7d I, Vaughan Knight , hereby add my Signed-off-by to this commit: b443e9eaa3cdcedd7cfab435e64f8a9999ca44d8 I, Vaughan Knight , hereby add my Signed-off-by to this commit: ab1205d699f99887e9a7e59f2bf8b0a692690df6 I, Vaughan Knight , hereby add my Signed-off-by to this commit: 7c115fa1213cbc042ffbe6e72f208a190b5d7584 I, Vaughan Knight , hereby add my Signed-off-by to this commit: e047c9a2711ef5aa2e5cb2cbb094fa21914eb8d5 I, Vaughan Knight , hereby add my Signed-off-by to this commit: aa81382704ccec7a8da108ef5c293845a9468c8f I, Vaughan Knight , hereby add my Signed-off-by to this commit: 8640c8cf242b1872701fa46b5ba338d28c09a512 I, Vaughan Knight , hereby add my Signed-off-by to this commit: 880fcf78c1e29342b17086bce6375d31a3dbb224 Signed-off-by: Vaughan Knight From 39e45a61b999f862b4b3cc7a6adb430b25dfe08d Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Tue, 18 Jun 2024 10:52:07 +1000 Subject: [PATCH 19/31] Many tests reworked, a few to go Many tests reworked, a few to go. Consolidated a lot of the hand crafted json objects into objects that get serialized as the purist JsonObject format was prone to errors - in some cases tests were passing even with bad typing. --- .../EmissionsForecastsCommandTests.cs | 2 +- .../mock/ElectricityMapDataSourceMocker.cs | 2 +- .../ElectricityMapsFreeDataSourceMocker.cs | 2 +- .../mock/JsonDataSourceMocker.cs | 2 +- .../mock/WattTimeDataSourceMocker.cs | 33 +++- .../src/Client/WattTimeClient.cs | 10 +- .../src/Constants/Paths.cs | 2 +- .../src/WattTimeDataSource.cs | 6 +- .../test/Client/TestData.cs | 141 ++++++++++++------ .../test/Client/WattTimeClientTests.cs | 55 +++---- .../test/WattTimeDataSourceTests.cs | 6 +- .../CarbonAwareControllerTests.cs | 2 +- .../src/Interfaces/IDataSourceMocker.cs | 2 +- 13 files changed, 164 insertions(+), 101 deletions(-) diff --git a/src/CarbonAware.CLI/test/integrationTests/Commands/EmissionsForecasts/EmissionsForecastsCommandTests.cs b/src/CarbonAware.CLI/test/integrationTests/Commands/EmissionsForecasts/EmissionsForecastsCommandTests.cs index 2c232d187..5daf173c8 100644 --- a/src/CarbonAware.CLI/test/integrationTests/Commands/EmissionsForecasts/EmissionsForecastsCommandTests.cs +++ b/src/CarbonAware.CLI/test/integrationTests/Commands/EmissionsForecasts/EmissionsForecastsCommandTests.cs @@ -99,7 +99,7 @@ public async Task EmissionsForecasts_RequestedAtOptions_ReturnsExpectedData() IgnoreTestForDataSource("data source does not implement '--requested-at'", DataSourceType.ElectricityMaps); // Arrange - _dataSourceMocker.SetupBatchForecastMock(); + _dataSourceMocker.SetupHistoricalBatchForecastMock(); // Act var exitCode = await InvokeCliAsync($"emissions-forecasts -l eastus -r 2022-09-01"); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs index aa36674d7..b10a0a288 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs @@ -126,7 +126,7 @@ public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string locat SetupResponseGivenGetRequest(Paths.PastRange, pastRange); } - public void SetupBatchForecastMock() + public void SetupHistoricalBatchForecastMock() { throw new NotImplementedException(); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMapsFree/mock/ElectricityMapsFreeDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMapsFree/mock/ElectricityMapsFreeDataSourceMocker.cs index f8a645aca..7ddf800b1 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMapsFree/mock/ElectricityMapsFreeDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.ElectricityMapsFree/mock/ElectricityMapsFreeDataSourceMocker.cs @@ -49,7 +49,7 @@ public void SetupForecastMock() throw new NotImplementedException(); } - public void SetupBatchForecastMock() + public void SetupHistoricalBatchForecastMock() { throw new NotImplementedException(); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.Json/mock/JsonDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.Json/mock/JsonDataSourceMocker.cs index ee7bf4ba5..5e09ba3de 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.Json/mock/JsonDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.Json/mock/JsonDataSourceMocker.cs @@ -42,7 +42,7 @@ public void Initialize() { } public void Reset() { } public void Dispose() { } - public void SetupBatchForecastMock() + public void SetupHistoricalBatchForecastMock() { throw new NotImplementedException(); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs index 3c95e5350..dce0b7625 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs @@ -26,6 +26,8 @@ public WattTimeDataSourceMocker() { _server = WireMockServer.Start(); Environment.SetEnvironmentVariable("DataSources__Configurations__WattTime__BaseURL", _server.Url!); + Environment.SetEnvironmentVariable("DataSources__Configurations__WattTime__AuthenticationBaseUrl", _server.Url!); + Initialize(); } @@ -111,7 +113,7 @@ public void SetupForecastMock() SetupResponseGivenGetRequest(Paths.Forecast, JsonSerializer.Serialize(forecastResponse)); } - public void SetupBatchForecastMock() + public void SetupHistoricalBatchForecastMock() { var start = new DateTimeOffset(2021, 9, 1, 8, 30, 0, TimeSpan.Zero); var end = start + TimeSpan.FromDays(1.0); @@ -142,14 +144,29 @@ public void SetupBatchForecastMock() GeneratedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero) }; - var forecastBatchData = new List { - new ForecastEmissionsDataResponse() + //var forecastBatchData = new List { + // new ForecastEmissionsDataResponse() + // { + // Data = forecastData, + // Meta = meta + // } + //}; + + var historicalForecastResponse = new HistoricalForecastEmissionsDataResponse() + { + Data = new List() { - Data = forecastData, - Meta = meta - } + new HistoricalEmissionsData() + { + Forecast = forecastData, + GeneratedAt = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero) + } + }, + Meta = meta }; - SetupResponseGivenGetRequest(Paths.Forecast, JsonSerializer.Serialize(forecastBatchData)); + + + SetupResponseGivenGetRequest(Paths.ForecastHistorical, JsonSerializer.Serialize(historicalForecastResponse)); } public void Initialize() @@ -180,7 +197,7 @@ private void SetupResponseGivenGetRequest(string path, string body) ); } private void SetupBaMock(RegionResponse? content = null) => - SetupResponseGivenGetRequest(Paths.BalancingAuthorityFromLocation, JsonSerializer.Serialize(content ?? defaultRegion)); + SetupResponseGivenGetRequest(Paths.RegionFromLocation, JsonSerializer.Serialize(content ?? defaultRegion)); private void SetupLoginMock(LoginResult? content = null) => SetupResponseGivenGetRequest(Paths.Login, JsonSerializer.Serialize(content ?? defaultLoginResult)); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index 2d0b50951..670b34a4f 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -122,9 +122,9 @@ public Task GetCurrentForecastAsync(RegionRespons var parameters = new Dictionary() { { QueryStrings.Region, region }, - { QueryStrings.SignalType, SignalTypes.co2_moer }, { QueryStrings.StartTime, requestedAt.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) }, - { QueryStrings.EndTime, requestedAt.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) } + { QueryStrings.EndTime, requestedAt.ToUniversalTime().ToString("O", CultureInfo.InvariantCulture) }, + { QueryStrings.SignalType, SignalTypes.co2_moer } }; var tags = new Dictionary() @@ -215,7 +215,7 @@ private async Task GetStreamWithAuthRetryAsync(string uriPath) private async Task EnsureTokenAsync() { - if (this._client.DefaultRequestHeaders.Authorization == null) + if (this._authenticationClient.DefaultRequestHeaders.Authorization == null) { await this.UpdateAuthTokenAsync(); } @@ -249,7 +249,7 @@ private async Task UpdateAuthTokenAsync() private void SetBasicAuthenticationHeader() { var authToken = Encoding.UTF8.GetBytes($"{this._configuration.Username}:{this._configuration.Password}"); - this._client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(AuthenticationHeaderTypes.Basic, Convert.ToBase64String(authToken)); + //this._client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(AuthenticationHeaderTypes.Basic, Convert.ToBase64String(authToken)); this._authenticationClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(AuthenticationHeaderTypes.Basic, Convert.ToBase64String(authToken)); } @@ -311,7 +311,7 @@ private async Task GetBalancingAuthorityFromCacheAsync(string la { QueryStrings.Longitude, longitude }, { QueryStrings.SignalType, SignalTypes.co2_moer } }; - var result = await this.MakeRequestGetStreamAsync(Paths.BalancingAuthorityFromLocation, parameters, tags); + var result = await this.MakeRequestGetStreamAsync(Paths.RegionFromLocation, parameters, tags); var baValue = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting Balancing Authority for latitude {latitude} and longitude {longitude}"); entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(_configuration.BalancingAuthorityCacheTTL); entry.Value = baValue; diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs index 8704d8bfb..b726c2c97 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Constants/Paths.cs @@ -6,7 +6,7 @@ internal class Paths public const string Forecast = "forecast"; public const string ForecastHistorical = "forecast/historical"; - public const string BalancingAuthorityFromLocation = "region-from-loc"; + public const string RegionFromLocation = "region-from-loc"; public const string Login = "login"; public const string Historical = "historical"; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs index 39a114777..7f71cb842 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs @@ -62,12 +62,12 @@ public async Task> GetCarbonIntensityAsync(Location l Logger.LogInformation($"Getting carbon intensity for location {location} for period {periodStartTime} to {periodEndTime}."); var balancingAuthority = await this.GetBalancingAuthority(location); var (newStartTime, newEndTime) = IntervalHelper.ExtendTimeByWindow(periodStartTime, periodEndTime, MinSamplingWindow); - var historialResponse = await this.WattTimeClient.GetDataAsync(balancingAuthority, newStartTime, newEndTime); + var historicalResponse = await this.WattTimeClient.GetDataAsync(balancingAuthority, newStartTime, newEndTime); if (Logger.IsEnabled(LogLevel.Debug)) { - Logger.LogDebug($"Found {historialResponse.Data.Count()} total forecasts for location {location} for period {periodStartTime} to {periodEndTime}."); + Logger.LogDebug($"Found {historicalResponse.Data.Count()} total forecasts for location {location} for period {periodStartTime} to {periodEndTime}."); } - var windowData = ConvertToEmissionsData(historialResponse); + var windowData = ConvertToEmissionsData(historicalResponse); var filteredData = IntervalHelper.FilterByDuration(windowData, periodStartTime, periodEndTime); if (!filteredData.Any()) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs index 90f6f1b8d..699f46991 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs @@ -1,81 +1,126 @@ +using CarbonAware.DataSources.WattTime.Constants; +using CarbonAware.DataSources.WattTime.Model; using System; -using System.Text.Json.Nodes; +using System.Collections.Generic; +using System.Text.Json; namespace CarbonAware.DataSources.WattTime.Client.Tests; internal static class TestData { - internal static string GetGridDataJsonString() + public class TestDataConstants { - var json = new JsonArray( - new JsonObject - { - ["ba"] = "ba", - ["datatype"] = "dt", - ["frequency"] = 300, - ["market"] = "mkt", - ["point_time"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), - ["value"] = 999.99, - ["version"] = "1.0" - } - ); + public const string Region = "TEST_REGION"; + public const string RegionFullName = "Test Region Full Name"; + public const string Market = "mkt"; + public static DateTimeOffset GeneratedAt = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero); + public static DateTimeOffset PointTime = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero); + public static DateTime Date = new DateTime(2099, 1, 1, 0, 0, 0); + public const float Value = 999.99f; + public const string Version = "1.0"; + } + + internal static string GetGridDataResponseJsonString() + { + return JsonSerializer.Serialize(_GetGridDataResponse()); + } + private static GridEmissionsDataResponse _GetGridDataResponse() + { + var gridEmissionDataResponse = new GridEmissionsDataResponse() + { + Meta = _GetGridDataMetaResponse(), + Data = _GetGridEmissionDataPoints() + }; + return gridEmissionDataResponse; + } + + private static GridEmissionsMetaData _GetGridDataMetaResponse() + { + var gridEmissionsMetaData = new GridEmissionsMetaData() + { + Region = TestDataConstants.Region, + GeneratedAt = TestDataConstants.GeneratedAt, + GeneratedAtPeriodSeconds = 30, + Model = new GridEmissionsModelData() + { + Date = TestDataConstants.Date, + Type = SignalTypes.co2_moer + }, + DataPointPeriodSeconds = 30, + SignalType = SignalTypes.co2_moer, + Units = "co2_moer" + }; + + return gridEmissionsMetaData; + } + private static List _GetGridEmissionDataPoints() + { + return new List() + { + _GetGridEmissionDataPoint() + }; + } - return json.ToString(); + private static GridEmissionDataPoint _GetGridEmissionDataPoint() + { + return new GridEmissionDataPoint() + { + Frequency = 300, + Market = TestDataConstants.Market, + PointTime = TestDataConstants.PointTime, + Value = TestDataConstants.Value, + Version = TestDataConstants.Version + }; } internal static string GetCurrentForecastJsonString() { + return JsonSerializer.Serialize(_GetCurrentForecastEmissionsDataResponse()); + } - var json = new JsonObject + private static ForecastEmissionsDataResponse _GetCurrentForecastEmissionsDataResponse() + { + return new ForecastEmissionsDataResponse() { - ["generated_at"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), - ["forecast"] = new JsonArray - { - new JsonObject - { - ["ba"] = "ba", - ["point_time"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), - ["value"] = 999.99, - ["version"] = "1.0" - } - } + Data = _GetGridEmissionDataPoints(), + Meta = _GetGridDataMetaResponse() }; + } + - return json.ToString(); + internal static string GetHistoricalForecastDataJsonString() + { + return JsonSerializer.Serialize(_GetHistoricalForecastEmissionsDataResponse()); } - internal static string GetForecastByDateJsonString() + private static HistoricalForecastEmissionsDataResponse _GetHistoricalForecastEmissionsDataResponse() { - var json = new JsonArray + return new HistoricalForecastEmissionsDataResponse() { - new JsonObject + Meta = _GetGridDataMetaResponse(), + Data = new List { - ["generated_at"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), - ["forecast"] = new JsonArray + new HistoricalEmissionsData() { - new JsonObject - { - ["ba"] = "ba", - ["point_time"] = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), - ["value"] = 999.99, - ["version"] = "1.0" - } + Forecast = _GetGridEmissionDataPoints(), + GeneratedAt = TestDataConstants.GeneratedAt } } }; + } - return json.ToString(); + internal static string GetRegionJsonString() + { + return JsonSerializer.Serialize(_GetRegion()); } - internal static string GetBalancingAuthorityJsonString() + private static RegionResponse _GetRegion() { - var json = new JsonObject + return new RegionResponse() { - ["id"] = "12345", - ["abbrev"] = "TEST_BA", - ["name"] = "Test Balancing Authority" + Region = TestDataConstants.Region, + RegionFullName = TestDataConstants.RegionFullName, + SignalType = SignalTypes.co2_moer }; - - return json.ToString(); } } \ No newline at end of file diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index 52d244a9f..c6715f79c 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -121,8 +121,8 @@ public async Task GetDataAsync_DeserializesExpectedResponse() this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => { - return r.RequestUri!.ToString().Equals("https://api2.watttime.org/v2/data?ba=balauth&starttime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&endtime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00") && r.Method == HttpMethod.Get; - }, System.Net.HttpStatusCode.OK, TestData.GetGridDataJsonString()); + return r.RequestUri!.ToString().Equals($"https://api.watttime.org/v3/data?region=region&starttime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&endtime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type={SignalTypes.co2_moer}") && r.Method == HttpMethod.Get; + }, System.Net.HttpStatusCode.OK, TestData.GetGridDataResponseJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -144,7 +144,7 @@ public async Task GetDataAsync_DeserializesExpectedResponse() [Test] public async Task GetDataAsync_RefreshesTokenWhenExpired() { - this.SetupBasicHandlers(TestData.GetGridDataJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(TestData.GetGridDataResponseJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -158,7 +158,7 @@ public async Task GetDataAsync_RefreshesTokenWhenExpired() [Test] public async Task GetDataAsync_RefreshesTokenWhenNoneSet() { - this.SetupBasicHandlers(TestData.GetGridDataJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(TestData.GetGridDataResponseJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -174,23 +174,23 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => { - return r.RequestUri!.ToString().Equals("https://api2.watttime.org/v2/forecast?ba=balauth") && r.Method == HttpMethod.Get; + return r.RequestUri!.ToString().Equals(@"https://api.watttime.org/v3/forecast?region=region&signal_type=co2_moer") && r.Method == HttpMethod.Get; }, System.Net.HttpStatusCode.OK, TestData.GetCurrentForecastJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var regionResponse = new RegionResponse() { Region = "region" }; + const string REGION = "region"; - var forecastResponse = await client.GetCurrentForecastAsync(regionResponse.Region); - var overloadedForecast = await client.GetCurrentForecastAsync(regionResponse); + var forecastResponse = await client.GetCurrentForecastAsync(REGION); + var overloadedForecast = await client.GetCurrentForecastAsync(REGION); Assert.AreEqual(forecastResponse.Meta.GeneratedAt, overloadedForecast.Meta.GeneratedAt); Assert.AreEqual(forecastResponse.Data.First(), overloadedForecast.Data.First()); Assert.IsNotNull(forecastResponse); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); - Assert.AreEqual("region", forecastResponse?.Meta.Region); + Assert.AreEqual(TestData.TestDataConstants.Region, forecastResponse?.Meta.Region); var forecastDataPoint = forecastResponse?.Data.First(); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint?.PointTime); @@ -210,7 +210,7 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenExpired() Assert.IsNotNull(forecastResponse); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); - Assert.AreEqual("region", forecastResponse?.Meta.Region); + Assert.AreEqual(TestData.TestDataConstants.Region, forecastResponse?.Meta.Region); } [Test] @@ -232,7 +232,7 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenNoneSet() Assert.IsNotNull(forecastResponse); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); - Assert.AreEqual("region", forecastResponse?.Meta.Region); + Assert.AreEqual(TestData.TestDataConstants.Region, forecastResponse?.Meta.Region); } [Test] @@ -242,7 +242,7 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() this.AddHandler_RequestResponse(r => { return r.RequestUri!.ToString().Equals("https://api2.watttime.org/v2/forecast?ba=balauth&starttime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&endtime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00") && r.Method == HttpMethod.Get; - }, System.Net.HttpStatusCode.OK, TestData.GetForecastByDateJsonString()); + }, System.Net.HttpStatusCode.OK, TestData.GetHistoricalForecastDataJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -255,18 +255,18 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() Assert.AreEqual(forecastResponse.Data[0].Forecast.First(), overloadedForecast.Data[0].Forecast.First()); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse.Meta.GeneratedAt); - Assert.AreEqual("region", forecastResponse.Meta.Region); + Assert.AreEqual(TestData.TestDataConstants.Region, forecastResponse.Meta.Region); var forecastDataPoint = forecastResponse.Data[0].Forecast.ToList().First(); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint.PointTime); - Assert.AreEqual("999.99", forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues + Assert.AreEqual(TestData.TestDataConstants.PointTime, forecastDataPoint.PointTime); + Assert.AreEqual(TestData.TestDataConstants.Value, forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", forecastDataPoint.Version); } [Test] public async Task GetForecastOnDateAsync_RefreshesTokenWhenExpired() { - this.SetupBasicHandlers(TestData.GetForecastByDateJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(TestData.GetHistoricalForecastDataJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -287,7 +287,7 @@ public async Task GetForecastOnDateAsync_RefreshesTokenWhenNoneSet() return client; }); - this.SetupBasicHandlers(TestData.GetForecastByDateJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(TestData.GetHistoricalForecastDataJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -302,8 +302,9 @@ public async Task GetBalancingAuthorityAsync_DeserializesExpectedResponse() this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => { - return r.RequestUri!.ToString().Equals("https://api2.watttime.org/v2/ba-from-loc?latitude=lat&longitude=long") && r.Method == HttpMethod.Get; - }, System.Net.HttpStatusCode.OK, TestData.GetBalancingAuthorityJsonString()); + return r.RequestUri!.ToString().Equals("https://api.watttime.org/v3/region-from-loc?latitude=lat&longitude=long&signal_type=co2_moer") && r.Method == HttpMethod.Get; + }, System.Net.HttpStatusCode.OK, TestData.GetRegionJsonString()); + var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -311,15 +312,15 @@ public async Task GetBalancingAuthorityAsync_DeserializesExpectedResponse() var regionResponse = await client.GetRegionAsync("lat", "long"); Assert.IsNotNull(regionResponse); - Assert.AreEqual("TEST_BA", regionResponse?.Region); - Assert.AreEqual("Test Balancing Authority", regionResponse?.RegionFullName); + Assert.AreEqual(TestData.TestDataConstants.Region, regionResponse?.Region); + Assert.AreEqual(TestData.TestDataConstants.RegionFullName, regionResponse?.RegionFullName); Assert.AreEqual(SignalTypes.co2_moer, regionResponse?.SignalType); } [Test] public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenExpired() { - this.SetupBasicHandlers(TestData.GetBalancingAuthorityJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(TestData.GetRegionJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -342,7 +343,7 @@ public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenNoneSet() return client; }); - this.SetupBasicHandlers(TestData.GetBalancingAuthorityJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(TestData.GetRegionJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -429,12 +430,12 @@ private void AddHandlers_Auth(string? validToken = null) AddHandler_RequestResponse(r => { - return (r.RequestUri == new Uri("https://api2.watttime.org/v2/login") && ($"Basic {this.BasicAuthValue}".Equals(r.Headers.Authorization?.ToString()))); + return (r.RequestUri == new Uri("https://api.watttime.org/login") && ($"Basic {this.BasicAuthValue}".Equals(r.Headers.Authorization?.ToString()))); }, System.Net.HttpStatusCode.OK, "{\"token\":\"" + validToken + "\"}"); AddHandler_RequestResponse(r => { - return !(r.RequestUri == new Uri("https://api2.watttime.org/v2/login") && ($"Basic {this.BasicAuthValue}".Equals(r.Headers.Authorization?.ToString()))) && r.Headers.Authorization?.ToString() != $"Bearer {validToken}"; + return !(r.RequestUri == new Uri("https://api.watttime.org/login") && ($"Basic {this.BasicAuthValue}".Equals(r.Headers.Authorization?.ToString()))) && r.Headers.Authorization?.ToString() != $"Bearer {validToken}"; }, System.Net.HttpStatusCode.Forbidden); } @@ -449,7 +450,7 @@ private void SetupBasicHandlers(StreamContent responseContent, string? validToke // Catch-all for "requesting url that is not login and has valid token" this.Handler - .SetupRequest(r => r.RequestUri != new Uri("https://api2.watttime.org/v2/login") && r.Headers.Authorization?.ToString() == $"Bearer {validToken}") + .SetupRequest(r => r.RequestUri != new Uri("https://api.watttime.org/login") && r.Headers.Authorization?.ToString() == $"Bearer {validToken}") .ReturnsResponse(System.Net.HttpStatusCode.OK, responseContent); } @@ -463,7 +464,7 @@ private void SetupBasicHandlers(string responseContent, string? validToken = nul AddHandlers_Auth(validToken); // Catch-all for "requesting url that is not login and has valid token" - AddHandler_RequestResponse(r => r.RequestUri != new Uri("https://api2.watttime.org/v2/login") && r.Headers.Authorization?.ToString() == $"Bearer {validToken}", System.Net.HttpStatusCode.OK, responseContent); + AddHandler_RequestResponse(r => r.RequestUri != new Uri("https://api.watttime.org/login") && r.Headers.Authorization?.ToString() == $"Bearer {validToken}", System.Net.HttpStatusCode.OK, responseContent); } /** diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs index 4e9942efb..b1620205a 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs @@ -47,7 +47,7 @@ public void Setup() this.DataSource = new WattTimeDataSource(this.Logger.Object, this.WattTimeClient.Object, this.LocationSource.Object); this.DefaultLocation = new Location() { Name = "eastus" }; - this.DefaultRegion = new RegionResponse() { Region = "BA" }; + this.DefaultRegion = new RegionResponse() { Region = "TEST_REGION", RegionFullName = "Test Region Full Name", SignalType = SignalTypes.co2_moer }; this.DefaultDataStartTime = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); MockBalancingAuthorityLocationMapping(); } @@ -91,8 +91,8 @@ public async Task GetCarbonIntensity_ReturnsEmptyListWhenNoRecordsFound() this.WattTimeClient.Setup(w => w.GetDataAsync( this.DefaultRegion, - startDate, - endDate) + It.IsAny(), + It.IsAny()) ).ReturnsAsync(() => new GridEmissionsDataResponse()); var result = await this.DataSource.GetCarbonIntensityAsync(new List() { this.DefaultLocation }, startDate, endDate); diff --git a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs index 975d00d04..fc19dde48 100644 --- a/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs +++ b/src/CarbonAware.WebApi/test/integrationTests/CarbonAwareControllerTests.cs @@ -280,7 +280,7 @@ public async Task EmissionsForecastsBatch_SupportedDataSources_ReturnsOk(string var expectedRequestedAt = DateTimeOffset.Parse(reqAt); var expectedDataStartAt = DateTimeOffset.Parse(start); var expectedDataEndAt = DateTimeOffset.Parse(end); - _dataSourceMocker?.SetupBatchForecastMock(); + _dataSourceMocker?.SetupHistoricalBatchForecastMock(); var inputData = Enumerable.Range(0, nelems).Select(x => new { requestedAt = reqAt, diff --git a/src/CarbonAware/src/Interfaces/IDataSourceMocker.cs b/src/CarbonAware/src/Interfaces/IDataSourceMocker.cs index ae28a30f0..a8557bca3 100644 --- a/src/CarbonAware/src/Interfaces/IDataSourceMocker.cs +++ b/src/CarbonAware/src/Interfaces/IDataSourceMocker.cs @@ -15,7 +15,7 @@ internal interface IDataSourceMocker public abstract void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string location); public abstract void SetupForecastMock(); - public abstract void SetupBatchForecastMock(); + public abstract void SetupHistoricalBatchForecastMock(); /// /// Initializes the DataSourceMocker with clean setup From fbfcac1558d8990404c0605aad9325e220ab29c5 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Tue, 18 Jun 2024 16:13:23 +1000 Subject: [PATCH 20/31] Further test updates Further test updates --- .../test/Client/TestData.cs | 1 + .../test/Client/WattTimeClientTests.cs | 10 +++++----- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs index 699f46991..6609d2d1b 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs @@ -18,6 +18,7 @@ public class TestDataConstants public static DateTime Date = new DateTime(2099, 1, 1, 0, 0, 0); public const float Value = 999.99f; public const string Version = "1.0"; + public const string SignalType = SignalTypes.co2_moer; } internal static string GetGridDataResponseJsonString() diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index c6715f79c..48d014ea3 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -121,7 +121,7 @@ public async Task GetDataAsync_DeserializesExpectedResponse() this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => { - return r.RequestUri!.ToString().Equals($"https://api.watttime.org/v3/data?region=region&starttime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&endtime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type={SignalTypes.co2_moer}") && r.Method == HttpMethod.Get; + return r.RequestUri!.ToString().Equals("https://api.watttime.org/v3/historical?region=region&start=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&end=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type=co2_moer") && r.Method == HttpMethod.Get; }, System.Net.HttpStatusCode.OK, TestData.GetGridDataResponseJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -131,12 +131,12 @@ public async Task GetDataAsync_DeserializesExpectedResponse() Assert.IsTrue(emissionsResponse.Data.Count() > 0); var meta = emissionsResponse.Meta; - Assert.AreEqual("region", meta.Region); - Assert.AreEqual("signal_type", meta.SignalType); + Assert.AreEqual(TestData.TestDataConstants.Region, meta.Region); + Assert.AreEqual(TestData.TestDataConstants.SignalType, meta.SignalType); var gridDataPoint = emissionsResponse.Data.ToList().First(); Assert.AreEqual(300, gridDataPoint.Frequency); - Assert.AreEqual("mkt", gridDataPoint.Market); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), gridDataPoint.PointTime); + Assert.AreEqual(TestData.TestDataConstants.Market, gridDataPoint.Market); + Assert.AreEqual(TestData.TestDataConstants.PointTime, gridDataPoint.PointTime); Assert.AreEqual("999.99", gridDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", gridDataPoint.Version); } From 9aabd27438d1d7e21de830314af6658ceaaad789 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Tue, 18 Jun 2024 17:13:28 +1000 Subject: [PATCH 21/31] Further updates, just 1 test left to remediate Further updates, just 1 test left to remediate --- .../test/Client/WattTimeClientTests.cs | 14 +++++++------- .../test/WattTimeDataSourceTests.cs | 5 +++-- 2 files changed, 10 insertions(+), 9 deletions(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index 48d014ea3..8952db62d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -152,7 +152,7 @@ public async Task GetDataAsync_RefreshesTokenWhenExpired() var emissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(), new DateTimeOffset()); Assert.IsTrue(emissionsResponse.Data.Count() > 0); - Assert.AreEqual("region", emissionsResponse.Meta.Region); + Assert.AreEqual(TestData.TestDataConstants.Region, emissionsResponse.Meta.Region); } [Test] @@ -165,7 +165,7 @@ public async Task GetDataAsync_RefreshesTokenWhenNoneSet() var gridEmissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(), new DateTimeOffset()); Assert.IsTrue(gridEmissionsResponse.Data.Count() > 0); - Assert.AreEqual("region", gridEmissionsResponse.Meta.Region); + Assert.AreEqual(TestData.TestDataConstants.Region, gridEmissionsResponse.Meta.Region); } [Test] @@ -241,15 +241,15 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => { - return r.RequestUri!.ToString().Equals("https://api2.watttime.org/v2/forecast?ba=balauth&starttime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&endtime=2022-04-22T00%3a00%3a00.0000000%2b00%3a00") && r.Method == HttpMethod.Get; + return r.RequestUri!.ToString().Equals("https://api.watttime.org/v3/forecast/historical?region=region&start=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&end=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type=co2_moer") && r.Method == HttpMethod.Get; }, System.Net.HttpStatusCode.OK, TestData.GetHistoricalForecastDataJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new RegionResponse() { Region = "region" }; + var region = new RegionResponse() { Region = "region" }; - var forecastResponse = await client.GetForecastOnDateAsync(ba.Region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); - var overloadedForecast = await client.GetForecastOnDateAsync(ba, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); + var forecastResponse = await client.GetForecastOnDateAsync(region.Region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); + var overloadedForecast = await client.GetForecastOnDateAsync(region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); Assert.AreEqual(forecastResponse!.Meta.GeneratedAt, overloadedForecast!.Meta.GeneratedAt); Assert.AreEqual(forecastResponse.Data[0].Forecast.First(), overloadedForecast.Data[0].Forecast.First()); @@ -259,7 +259,7 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() var forecastDataPoint = forecastResponse.Data[0].Forecast.ToList().First(); Assert.AreEqual(TestData.TestDataConstants.PointTime, forecastDataPoint.PointTime); - Assert.AreEqual(TestData.TestDataConstants.Value, forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues + Assert.AreEqual(TestData.TestDataConstants.Value.ToString("0.00", CultureInfo.InvariantCulture), forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", forecastDataPoint.Version); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs index b1620205a..d20889600 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs @@ -1,4 +1,5 @@ using CarbonAware.DataSources.WattTime.Client; +using CarbonAware.DataSources.WattTime.Client.Tests; using CarbonAware.DataSources.WattTime.Constants; using CarbonAware.DataSources.WattTime.Model; using CarbonAware.Exceptions; @@ -119,7 +120,7 @@ public async Task GetCarbonIntensityForecastAsync_ReturnsResultsWhenRecordsFound // Arrange var startDate = this.DefaultDataStartTime; var endDate = startDate.AddMinutes(1); - var generatedAt = new DateTimeOffset(2022, 4, 18, 12, 30, 00, TimeSpan.FromHours(-6)); + var generatedAt = TestData.TestDataConstants.GeneratedAt;// new DateTimeOffset(2022, 4, 18, 12, 30, 00, TimeSpan.FromHours(-6)); var lbsPerMwhEmissions = 10; var gPerKwhEmissions = this.DataSource.ConvertMoerToGramsPerKilowattHour(lbsPerMwhEmissions); var expectedDuration = TimeSpan.FromMinutes(5); @@ -324,7 +325,7 @@ private HistoricalForecastEmissionsDataResponse GenerateHistoricalForecastRespon new HistoricalEmissionsData() { Forecast = data, - GeneratedAt = DateTimeOffset.Now + GeneratedAt = TestData.TestDataConstants.GeneratedAt } }, Meta = meta From 8127a92a28c13e295d6ad618f707c1e817aa92b2 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Tue, 18 Jun 2024 17:48:20 +1000 Subject: [PATCH 22/31] Updated to add authentication client to the service builder for the tests Updated to add authentication client to the service builder for the tests. All tests now passing. --- .../src/Client/WattTimeClient.cs | 2 +- .../ServiceCollectionExtensions.cs | 20 +++++++++++------- .../ServiceCollectionExtensionTests.cs | 21 +++++++++++-------- 3 files changed, 25 insertions(+), 18 deletions(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index 670b34a4f..d602828c8 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -215,7 +215,7 @@ private async Task GetStreamWithAuthRetryAsync(string uriPath) private async Task EnsureTokenAsync() { - if (this._authenticationClient.DefaultRequestHeaders.Authorization == null) + if (this._client.DefaultRequestHeaders.Authorization == null) { await this.UpdateAuthTokenAsync(); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/ServiceCollectionExtensions.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/ServiceCollectionExtensions.cs index b3ba9d358..d05f1a33a 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/ServiceCollectionExtensions.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/ServiceCollectionExtensions.cs @@ -39,6 +39,7 @@ private static void AddWattTimeClient(IServiceCollection services, IConfiguratio }); var httpClientBuilder = services.AddHttpClient(IWattTimeClient.NamedClient); + var authenticationClientBuilder = services.AddHttpClient(IWattTimeClient.NamedAuthenticationClient); var Proxy = configSection.GetSection("Proxy").Get(); if (Proxy != null && Proxy.UseProxy == true) @@ -47,15 +48,18 @@ private static void AddWattTimeClient(IServiceCollection services, IConfiguratio { throw new Exceptions.ConfigurationException("Proxy Url is not configured."); } - httpClientBuilder.ConfigurePrimaryHttpMessageHandler(() => - new HttpClientHandler() { - Proxy = new WebProxy { - Address = new Uri(Proxy.Url), - Credentials = new NetworkCredential(Proxy.Username, Proxy.Password), - BypassProxyOnLocal = true - } + var handler = new HttpClientHandler() + { + Proxy = new WebProxy + { + Address = new Uri(Proxy.Url), + Credentials = new NetworkCredential(Proxy.Username, Proxy.Password), + BypassProxyOnLocal = true } - ); + }; + + httpClientBuilder.ConfigurePrimaryHttpMessageHandler(() => handler ); + authenticationClientBuilder.ConfigurePrimaryHttpMessageHandler(() => handler ); } services.TryAddSingleton(); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs index 653b2f18e..f6ee78614 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Configuration/ServiceCollectionExtensionTests.cs @@ -26,8 +26,10 @@ class ServiceCollectionExtensionTests private readonly string ProxyPassword = $"DataSources:Configurations:WattTimeTest:Proxy:Password"; private readonly string UseProxyKey = $"DataSources:Configurations:WattTimeTest:Proxy:UseProxy"; + + [Test] - public void ClientProxyTest_With_Invalid_ProxyURL_ThrowsException() + public void ClientProxyTest_With_Missing_ProxyURL_ThrowsException() { // Arrange var settings = new Dictionary { @@ -35,7 +37,6 @@ public void ClientProxyTest_With_Invalid_ProxyURL_ThrowsException() { EmissionsDataSourceKey, EmissionsDataSourceValue }, { UsernameKey, Username }, { PasswordKey, Password }, - { ProxyUrl, "http://fakeproxy:8080" }, { UseProxyKey, "true" }, }; @@ -44,16 +45,14 @@ public void ClientProxyTest_With_Invalid_ProxyURL_ThrowsException() .AddEnvironmentVariables() .Build(); var serviceCollection = new ServiceCollection(); - serviceCollection.AddWattTimeForecastDataSource(configuration.DataSources()); - var serviceProvider = serviceCollection.BuildServiceProvider(); - var client = serviceProvider.GetRequiredService(); // Act & Assert - Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); + Assert.Throws(() => serviceCollection.AddWattTimeForecastDataSource(configuration.DataSources())); + Assert.Throws(() => serviceCollection.AddWattTimeEmissionsDataSource(configuration.DataSources())); } [Test] - public void ClientProxyTest_With_Missing_ProxyURL_ThrowsException() + public void ClientProxyTest_With_Invalid_ProxyURL_ThrowsException() { // Arrange var settings = new Dictionary { @@ -61,6 +60,7 @@ public void ClientProxyTest_With_Missing_ProxyURL_ThrowsException() { EmissionsDataSourceKey, EmissionsDataSourceValue }, { UsernameKey, Username }, { PasswordKey, Password }, + { ProxyUrl, "http://fakeproxy:8080" }, { UseProxyKey, "true" }, }; @@ -69,10 +69,13 @@ public void ClientProxyTest_With_Missing_ProxyURL_ThrowsException() .AddEnvironmentVariables() .Build(); var serviceCollection = new ServiceCollection(); + serviceCollection.AddWattTimeForecastDataSource(configuration.DataSources()); + + var serviceProvider = serviceCollection.BuildServiceProvider(); + var client = serviceProvider.GetRequiredService(); // Act & Assert - Assert.Throws(() => serviceCollection.AddWattTimeForecastDataSource(configuration.DataSources())); - Assert.Throws(() => serviceCollection.AddWattTimeEmissionsDataSource(configuration.DataSources())); + Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); } [TestCase(true, TestName = "ClientProxyTest, successful: denotes adding WattTime data sources using proxy.")] From 4ff0151bd1528a6e00ff565e906d36caed86a0fb Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Wed, 19 Jun 2024 08:08:16 +1000 Subject: [PATCH 23/31] Renaming of Balancing Authority to Region Renaming of Balancing Authority to Region through all code and comments. This will also need updating through documentation. --- .../src/Client/IWattTimeClient.cs | 56 +++++++++---------- .../src/Client/WattTimeClient.cs | 38 ++++++------- .../src/Client/WattTimeClientHttpException.cs | 2 +- .../WattTimeClientConfiguration.cs | 4 +- .../src/Model/GridEmissionDataPoint.cs | 2 +- .../src/Model/RegionResponse.cs | 2 +- .../src/WattTimeDataSource.cs | 26 ++++----- .../test/Client/WattTimeClientTests.cs | 50 ++++++++--------- .../test/WattTimeDataSourceTests.cs | 4 +- 9 files changed, 92 insertions(+), 92 deletions(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs index 73d432861..f69cbf11c 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/IWattTimeClient.cs @@ -11,90 +11,90 @@ internal interface IWattTimeClient public const string NamedAuthenticationClient = "WattTimeAuthenticationClient"; /// - /// Async method to get observed emission data for a given balancing authority and time period. + /// Async method to get observed emission data for a given region and time period. /// - /// Balancing authority abbreviation + /// Region abbreviation /// Start time of the time period /// End time of the time period /// An which contains all emissions data points in a period. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetDataAsync(string balancingAuthorityAbbreviation, DateTimeOffset startTime, DateTimeOffset endTime); + Task GetDataAsync(string regionAbbreviation, DateTimeOffset startTime, DateTimeOffset endTime); /// - /// Async method to get observed emission data for a given balancing authority and time period. + /// Async method to get observed emission data for a given region and time period. /// - /// Balancing authority + /// Region /// Start time of the time period /// End time of the time period /// An which contains all emissions data points in a period. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetDataAsync(RegionResponse balancingAuthority, DateTimeOffset startTime, DateTimeOffset endTime); + Task GetDataAsync(RegionResponse region, DateTimeOffset startTime, DateTimeOffset endTime); /// - /// Async method to get the most recent 24 hour forecasted emission data for a given balancing authority. + /// Async method to get the most recent 24 hour forecasted emission data for a given region. /// - /// Balancing authority abbreviation + /// region abbreviation /// An which contains forecasted emissions data points. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetCurrentForecastAsync(string balancingAuthorityAbbreviation); + Task GetCurrentForecastAsync(string regionAbbreviation); /// - /// Async method to get the most recent 24 hour forecasted emission data for a given balancing authority. + /// Async method to get the most recent 24 hour forecasted emission data for a given region. /// - /// Balancing authority + /// region /// An which contains forecasted emissions data points. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetCurrentForecastAsync(RegionResponse balancingAuthority); + Task GetCurrentForecastAsync(RegionResponse region); /// - /// Async method to get generated forecast at requested time and balancing authority. + /// Async method to get generated forecast at requested time and region. /// - /// Balancing authority abbreviation + /// region abbreviation /// The historical time used to fetch the most recent forecast generated as of that time. /// An which contains forecasted emissions data points or null if no Forecast generated at the requested time. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. Task GetForecastOnDateAsync(string region, DateTimeOffset requestedAt); /// - /// Async method to get generated forecast at requested time and balancing authority. + /// Async method to get generated forecast at requested time and region. /// - /// Balancing authority + /// region /// The historical time used to fetch the most recent forecast generated as of that time. /// An which contains forecasted emissions data points or null if no Forecast generated at the requested time. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetForecastOnDateAsync(RegionResponse balancingAuthority, DateTimeOffset requestedAt); + Task GetForecastOnDateAsync(RegionResponse region, DateTimeOffset requestedAt); /// - /// Async method to get the balancing authority for a given location. + /// Async method to get the region for a given location. /// /// Latitude of the location /// Longitude of the location - /// An which contains the balancing authority details. + /// An which contains the region details. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. Task GetRegionAsync(string latitude, string longitude); /// - /// Async method to get the balancing authority abbreviation for a given location. + /// Async method to get the region abbreviation for a given location. /// /// Latitude of the location /// Longitude of the location - /// A string which contains the balancing authority details. + /// A string which contains the region details. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetBalancingAuthorityAbbreviationAsync(string latitude, string longitude); + Task GetRegionAbbreviationAsync(string latitude, string longitude); /// - /// Async method to get binary data (representing a zip file) of the historical emissions data for the given balancing authority. + /// Async method to get binary data (representing a zip file) of the historical emissions data for the given region. /// - /// Balancing authority abbreviation + /// region abbreviation /// An which contains the binary data stream of the .zip file. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetHistoricalDataAsync(string balancingAuthorityAbbreviation); + Task GetHistoricalDataAsync(string regionAbbreviation); /// - /// Async method to get binary data (representing a zip file) of the historical emissions data for the given balancing authority. + /// Async method to get binary data (representing a zip file) of the historical emissions data for the given region. /// - /// Balancing authority + /// region /// An which contains the data Stream of the .zip file. /// Can be thrown when errors occur connecting to WattTime client. See the WattTimeClientException class for documentation of expected status codes. - Task GetHistoricalDataAsync(RegionResponse balancingAuthority); + Task GetHistoricalDataAsync(RegionResponse region); } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index d602828c8..08e5c6d49 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -88,7 +88,7 @@ public Task GetDataAsync(RegionResponse region, DateT public async Task GetCurrentForecastAsync(string region) { - _log.LogInformation("Requesting current forecast from balancing authority {balancingAuthority}", region); + _log.LogInformation("Requesting current forecast from region: {region}", region); var parameters = new Dictionary() { @@ -109,15 +109,15 @@ public async Task GetCurrentForecastAsync(string } /// - public Task GetCurrentForecastAsync(RegionResponse balancingAuthority) + public Task GetCurrentForecastAsync(RegionResponse region) { - return this.GetCurrentForecastAsync(balancingAuthority.Region); + return this.GetCurrentForecastAsync(region.Region); } /// public async Task GetForecastOnDateAsync(string region, DateTimeOffset requestedAt) { - _log.LogInformation($"Requesting forecast from balancingAuthority {region} generated at {requestedAt}."); + _log.LogInformation($"Requesting forecast from region {region} generated at {requestedAt}."); var parameters = new Dictionary() { @@ -147,24 +147,24 @@ public Task GetCurrentForecastAsync(RegionRespons /// public async Task GetRegionAsync(string latitude, string longitude) { - _log.LogInformation("Requesting balancing authority for lattitude {lattitude} and longitude {longitude}", latitude, longitude); - return await GetBalancingAuthorityFromCacheAsync(latitude, longitude); + _log.LogInformation("Requesting region for lattitude {lattitude} and longitude {longitude}", latitude, longitude); + return await GetRegionFromCacheAsync(latitude, longitude); } /// - public async Task GetBalancingAuthorityAbbreviationAsync(string latitude, string longitude) + public async Task GetRegionAbbreviationAsync(string latitude, string longitude) { return (await this.GetRegionAsync(latitude, longitude))?.Region; } /// - public async Task GetHistoricalDataAsync(string balancingAuthorityAbbreviation) + public async Task GetHistoricalDataAsync(string regionAbbreviation) { - _log.LogInformation("Requesting historical data for balancing authority {balancingAuthority}", balancingAuthorityAbbreviation); + _log.LogInformation("Requesting historical data for region {regionAbbreviation}", regionAbbreviation); var parameters = new Dictionary() { - { QueryStrings.Region, balancingAuthorityAbbreviation } + { QueryStrings.Region, regionAbbreviation } }; var url = BuildUrlWithQueryString(Paths.Historical, parameters); @@ -179,9 +179,9 @@ public async Task GetHistoricalDataAsync(string balancingAuthorityAbbrev } /// - public Task GetHistoricalDataAsync(RegionResponse balancingAuthority) + public Task GetHistoricalDataAsync(RegionResponse region) { - return this.GetHistoricalDataAsync(balancingAuthority.Region); + return this.GetHistoricalDataAsync(region.Region); } private async Task GetAsyncWithAuthRetry(string uriPath) @@ -293,10 +293,10 @@ private string BuildUrlWithQueryString(string url, IDictionary q return result; } - private async Task GetBalancingAuthorityFromCacheAsync(string latitude, string longitude) + private async Task GetRegionFromCacheAsync(string latitude, string longitude) { var key = new Tuple(latitude, longitude); - var balancingAuthority = await this._memoryCache.GetOrCreateAsync(key, async entry => + var region = await this._memoryCache.GetOrCreateAsync(key, async entry => { var parameters = new Dictionary() { @@ -312,11 +312,11 @@ private async Task GetBalancingAuthorityFromCacheAsync(string la { QueryStrings.SignalType, SignalTypes.co2_moer } }; var result = await this.MakeRequestGetStreamAsync(Paths.RegionFromLocation, parameters, tags); - var baValue = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting Balancing Authority for latitude {latitude} and longitude {longitude}"); - entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(_configuration.BalancingAuthorityCacheTTL); - entry.Value = baValue; - return baValue; + var regionResponse = await JsonSerializer.DeserializeAsync(result, _options) ?? throw new WattTimeClientException($"Error getting Region for latitude {latitude} and longitude {longitude}"); + entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(_configuration.RegionCacheTTL); + entry.Value = regionResponse; + return regionResponse; }); - return balancingAuthority; + return region; } } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientHttpException.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientHttpException.cs index 39f9cacf4..58e31eb8c 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientHttpException.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClientHttpException.cs @@ -22,7 +22,7 @@ public WattTimeClientHttpException(string message, HttpResponseMessage response) /// Gets the status code for the exception. See remarks for the status codes that can be returned. /// /// - /// 400: Returned when the lattitude/longitude provided aren't associated with a known balancing authority. + /// 400: Returned when the lattitude/longitude provided aren't associated with a known region. /// 401: Returned when no authorization header is passed. You should not expect to receive this status code. /// 403: Returned when an invalid username or password is used for login. Please check your configuration and verify your account when this error is received. /// 429: Returned when the number of requests has exceeded the WattTime rate limit, currently at 3,000 per rolling 5 minute window. For current limits, see https://www.watttime.org/api-documentation/#restrictions diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs index 8a8314baf..a586523ee 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Configuration/WattTimeClientConfiguration.cs @@ -32,10 +32,10 @@ internal class WattTimeClientConfiguration public string AuthenticationBaseUrl { get; set; } = "https://api.watttime.org/"; /// - /// Gets or sets the cached expiration time (in seconds) for a BalancingAuthority instance. + /// Gets or sets the cached expiration time (in seconds) for a Region instance. /// It defaults to 86400 secs. /// - public int BalancingAuthorityCacheTTL { get; set; } = 86400; + public int RegionCacheTTL { get; set; } = 86400; /// /// Validate that this object is properly configured. diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs index 97eed6b89..3f0c0d9e0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/GridEmissionDataPoint.cs @@ -3,7 +3,7 @@ namespace CarbonAware.DataSources.WattTime.Model; /// -/// An object describing the emissions for a given time period and balancing authority. +/// An object describing the emissions for a given time period and region. /// [Serializable] internal record GridEmissionDataPoint diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/RegionResponse.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/RegionResponse.cs index 19632061d..ee334c67e 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/RegionResponse.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Model/RegionResponse.cs @@ -3,7 +3,7 @@ namespace CarbonAware.DataSources.WattTime.Model; /// -/// The details of the balancing authority (BA) serving a particular location. +/// The details of the region serving a particular location. /// [Serializable] internal record RegionResponse diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs index 7f71cb842..cca777aaa 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/WattTimeDataSource.cs @@ -35,7 +35,7 @@ internal class WattTimeDataSource : IEmissionsDataSource, IForecastDataSource /// /// The logger for the datasource /// The WattTime Client - /// The location source to be used to convert a location to BA's. + /// The location source to be used to convert a location to named region's. public WattTimeDataSource(ILogger logger, IWattTimeClient client, ILocationSource locationSource) { this.Logger = logger; @@ -60,9 +60,9 @@ public async Task> GetCarbonIntensityAsync(IEnumerabl public async Task> GetCarbonIntensityAsync(Location location, DateTimeOffset periodStartTime, DateTimeOffset periodEndTime) { Logger.LogInformation($"Getting carbon intensity for location {location} for period {periodStartTime} to {periodEndTime}."); - var balancingAuthority = await this.GetBalancingAuthority(location); + var region = await this.GetRegion(location); var (newStartTime, newEndTime) = IntervalHelper.ExtendTimeByWindow(periodStartTime, periodEndTime, MinSamplingWindow); - var historicalResponse = await this.WattTimeClient.GetDataAsync(balancingAuthority, newStartTime, newEndTime); + var historicalResponse = await this.WattTimeClient.GetDataAsync(region, newStartTime, newEndTime); if (Logger.IsEnabled(LogLevel.Debug)) { Logger.LogDebug($"Found {historicalResponse.Data.Count()} total forecasts for location {location} for period {periodStartTime} to {periodEndTime}."); @@ -81,8 +81,8 @@ public async Task> GetCarbonIntensityAsync(Location l public async Task GetCurrentCarbonIntensityForecastAsync(Location location) { this.Logger.LogInformation($"Getting carbon intensity forecast for location {location}"); - var balancingAuthority = await this.GetBalancingAuthority(location); - var forecast = await this.WattTimeClient.GetCurrentForecastAsync(balancingAuthority); + var region = await this.GetRegion(location); + var forecast = await this.WattTimeClient.GetCurrentForecastAsync(region); return ForecastToEmissionsForecast(forecast, location, DateTimeOffset.UtcNow); } @@ -90,9 +90,9 @@ public async Task GetCurrentCarbonIntensityForecastAsync(Loca public async Task GetHistoricalCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt) { this.Logger.LogInformation($"Getting carbon intensity forecast for location {location} requested at {requestedAt}"); - var balancingAuthority = await this.GetBalancingAuthority(location); + var region = await this.GetRegion(location); var roundedRequestedAt = TimeToLowestIncrement(requestedAt); - var forecast = await this.WattTimeClient.GetForecastOnDateAsync(balancingAuthority, roundedRequestedAt); + var forecast = await this.WattTimeClient.GetForecastOnDateAsync(region, roundedRequestedAt); if (forecast == null) { var ex = new ArgumentException($"No forecast was generated at the requested time {roundedRequestedAt}"); @@ -190,23 +190,23 @@ private TimeSpan FrequencyToTimeSpanOrDefault(int? frequency, TimeSpan defaultVa return (frequency != null) ? TimeSpan.FromSeconds((double)frequency) : defaultValue; } - private async Task GetBalancingAuthority(Location location) + private async Task GetRegion(Location location) { - RegionResponse balancingAuthority; + RegionResponse region; try { var geolocation = await this.LocationSource.ToGeopositionLocationAsync(location); - balancingAuthority = await WattTimeClient.GetRegionAsync(geolocation.LatitudeAsCultureInvariantString(), geolocation.LongitudeAsCultureInvariantString()); + region = await WattTimeClient.GetRegionAsync(geolocation.LatitudeAsCultureInvariantString(), geolocation.LongitudeAsCultureInvariantString()); } catch (Exception ex) when (ex is LocationConversionException || ex is WattTimeClientHttpException) { - Logger.LogError(ex, "Failed to convert the location {location} into a Balancing Authority.", location); + Logger.LogError(ex, "Failed to convert the location {location} into a Region.", location); throw; } - Logger.LogDebug("Converted location {location} to balancing authority {balancingAuthorityAbbreviation}", location, balancingAuthority.Region); + Logger.LogDebug("Converted location {location} to region {region}", location, region.Region); - return balancingAuthority; + return region; } private DateTimeOffset TimeToLowestIncrement(DateTimeOffset date, int minutes = 5) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index 8952db62d..454211169 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -71,12 +71,12 @@ public void AllPublicMethods_ThrowsWhenInvalidLogin() this.BasicAuthValue = "invalid"; var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - Assert.ThrowsAsync(async () => await client.GetDataAsync("ba", new DateTimeOffset(), new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync("ba")); - Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync("ba", new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetDataAsync(TestData.TestDataConstants.Region, new DateTimeOffset(), new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(TestData.TestDataConstants.Region)); + Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(TestData.TestDataConstants.Region, new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); - Assert.ThrowsAsync(async () => await client.GetBalancingAuthorityAbbreviationAsync("lat", "long")); - Assert.ThrowsAsync(async () => await client.GetHistoricalDataAsync("ba")); + Assert.ThrowsAsync(async () => await client.GetRegionAbbreviationAsync("lat", "long")); + Assert.ThrowsAsync(async () => await client.GetHistoricalDataAsync(TestData.TestDataConstants.Region)); } [Test] @@ -86,15 +86,15 @@ public void AllPublicMethods_ThrowClientException_WhenNull() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new RegionResponse() { Region = "balauth" }; + var region = new RegionResponse() { Region = "balauth" }; Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); - Assert.ThrowsAsync(async () => await client.GetDataAsync(ba.Region, new DateTimeOffset(), new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetDataAsync(ba, new DateTimeOffset(), new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba.Region)); - Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba)); - Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba.Region, new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba, new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetDataAsync(region.Region, new DateTimeOffset(), new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetDataAsync(region, new DateTimeOffset(), new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(region.Region)); + Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(region)); + Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(region.Region, new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(region, new DateTimeOffset())); } [Test] @@ -104,15 +104,15 @@ public void AllPublicMethods_ThrowJsonException_WhenBadJsonIsReturned() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var ba = new RegionResponse() { Region = "balauth" }; + var region = new RegionResponse() { Region = TestData.TestDataConstants.Region }; Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); - Assert.ThrowsAsync(async () => await client.GetDataAsync(ba.Region, new DateTimeOffset(), new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetDataAsync(ba, new DateTimeOffset(), new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba.Region)); - Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(ba)); - Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba.Region, new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(ba, new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetDataAsync(region.Region, new DateTimeOffset(), new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetDataAsync(region, new DateTimeOffset(), new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(region.Region)); + Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(region)); + Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(region.Region, new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(region, new DateTimeOffset())); } [Test] @@ -297,7 +297,7 @@ public async Task GetForecastOnDateAsync_RefreshesTokenWhenNoneSet() } [Test] - public async Task GetBalancingAuthorityAsync_DeserializesExpectedResponse() + public async Task GetRegionAsync_DeserializesExpectedResponse() { this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => @@ -318,7 +318,7 @@ public async Task GetBalancingAuthorityAsync_DeserializesExpectedResponse() } [Test] - public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenExpired() + public async Task GetRegionAsync_RefreshesTokenWhenExpired() { this.SetupBasicHandlers(TestData.GetRegionJsonString(), "REFRESHTOKEN"); @@ -332,7 +332,7 @@ public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenExpired() } [Test] - public async Task GetBalancingAuthorityAsync_RefreshesTokenWhenNoneSet() + public async Task GetRegionAsync_RefreshesTokenWhenNoneSet() { // Override http client mock to remove authorization header Mock.Get(this.HttpClientFactory).Setup(x => x.CreateClient(IWattTimeClient.NamedClient)) @@ -363,7 +363,7 @@ public async Task GetHistoricalDataAsync_StreamsExpectedContent() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var result = await client.GetHistoricalDataAsync("ba"); + var result = await client.GetHistoricalDataAsync(TestData.TestDataConstants.Region); var sr = new StreamReader(result); string streamResult = sr.ReadToEnd(); @@ -390,7 +390,7 @@ public async Task GetHistoricalDataAsync_RefreshesTokenWhenExpired() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var result = await client.GetHistoricalDataAsync("ba"); + var result = await client.GetHistoricalDataAsync(TestData.TestDataConstants.Region); var sr = new StreamReader(result); string streamResult = sr.ReadToEnd(); @@ -408,7 +408,7 @@ public async Task GetHistoricalDataAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var result = await client.GetHistoricalDataAsync("ba"); + var result = await client.GetHistoricalDataAsync(TestData.TestDataConstants.Region); var sr = new StreamReader(result); string streamResult = sr.ReadToEnd(); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs index d20889600..c1e292a26 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs @@ -50,7 +50,7 @@ public void Setup() this.DefaultLocation = new Location() { Name = "eastus" }; this.DefaultRegion = new RegionResponse() { Region = "TEST_REGION", RegionFullName = "Test Region Full Name", SignalType = SignalTypes.co2_moer }; this.DefaultDataStartTime = new DateTimeOffset(2022, 4, 18, 12, 32, 42, TimeSpan.FromHours(-6)); - MockBalancingAuthorityLocationMapping(); + MockRegionLocationMapping(); } [Test] @@ -281,7 +281,7 @@ public async Task GetCarbonIntensity_CalculatesDurationBasedOnFrequency(double[] CollectionAssert.AreEqual(expectedDurationList, actualDurationList); } - private void MockBalancingAuthorityLocationMapping() + private void MockRegionLocationMapping() { this.LocationSource.Setup(r => r.ToGeopositionLocationAsync(this.DefaultLocation)).Returns(Task.FromResult(this.DefaultLocation)); var latitude = this.DefaultLocation.Latitude.ToString(); From d93320f11fe205fd7d5f18cd559c05185ee84056 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Wed, 19 Jun 2024 08:53:53 +1000 Subject: [PATCH 24/31] Fixed spelling error in latitude Fixed spelling error in latitude --- .../src/Client/WattTimeClient.cs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs index 08e5c6d49..3dba1c647 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/src/Client/WattTimeClient.cs @@ -147,7 +147,7 @@ public Task GetCurrentForecastAsync(RegionRespons /// public async Task GetRegionAsync(string latitude, string longitude) { - _log.LogInformation("Requesting region for lattitude {lattitude} and longitude {longitude}", latitude, longitude); + _log.LogInformation("Requesting region for latitude {latitude} and longitude {longitude}", latitude, longitude); return await GetRegionFromCacheAsync(latitude, longitude); } From 91799f778b9b544575837b85f3eba27728218e1f Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Wed, 19 Jun 2024 11:10:05 +1000 Subject: [PATCH 25/31] Fixed a bug where location sources were loading twice Fixed a bug where location sources were loading twice. Added a semaphore to stop any threading issues, and also stopped it loading twice in the service configuration. --- .../src/LocationSource.cs | 43 ++++++++++++++----- .../ServiceCollectionExtensions.cs | 22 ++++++++-- 2 files changed, 51 insertions(+), 14 deletions(-) diff --git a/src/CarbonAware.LocationSources/src/LocationSource.cs b/src/CarbonAware.LocationSources/src/LocationSource.cs index 793b09dd8..96eb5e5a7 100644 --- a/src/CarbonAware.LocationSources/src/LocationSource.cs +++ b/src/CarbonAware.LocationSources/src/LocationSource.cs @@ -6,6 +6,7 @@ using Microsoft.Extensions.Options; using System.Reflection; using System.Text.Json; +using System.Threading; namespace CarbonAware.LocationSources; @@ -22,7 +23,6 @@ internal class LocationSource : ILocationSource private LocationDataSourcesConfiguration _configuration => _configurationMonitor.CurrentValue; - /// /// Creates a new instance of the class. /// @@ -61,14 +61,21 @@ private async Task LoadLocationJsonFileAsync() var sourceFiles = !_configuration.LocationSourceFiles.Any() ? DiscoverFiles() : _configuration.LocationSourceFiles; foreach (var source in sourceFiles) { - using Stream stream = GetStreamFromFileLocation(source); - var namedGeoMap = await JsonSerializer.DeserializeAsync>(stream, options); - foreach (var locationKey in namedGeoMap!.Keys) + if (File.Exists(source.DataFileLocation!)) + { + using Stream stream = GetStreamFromFileLocation(source); + var namedGeoMap = await JsonSerializer.DeserializeAsync>(stream, options); + foreach (var locationKey in namedGeoMap!.Keys) + { + var geoInstance = namedGeoMap[locationKey]; + geoInstance.AssertValid(); + var key = BuildKey(source, locationKey); + AddToLocationMap(key, geoInstance, source.DataFileLocation, keyCounter); + } + } + else { - var geoInstance = namedGeoMap[locationKey]; - geoInstance.AssertValid(); - var key = BuildKey(source, locationKey); - AddToLocationMap(key, geoInstance, source.DataFileLocation, keyCounter); + _logger.LogError($"Configured data source not found at '{source.DataFileLocation}'"); } } } @@ -78,11 +85,27 @@ private String BuildKey(LocationSourceFile locationData, string locationName) return $"{locationData.Prefix}{locationData.Delimiter}{locationName}"; } + /// + /// Semaphore is to stop any concurrent loading of the file, which + /// could in turn update the list twice and result in duplicate entries, + /// in turn resulting in suffixed keys to avoid clashes when not + /// required. + /// + private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(1, 1); private async Task LoadLocationFromFileIfNotPresentAsync() { - if (!_allLocations.Any()) + await _semaphore.WaitAsync(); + + try + { + if (!_allLocations.Any()) + { + await LoadLocationJsonFileAsync(); + } + } + finally { - await LoadLocationJsonFileAsync(); + _semaphore.Release(); } } diff --git a/src/GSF.CarbonAware/src/Configuration/ServiceCollectionExtensions.cs b/src/GSF.CarbonAware/src/Configuration/ServiceCollectionExtensions.cs index 440e657b4..ee38dddad 100644 --- a/src/GSF.CarbonAware/src/Configuration/ServiceCollectionExtensions.cs +++ b/src/GSF.CarbonAware/src/Configuration/ServiceCollectionExtensions.cs @@ -6,6 +6,7 @@ using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.DependencyInjection.Extensions; +using System.Linq; namespace GSF.CarbonAware.Configuration; @@ -26,21 +27,34 @@ private static IServiceCollection ConfigureLocationDataSourcesConfiguration(thi /// public static IServiceCollection AddEmissionsServices(this IServiceCollection services, IConfiguration configuration) { - services.ConfigureLocationDataSourcesConfiguration(configuration); - services.TryAddSingleton(); + AddLocationServices(services, configuration); services.AddDataSourceService(configuration); services.TryAddSingleton(); services.TryAddSingleton(); return services; } + /// + /// This stops the location configuration being loaded twice if needed for + /// historical emissions and forecasted emissions services. + /// + /// + /// + private static void AddLocationServices(IServiceCollection services, IConfiguration configuration) + { + if (!services.Any(x => x.ServiceType == typeof(ILocationSource))) + { + services.ConfigureLocationDataSourcesConfiguration(configuration); + services.TryAddSingleton(); + } + } + /// /// Add services needed in order to use an Forecast service. /// public static IServiceCollection AddForecastServices(this IServiceCollection services, IConfiguration configuration) { - services.ConfigureLocationDataSourcesConfiguration(configuration); - services.TryAddSingleton(); + AddLocationServices(services, configuration); services.AddDataSourceService(configuration); services.TryAddSingleton(); services.TryAddSingleton(); From 466581396933362dbb37c9d6f148b720a39c4599 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Wed, 19 Jun 2024 15:48:53 +1000 Subject: [PATCH 26/31] Fixed typo for method name Fixed typo for method name --- .../src/Configuration/ServiceCollectionExtensions.cs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/GSF.CarbonAware/src/Configuration/ServiceCollectionExtensions.cs b/src/GSF.CarbonAware/src/Configuration/ServiceCollectionExtensions.cs index ee38dddad..e5d0e0ead 100644 --- a/src/GSF.CarbonAware/src/Configuration/ServiceCollectionExtensions.cs +++ b/src/GSF.CarbonAware/src/Configuration/ServiceCollectionExtensions.cs @@ -27,7 +27,7 @@ private static IServiceCollection ConfigureLocationDataSourcesConfiguration(thi /// public static IServiceCollection AddEmissionsServices(this IServiceCollection services, IConfiguration configuration) { - AddLocationServices(services, configuration); + AddLocationService(services, configuration); services.AddDataSourceService(configuration); services.TryAddSingleton(); services.TryAddSingleton(); @@ -40,7 +40,7 @@ public static IServiceCollection AddEmissionsServices(this IServiceCollection se /// /// /// - private static void AddLocationServices(IServiceCollection services, IConfiguration configuration) + private static void AddLocationService(IServiceCollection services, IConfiguration configuration) { if (!services.Any(x => x.ServiceType == typeof(ILocationSource))) { @@ -54,7 +54,7 @@ private static void AddLocationServices(IServiceCollection services, IConfigurat /// public static IServiceCollection AddForecastServices(this IServiceCollection services, IConfiguration configuration) { - AddLocationServices(services, configuration); + AddLocationService(services, configuration); services.AddDataSourceService(configuration); services.TryAddSingleton(); services.TryAddSingleton(); From b228cf7aecd404cbe1a2a36dc2d9e54380c394b4 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Wed, 19 Jun 2024 15:52:08 +1000 Subject: [PATCH 27/31] DCO Remediation Commit for Vaughan Knight I, Vaughan Knight , hereby add my Signed-off-by to this commit: e324f365a3086830f3ac2d0e5ad4d37112fa5c6a I, Vaughan Knight , hereby add my Signed-off-by to this commit: be6663cba48dba0af2ace3db90519377101d2d4d I, Vaughan Knight , hereby add my Signed-off-by to this commit: 39e45a61b999f862b4b3cc7a6adb430b25dfe08d I, Vaughan Knight , hereby add my Signed-off-by to this commit: fbfcac1558d8990404c0605aad9325e220ab29c5 I, Vaughan Knight , hereby add my Signed-off-by to this commit: 9aabd27438d1d7e21de830314af6658ceaaad789 I, Vaughan Knight , hereby add my Signed-off-by to this commit: 8127a92a28c13e295d6ad618f707c1e817aa92b2 I, Vaughan Knight , hereby add my Signed-off-by to this commit: 4ff0151bd1528a6e00ff565e906d36caed86a0fb I, Vaughan Knight , hereby add my Signed-off-by to this commit: d93320f11fe205fd7d5f18cd559c05185ee84056 I, Vaughan Knight , hereby add my Signed-off-by to this commit: 91799f778b9b544575837b85f3eba27728218e1f I, Vaughan Knight , hereby add my Signed-off-by to this commit: 466581396933362dbb37c9d6f148b720a39c4599 Signed-off-by: Vaughan Knight From 81376591ce3cdaf8ed29f8d8608e24d2ae8a453b Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Wed, 19 Jun 2024 16:42:46 +1000 Subject: [PATCH 28/31] Updates based on code review for WattTime Tests Updates based on code review for WattTime Tests. Mostly cleanup of constants which were removed elsewhere in tests. --- ...bonAware.DataSources.WattTime.Mocks.csproj | 1 + .../mock/WattTimeDataSourceMocker.cs | 55 +++++++-------- .../test/Client/WattTimeClientTests.cs | 68 +++++++++---------- .../{TestData.cs => WattTimeTestData.cs} | 25 +++---- .../test/WattTimeDataSourceTests.cs | 4 +- 5 files changed, 73 insertions(+), 80 deletions(-) rename src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/{TestData.cs => WattTimeTestData.cs} (85%) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj index 061c0cbab..1649e1ed9 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj @@ -13,6 +13,7 @@ + diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs index dce0b7625..c21e759f0 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/mock/WattTimeDataSourceMocker.cs @@ -1,4 +1,5 @@ -using CarbonAware.DataSources.WattTime.Constants; +using CarbonAware.DataSources.WattTime.Client.Tests; +using CarbonAware.DataSources.WattTime.Constants; using CarbonAware.DataSources.WattTime.Model; using CarbonAware.Interfaces; using System.Net; @@ -15,9 +16,9 @@ internal class WattTimeDataSourceMocker : IDataSourceMocker private static readonly RegionResponse defaultRegion = new() { - Region = "TEST_REGION", - RegionFullName = "Test Region Full Name", - SignalType = SignalTypes.co2_moer + Region = WattTimeTestData.Constants.Region, + RegionFullName = WattTimeTestData.Constants.RegionFullName, + SignalType = WattTimeTestData.Constants.SignalType }; private static readonly LoginResult defaultLoginResult = new() { Token = "myDefaultToken123" }; @@ -42,13 +43,12 @@ public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string locat var newDataPoint = new GridEmissionDataPoint() { PointTime = pointTime, - Value = 999.99F, - Version = "1.0", - Frequency = 300, - Market = "mkt", + Value = WattTimeTestData.Constants.Value, + Version = WattTimeTestData.Constants.Version, + Frequency = WattTimeTestData.Constants.Frequency, + Market = WattTimeTestData.Constants.Market }; - data.Add(newDataPoint); pointTime = newDataPoint.PointTime + duration; } @@ -56,7 +56,7 @@ public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string locat var meta = new GridEmissionsMetaData() { Region = defaultRegion.Region, - SignalType = SignalTypes.co2_moer + SignalType = WattTimeTestData.Constants.SignalType }; var gridEmissionsResponse = new GridEmissionsDataResponse() @@ -83,11 +83,11 @@ public void SetupForecastMock() { var newForecastPoint = new GridEmissionDataPoint() { - Frequency = 300, - Market = "mkt", + Frequency = WattTimeTestData.Constants.Frequency, + Market = WattTimeTestData.Constants.Market, PointTime = start, Value = currValue, - Version = "1.0" + Version = WattTimeTestData.Constants.Version }; newForecastPoint.PointTime = pointTime; newForecastPoint.Value = currValue; @@ -99,8 +99,8 @@ public void SetupForecastMock() var meta = new GridEmissionsMetaData() { Region = defaultRegion.Region, - SignalType = SignalTypes.co2_moer, - GeneratedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero) + SignalType = WattTimeTestData.Constants.SignalType, + GeneratedAt = WattTimeTestData.Constants.GeneratedAt }; var forecastResponse = new ForecastEmissionsDataResponse() @@ -109,7 +109,6 @@ public void SetupForecastMock() Meta = meta }; - SetupResponseGivenGetRequest(Paths.Forecast, JsonSerializer.Serialize(forecastResponse)); } @@ -124,11 +123,11 @@ public void SetupHistoricalBatchForecastMock() { var newForecastPoint = new GridEmissionDataPoint() { - Frequency = 300, - Market = "mkt", + Frequency = WattTimeTestData.Constants.Frequency, + Market = WattTimeTestData.Constants.Market, PointTime = start, Value = currValue, - Version = "1.0" + Version = WattTimeTestData.Constants.Version }; newForecastPoint.PointTime = pointTime; newForecastPoint.Value = currValue; @@ -140,18 +139,10 @@ public void SetupHistoricalBatchForecastMock() var meta = new GridEmissionsMetaData() { Region = defaultRegion.Region, - SignalType = SignalTypes.co2_moer, - GeneratedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero) + SignalType = WattTimeTestData.Constants.SignalType, + GeneratedAt = WattTimeTestData.Constants.GeneratedAt }; - //var forecastBatchData = new List { - // new ForecastEmissionsDataResponse() - // { - // Data = forecastData, - // Meta = meta - // } - //}; - var historicalForecastResponse = new HistoricalForecastEmissionsDataResponse() { Data = new List() @@ -159,7 +150,7 @@ public void SetupHistoricalBatchForecastMock() new HistoricalEmissionsData() { Forecast = forecastData, - GeneratedAt = new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero) + GeneratedAt = WattTimeTestData.Constants.GeneratedAt } }, Meta = meta @@ -171,7 +162,7 @@ public void SetupHistoricalBatchForecastMock() public void Initialize() { - SetupBaMock(); + SetupRegionMock(); SetupLoginMock(); } @@ -196,7 +187,7 @@ private void SetupResponseGivenGetRequest(string path, string body) .WithBody(body) ); } - private void SetupBaMock(RegionResponse? content = null) => + private void SetupRegionMock(RegionResponse? content = null) => SetupResponseGivenGetRequest(Paths.RegionFromLocation, JsonSerializer.Serialize(content ?? defaultRegion)); private void SetupLoginMock(LoginResult? content = null) => diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index 454211169..13501773b 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -71,12 +71,12 @@ public void AllPublicMethods_ThrowsWhenInvalidLogin() this.BasicAuthValue = "invalid"; var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - Assert.ThrowsAsync(async () => await client.GetDataAsync(TestData.TestDataConstants.Region, new DateTimeOffset(), new DateTimeOffset())); - Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(TestData.TestDataConstants.Region)); - Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(TestData.TestDataConstants.Region, new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetDataAsync(WattTimeTestData.Constants.Region, new DateTimeOffset(), new DateTimeOffset())); + Assert.ThrowsAsync(async () => await client.GetCurrentForecastAsync(WattTimeTestData.Constants.Region)); + Assert.ThrowsAsync(async () => await client.GetForecastOnDateAsync(WattTimeTestData.Constants.Region, new DateTimeOffset())); Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); Assert.ThrowsAsync(async () => await client.GetRegionAbbreviationAsync("lat", "long")); - Assert.ThrowsAsync(async () => await client.GetHistoricalDataAsync(TestData.TestDataConstants.Region)); + Assert.ThrowsAsync(async () => await client.GetHistoricalDataAsync(WattTimeTestData.Constants.Region)); } [Test] @@ -104,7 +104,7 @@ public void AllPublicMethods_ThrowJsonException_WhenBadJsonIsReturned() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var region = new RegionResponse() { Region = TestData.TestDataConstants.Region }; + var region = new RegionResponse() { Region = WattTimeTestData.Constants.Region }; Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); Assert.ThrowsAsync(async () => await client.GetDataAsync(region.Region, new DateTimeOffset(), new DateTimeOffset())); @@ -122,7 +122,7 @@ public async Task GetDataAsync_DeserializesExpectedResponse() this.AddHandler_RequestResponse(r => { return r.RequestUri!.ToString().Equals("https://api.watttime.org/v3/historical?region=region&start=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&end=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type=co2_moer") && r.Method == HttpMethod.Get; - }, System.Net.HttpStatusCode.OK, TestData.GetGridDataResponseJsonString()); + }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetGridDataResponseJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -131,12 +131,12 @@ public async Task GetDataAsync_DeserializesExpectedResponse() Assert.IsTrue(emissionsResponse.Data.Count() > 0); var meta = emissionsResponse.Meta; - Assert.AreEqual(TestData.TestDataConstants.Region, meta.Region); - Assert.AreEqual(TestData.TestDataConstants.SignalType, meta.SignalType); + Assert.AreEqual(WattTimeTestData.Constants.Region, meta.Region); + Assert.AreEqual(WattTimeTestData.Constants.SignalType, meta.SignalType); var gridDataPoint = emissionsResponse.Data.ToList().First(); Assert.AreEqual(300, gridDataPoint.Frequency); - Assert.AreEqual(TestData.TestDataConstants.Market, gridDataPoint.Market); - Assert.AreEqual(TestData.TestDataConstants.PointTime, gridDataPoint.PointTime); + Assert.AreEqual(WattTimeTestData.Constants.Market, gridDataPoint.Market); + Assert.AreEqual(WattTimeTestData.Constants.PointTime, gridDataPoint.PointTime); Assert.AreEqual("999.99", gridDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", gridDataPoint.Version); } @@ -144,7 +144,7 @@ public async Task GetDataAsync_DeserializesExpectedResponse() [Test] public async Task GetDataAsync_RefreshesTokenWhenExpired() { - this.SetupBasicHandlers(TestData.GetGridDataResponseJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(WattTimeTestData.GetGridDataResponseJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -152,20 +152,20 @@ public async Task GetDataAsync_RefreshesTokenWhenExpired() var emissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(), new DateTimeOffset()); Assert.IsTrue(emissionsResponse.Data.Count() > 0); - Assert.AreEqual(TestData.TestDataConstants.Region, emissionsResponse.Meta.Region); + Assert.AreEqual(WattTimeTestData.Constants.Region, emissionsResponse.Meta.Region); } [Test] public async Task GetDataAsync_RefreshesTokenWhenNoneSet() { - this.SetupBasicHandlers(TestData.GetGridDataResponseJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(WattTimeTestData.GetGridDataResponseJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); var gridEmissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(), new DateTimeOffset()); Assert.IsTrue(gridEmissionsResponse.Data.Count() > 0); - Assert.AreEqual(TestData.TestDataConstants.Region, gridEmissionsResponse.Meta.Region); + Assert.AreEqual(WattTimeTestData.Constants.Region, gridEmissionsResponse.Meta.Region); } [Test] @@ -175,7 +175,7 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() this.AddHandler_RequestResponse(r => { return r.RequestUri!.ToString().Equals(@"https://api.watttime.org/v3/forecast?region=region&signal_type=co2_moer") && r.Method == HttpMethod.Get; - }, System.Net.HttpStatusCode.OK, TestData.GetCurrentForecastJsonString()); + }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetCurrentForecastJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -190,7 +190,7 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() Assert.IsNotNull(forecastResponse); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); - Assert.AreEqual(TestData.TestDataConstants.Region, forecastResponse?.Meta.Region); + Assert.AreEqual(WattTimeTestData.Constants.Region, forecastResponse?.Meta.Region); var forecastDataPoint = forecastResponse?.Data.First(); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint?.PointTime); @@ -201,7 +201,7 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() [Test] public async Task GetCurrentForecastAsync_RefreshesTokenWhenExpired() { - this.SetupBasicHandlers(TestData.GetCurrentForecastJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(WattTimeTestData.GetCurrentForecastJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -210,7 +210,7 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenExpired() Assert.IsNotNull(forecastResponse); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); - Assert.AreEqual(TestData.TestDataConstants.Region, forecastResponse?.Meta.Region); + Assert.AreEqual(WattTimeTestData.Constants.Region, forecastResponse?.Meta.Region); } [Test] @@ -224,7 +224,7 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenNoneSet() client.DefaultRequestHeaders.Authorization = null; // Null authorization header return client; }); - this.SetupBasicHandlers(TestData.GetCurrentForecastJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(WattTimeTestData.GetCurrentForecastJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -232,7 +232,7 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenNoneSet() Assert.IsNotNull(forecastResponse); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); - Assert.AreEqual(TestData.TestDataConstants.Region, forecastResponse?.Meta.Region); + Assert.AreEqual(WattTimeTestData.Constants.Region, forecastResponse?.Meta.Region); } [Test] @@ -242,7 +242,7 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() this.AddHandler_RequestResponse(r => { return r.RequestUri!.ToString().Equals("https://api.watttime.org/v3/forecast/historical?region=region&start=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&end=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type=co2_moer") && r.Method == HttpMethod.Get; - }, System.Net.HttpStatusCode.OK, TestData.GetHistoricalForecastDataJsonString()); + }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetHistoricalForecastDataJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -255,18 +255,18 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() Assert.AreEqual(forecastResponse.Data[0].Forecast.First(), overloadedForecast.Data[0].Forecast.First()); Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse.Meta.GeneratedAt); - Assert.AreEqual(TestData.TestDataConstants.Region, forecastResponse.Meta.Region); + Assert.AreEqual(WattTimeTestData.Constants.Region, forecastResponse.Meta.Region); var forecastDataPoint = forecastResponse.Data[0].Forecast.ToList().First(); - Assert.AreEqual(TestData.TestDataConstants.PointTime, forecastDataPoint.PointTime); - Assert.AreEqual(TestData.TestDataConstants.Value.ToString("0.00", CultureInfo.InvariantCulture), forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues + Assert.AreEqual(WattTimeTestData.Constants.PointTime, forecastDataPoint.PointTime); + Assert.AreEqual(WattTimeTestData.Constants.Value.ToString("0.00", CultureInfo.InvariantCulture), forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues Assert.AreEqual("1.0", forecastDataPoint.Version); } [Test] public async Task GetForecastOnDateAsync_RefreshesTokenWhenExpired() { - this.SetupBasicHandlers(TestData.GetHistoricalForecastDataJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(WattTimeTestData.GetHistoricalForecastDataJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -287,7 +287,7 @@ public async Task GetForecastOnDateAsync_RefreshesTokenWhenNoneSet() return client; }); - this.SetupBasicHandlers(TestData.GetHistoricalForecastDataJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(WattTimeTestData.GetHistoricalForecastDataJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -303,7 +303,7 @@ public async Task GetRegionAsync_DeserializesExpectedResponse() this.AddHandler_RequestResponse(r => { return r.RequestUri!.ToString().Equals("https://api.watttime.org/v3/region-from-loc?latitude=lat&longitude=long&signal_type=co2_moer") && r.Method == HttpMethod.Get; - }, System.Net.HttpStatusCode.OK, TestData.GetRegionJsonString()); + }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetRegionJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -312,15 +312,15 @@ public async Task GetRegionAsync_DeserializesExpectedResponse() var regionResponse = await client.GetRegionAsync("lat", "long"); Assert.IsNotNull(regionResponse); - Assert.AreEqual(TestData.TestDataConstants.Region, regionResponse?.Region); - Assert.AreEqual(TestData.TestDataConstants.RegionFullName, regionResponse?.RegionFullName); + Assert.AreEqual(WattTimeTestData.Constants.Region, regionResponse?.Region); + Assert.AreEqual(WattTimeTestData.Constants.RegionFullName, regionResponse?.RegionFullName); Assert.AreEqual(SignalTypes.co2_moer, regionResponse?.SignalType); } [Test] public async Task GetRegionAsync_RefreshesTokenWhenExpired() { - this.SetupBasicHandlers(TestData.GetRegionJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(WattTimeTestData.GetRegionJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); @@ -343,7 +343,7 @@ public async Task GetRegionAsync_RefreshesTokenWhenNoneSet() return client; }); - this.SetupBasicHandlers(TestData.GetRegionJsonString(), "REFRESHTOKEN"); + this.SetupBasicHandlers(WattTimeTestData.GetRegionJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -363,7 +363,7 @@ public async Task GetHistoricalDataAsync_StreamsExpectedContent() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var result = await client.GetHistoricalDataAsync(TestData.TestDataConstants.Region); + var result = await client.GetHistoricalDataAsync(WattTimeTestData.Constants.Region); var sr = new StreamReader(result); string streamResult = sr.ReadToEnd(); @@ -390,7 +390,7 @@ public async Task GetHistoricalDataAsync_RefreshesTokenWhenExpired() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var result = await client.GetHistoricalDataAsync(TestData.TestDataConstants.Region); + var result = await client.GetHistoricalDataAsync(WattTimeTestData.Constants.Region); var sr = new StreamReader(result); string streamResult = sr.ReadToEnd(); @@ -408,7 +408,7 @@ public async Task GetHistoricalDataAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var result = await client.GetHistoricalDataAsync(TestData.TestDataConstants.Region); + var result = await client.GetHistoricalDataAsync(WattTimeTestData.Constants.Region); var sr = new StreamReader(result); string streamResult = sr.ReadToEnd(); diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeTestData.cs similarity index 85% rename from src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs rename to src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeTestData.cs index 6609d2d1b..d46143e4f 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/TestData.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeTestData.cs @@ -6,9 +6,9 @@ namespace CarbonAware.DataSources.WattTime.Client.Tests; -internal static class TestData +public static class WattTimeTestData { - public class TestDataConstants + public class Constants { public const string Region = "TEST_REGION"; public const string RegionFullName = "Test Region Full Name"; @@ -19,6 +19,7 @@ public class TestDataConstants public const float Value = 999.99f; public const string Version = "1.0"; public const string SignalType = SignalTypes.co2_moer; + public const int Frequency = 300; } internal static string GetGridDataResponseJsonString() @@ -39,12 +40,12 @@ private static GridEmissionsMetaData _GetGridDataMetaResponse() { var gridEmissionsMetaData = new GridEmissionsMetaData() { - Region = TestDataConstants.Region, - GeneratedAt = TestDataConstants.GeneratedAt, + Region = Constants.Region, + GeneratedAt = Constants.GeneratedAt, GeneratedAtPeriodSeconds = 30, Model = new GridEmissionsModelData() { - Date = TestDataConstants.Date, + Date = Constants.Date, Type = SignalTypes.co2_moer }, DataPointPeriodSeconds = 30, @@ -67,10 +68,10 @@ private static GridEmissionDataPoint _GetGridEmissionDataPoint() return new GridEmissionDataPoint() { Frequency = 300, - Market = TestDataConstants.Market, - PointTime = TestDataConstants.PointTime, - Value = TestDataConstants.Value, - Version = TestDataConstants.Version + Market = Constants.Market, + PointTime = Constants.PointTime, + Value = Constants.Value, + Version = Constants.Version }; } @@ -104,7 +105,7 @@ private static HistoricalForecastEmissionsDataResponse _GetHistoricalForecastEmi new HistoricalEmissionsData() { Forecast = _GetGridEmissionDataPoints(), - GeneratedAt = TestDataConstants.GeneratedAt + GeneratedAt = Constants.GeneratedAt } } }; @@ -119,8 +120,8 @@ private static RegionResponse _GetRegion() { return new RegionResponse() { - Region = TestDataConstants.Region, - RegionFullName = TestDataConstants.RegionFullName, + Region = Constants.Region, + RegionFullName = Constants.RegionFullName, SignalType = SignalTypes.co2_moer }; } diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs index c1e292a26..8675ee4da 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/WattTimeDataSourceTests.cs @@ -120,7 +120,7 @@ public async Task GetCarbonIntensityForecastAsync_ReturnsResultsWhenRecordsFound // Arrange var startDate = this.DefaultDataStartTime; var endDate = startDate.AddMinutes(1); - var generatedAt = TestData.TestDataConstants.GeneratedAt;// new DateTimeOffset(2022, 4, 18, 12, 30, 00, TimeSpan.FromHours(-6)); + var generatedAt = WattTimeTestData.Constants.GeneratedAt;// new DateTimeOffset(2022, 4, 18, 12, 30, 00, TimeSpan.FromHours(-6)); var lbsPerMwhEmissions = 10; var gPerKwhEmissions = this.DataSource.ConvertMoerToGramsPerKilowattHour(lbsPerMwhEmissions); var expectedDuration = TimeSpan.FromMinutes(5); @@ -325,7 +325,7 @@ private HistoricalForecastEmissionsDataResponse GenerateHistoricalForecastRespon new HistoricalEmissionsData() { Forecast = data, - GeneratedAt = TestData.TestDataConstants.GeneratedAt + GeneratedAt = WattTimeTestData.Constants.GeneratedAt } }, Meta = meta From f64ec8eb3ff8ae0fd01f0e52f2d47294b71adb77 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Thu, 20 Jun 2024 09:35:59 +1000 Subject: [PATCH 29/31] Cleaned up a lot of the string literals Cleaned up a lot of the string literals. They were causing too much fragility in the code base, and made it complex when updating the WattTime API. --- .../test/Client/WattTimeClientTests.cs | 100 +++++++++--------- 1 file changed, 49 insertions(+), 51 deletions(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index 13501773b..2b15c016d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -34,7 +34,8 @@ class WattTimeClientTests private string BasicAuthValue { get; set; } - private readonly string DefaultTokenValue = "myDefaultToken123"; + private readonly string _DEFAULT_TOKEN_VALUE = "myDefaultToken123"; + private readonly string _BASE_WATTTIME_LOGIN_URL = "https://api.watttime.org/login"; private IMemoryCache MemoryCache { get; set; } @@ -56,7 +57,7 @@ public void Initialize() .Returns(() => { var client = Handler.CreateClient(); - client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", this.DefaultTokenValue); + client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _DEFAULT_TOKEN_VALUE); return client; }); @@ -85,8 +86,8 @@ public void AllPublicMethods_ThrowClientException_WhenNull() this.SetupBasicHandlers("null"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var region = new RegionResponse() { Region = "balauth" }; + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); + var region = new RegionResponse() { Region = WattTimeTestData.Constants.Region }; Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); Assert.ThrowsAsync(async () => await client.GetDataAsync(region.Region, new DateTimeOffset(), new DateTimeOffset())); @@ -103,7 +104,7 @@ public void AllPublicMethods_ThrowJsonException_WhenBadJsonIsReturned() this.SetupBasicHandlers("This is bad json"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); var region = new RegionResponse() { Region = WattTimeTestData.Constants.Region }; Assert.ThrowsAsync(async () => await client.GetRegionAsync("lat", "long")); @@ -125,20 +126,20 @@ public async Task GetDataAsync_DeserializesExpectedResponse() }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetGridDataResponseJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); - var emissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); + var emissionsResponse = await client.GetDataAsync(WattTimeTestData.Constants.Region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); Assert.IsTrue(emissionsResponse.Data.Count() > 0); var meta = emissionsResponse.Meta; Assert.AreEqual(WattTimeTestData.Constants.Region, meta.Region); Assert.AreEqual(WattTimeTestData.Constants.SignalType, meta.SignalType); var gridDataPoint = emissionsResponse.Data.ToList().First(); - Assert.AreEqual(300, gridDataPoint.Frequency); + Assert.AreEqual(WattTimeTestData.Constants.Frequency, gridDataPoint.Frequency); Assert.AreEqual(WattTimeTestData.Constants.Market, gridDataPoint.Market); Assert.AreEqual(WattTimeTestData.Constants.PointTime, gridDataPoint.PointTime); - Assert.AreEqual("999.99", gridDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues - Assert.AreEqual("1.0", gridDataPoint.Version); + Assert.AreEqual(WattTimeTestData.Constants.Value.ToString("0.00", CultureInfo.InvariantCulture), gridDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues + Assert.AreEqual(WattTimeTestData.Constants.Version, gridDataPoint.Version); } [Test] @@ -147,9 +148,9 @@ public async Task GetDataAsync_RefreshesTokenWhenExpired() this.SetupBasicHandlers(WattTimeTestData.GetGridDataResponseJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); - var emissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(), new DateTimeOffset()); + var emissionsResponse = await client.GetDataAsync(WattTimeTestData.Constants.Region, new DateTimeOffset(), new DateTimeOffset()); Assert.IsTrue(emissionsResponse.Data.Count() > 0); Assert.AreEqual(WattTimeTestData.Constants.Region, emissionsResponse.Meta.Region); @@ -162,7 +163,7 @@ public async Task GetDataAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var gridEmissionsResponse = await client.GetDataAsync("region", new DateTimeOffset(), new DateTimeOffset()); + var gridEmissionsResponse = await client.GetDataAsync(WattTimeTestData.Constants.Region, new DateTimeOffset(), new DateTimeOffset()); Assert.IsTrue(gridEmissionsResponse.Data.Count() > 0); Assert.AreEqual(WattTimeTestData.Constants.Region, gridEmissionsResponse.Meta.Region); @@ -178,24 +179,22 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetCurrentForecastJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - - const string REGION = "region"; + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); - var forecastResponse = await client.GetCurrentForecastAsync(REGION); - var overloadedForecast = await client.GetCurrentForecastAsync(REGION); + var forecastResponse = await client.GetCurrentForecastAsync(WattTimeTestData.Constants.Region); + var overloadedForecast = await client.GetCurrentForecastAsync(WattTimeTestData.Constants.Region); Assert.AreEqual(forecastResponse.Meta.GeneratedAt, overloadedForecast.Meta.GeneratedAt); Assert.AreEqual(forecastResponse.Data.First(), overloadedForecast.Data.First()); Assert.IsNotNull(forecastResponse); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); + Assert.AreEqual(WattTimeTestData.Constants.GeneratedAt, forecastResponse?.Meta.GeneratedAt); Assert.AreEqual(WattTimeTestData.Constants.Region, forecastResponse?.Meta.Region); var forecastDataPoint = forecastResponse?.Data.First(); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastDataPoint?.PointTime); - Assert.AreEqual("999.99", forecastDataPoint?.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues - Assert.AreEqual("1.0", forecastDataPoint?.Version); + Assert.AreEqual(WattTimeTestData.Constants.PointTime, forecastDataPoint?.PointTime); + Assert.AreEqual(WattTimeTestData.Constants.Value.ToString("0.00", CultureInfo.InvariantCulture), forecastDataPoint?.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues + Assert.AreEqual(WattTimeTestData.Constants.Version, forecastDataPoint?.Version); } [Test] @@ -204,12 +203,12 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenExpired() this.SetupBasicHandlers(WattTimeTestData.GetCurrentForecastJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); - var forecastResponse = await client.GetCurrentForecastAsync("region"); + var forecastResponse = await client.GetCurrentForecastAsync(WattTimeTestData.Constants.Region); Assert.IsNotNull(forecastResponse); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); + Assert.AreEqual(WattTimeTestData.Constants.GeneratedAt, forecastResponse?.Meta.GeneratedAt); Assert.AreEqual(WattTimeTestData.Constants.Region, forecastResponse?.Meta.Region); } @@ -228,10 +227,10 @@ public async Task GetCurrentForecastAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var forecastResponse = await client.GetCurrentForecastAsync("region"); + var forecastResponse = await client.GetCurrentForecastAsync(WattTimeTestData.Constants.Region); Assert.IsNotNull(forecastResponse); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse?.Meta.GeneratedAt); + Assert.AreEqual(WattTimeTestData.Constants.GeneratedAt, forecastResponse?.Meta.GeneratedAt); Assert.AreEqual(WattTimeTestData.Constants.Region, forecastResponse?.Meta.Region); } @@ -245,8 +244,8 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetHistoricalForecastDataJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); - var region = new RegionResponse() { Region = "region" }; + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); + var region = new RegionResponse() { Region = WattTimeTestData.Constants.Region }; var forecastResponse = await client.GetForecastOnDateAsync(region.Region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); var overloadedForecast = await client.GetForecastOnDateAsync(region, new DateTimeOffset(2022, 4, 22, 0, 0, 0, TimeSpan.Zero)); @@ -254,13 +253,13 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() Assert.AreEqual(forecastResponse!.Meta.GeneratedAt, overloadedForecast!.Meta.GeneratedAt); Assert.AreEqual(forecastResponse.Data[0].Forecast.First(), overloadedForecast.Data[0].Forecast.First()); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse.Meta.GeneratedAt); + Assert.AreEqual(WattTimeTestData.Constants.GeneratedAt, forecastResponse.Meta.GeneratedAt); Assert.AreEqual(WattTimeTestData.Constants.Region, forecastResponse.Meta.Region); var forecastDataPoint = forecastResponse.Data[0].Forecast.ToList().First(); Assert.AreEqual(WattTimeTestData.Constants.PointTime, forecastDataPoint.PointTime); Assert.AreEqual(WattTimeTestData.Constants.Value.ToString("0.00", CultureInfo.InvariantCulture), forecastDataPoint.Value.ToString("0.00", CultureInfo.InvariantCulture)); //Format float to avoid precision issues - Assert.AreEqual("1.0", forecastDataPoint.Version); + Assert.AreEqual(WattTimeTestData.Constants.Version, forecastDataPoint.Version); } [Test] @@ -269,10 +268,10 @@ public async Task GetForecastOnDateAsync_RefreshesTokenWhenExpired() this.SetupBasicHandlers(WattTimeTestData.GetHistoricalForecastDataJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); - var forecastResponse = await client.GetForecastOnDateAsync("region", new DateTimeOffset()); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse!.Meta.GeneratedAt); + var forecastResponse = await client.GetForecastOnDateAsync(WattTimeTestData.Constants.Region, new DateTimeOffset()); + Assert.AreEqual(WattTimeTestData.Constants.GeneratedAt, forecastResponse!.Meta.GeneratedAt); } [Test] @@ -291,9 +290,9 @@ public async Task GetForecastOnDateAsync_RefreshesTokenWhenNoneSet() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var forecastResponse = await client.GetForecastOnDateAsync("region", new DateTimeOffset()); + var forecastResponse = await client.GetForecastOnDateAsync(WattTimeTestData.Constants.Region, new DateTimeOffset()); - Assert.AreEqual(new DateTimeOffset(2099, 1, 1, 0, 0, 0, TimeSpan.Zero), forecastResponse!.Meta.GeneratedAt); + Assert.AreEqual(WattTimeTestData.Constants.GeneratedAt, forecastResponse!.Meta.GeneratedAt); } [Test] @@ -307,7 +306,7 @@ public async Task GetRegionAsync_DeserializesExpectedResponse() var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); var regionResponse = await client.GetRegionAsync("lat", "long"); @@ -323,12 +322,12 @@ public async Task GetRegionAsync_RefreshesTokenWhenExpired() this.SetupBasicHandlers(WattTimeTestData.GetRegionJsonString(), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); var regionResponse = await client.GetRegionAsync("lat", "long"); Assert.IsNotNull(regionResponse); - Assert.AreEqual(SignalTypes.co2_moer, regionResponse?.SignalType); + Assert.AreEqual(WattTimeTestData.Constants.SignalType, regionResponse?.SignalType); } [Test] @@ -350,7 +349,7 @@ public async Task GetRegionAsync_RefreshesTokenWhenNoneSet() var regionResponse = await client.GetRegionAsync("lat", "long"); Assert.IsNotNull(regionResponse); - Assert.AreEqual(SignalTypes.co2_moer, regionResponse?.SignalType); + Assert.AreEqual(WattTimeTestData.Constants.SignalType, regionResponse?.SignalType); } [Test] @@ -361,7 +360,7 @@ public async Task GetHistoricalDataAsync_StreamsExpectedContent() this.SetupBasicHandlers(new StreamContent(testStream)); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); var result = await client.GetHistoricalDataAsync(WattTimeTestData.Constants.Region); var sr = new StreamReader(result); @@ -387,9 +386,7 @@ public async Task GetHistoricalDataAsync_RefreshesTokenWhenExpired() this.SetupBasicHandlers(new StreamContent(testStream), "REFRESHTOKEN"); - var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - var result = await client.GetHistoricalDataAsync(WattTimeTestData.Constants.Region); var sr = new StreamReader(result); string streamResult = sr.ReadToEnd(); @@ -406,7 +403,7 @@ public async Task GetHistoricalDataAsync_RefreshesTokenWhenNoneSet() this.SetupBasicHandlers(new StreamContent(testStream), "REFRESHTOKEN"); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); - client.SetBearerAuthenticationHeader(this.DefaultTokenValue); + client.SetBearerAuthenticationHeader(_DEFAULT_TOKEN_VALUE); var result = await client.GetHistoricalDataAsync(WattTimeTestData.Constants.Region); var sr = new StreamReader(result); @@ -421,7 +418,7 @@ public async Task GetHistoricalDataAsync_RefreshesTokenWhenNoneSet() */ private void AddHandlers_Auth(string? validToken = null) { - validToken ??= this.DefaultTokenValue; + validToken ??= _DEFAULT_TOKEN_VALUE; AddHandler_RequestResponse(r => { @@ -430,27 +427,28 @@ private void AddHandlers_Auth(string? validToken = null) AddHandler_RequestResponse(r => { - return (r.RequestUri == new Uri("https://api.watttime.org/login") && ($"Basic {this.BasicAuthValue}".Equals(r.Headers.Authorization?.ToString()))); + return (r.RequestUri == new Uri(_BASE_WATTTIME_LOGIN_URL) && ($"Basic {this.BasicAuthValue}".Equals(r.Headers.Authorization?.ToString()))); }, System.Net.HttpStatusCode.OK, "{\"token\":\"" + validToken + "\"}"); AddHandler_RequestResponse(r => { - return !(r.RequestUri == new Uri("https://api.watttime.org/login") && ($"Basic {this.BasicAuthValue}".Equals(r.Headers.Authorization?.ToString()))) && r.Headers.Authorization?.ToString() != $"Bearer {validToken}"; + return !(r.RequestUri == new Uri(_BASE_WATTTIME_LOGIN_URL) && ($"Basic {this.BasicAuthValue}".Equals(r.Headers.Authorization?.ToString()))) && r.Headers.Authorization?.ToString() != $"Bearer {validToken}"; }, System.Net.HttpStatusCode.Forbidden); } + /** * Helper to add client handlers for auth and basic content return */ private void SetupBasicHandlers(StreamContent responseContent, string? validToken = null) { - validToken ??= this.DefaultTokenValue; + validToken ??= _DEFAULT_TOKEN_VALUE; AddHandlers_Auth(validToken); // Catch-all for "requesting url that is not login and has valid token" this.Handler - .SetupRequest(r => r.RequestUri != new Uri("https://api.watttime.org/login") && r.Headers.Authorization?.ToString() == $"Bearer {validToken}") + .SetupRequest(r => r.RequestUri != new Uri(_BASE_WATTTIME_LOGIN_URL) && r.Headers.Authorization?.ToString() == $"Bearer {validToken}") .ReturnsResponse(System.Net.HttpStatusCode.OK, responseContent); } @@ -459,12 +457,12 @@ private void SetupBasicHandlers(StreamContent responseContent, string? validToke */ private void SetupBasicHandlers(string responseContent, string? validToken = null) { - validToken ??= this.DefaultTokenValue; + validToken ??= _DEFAULT_TOKEN_VALUE; AddHandlers_Auth(validToken); // Catch-all for "requesting url that is not login and has valid token" - AddHandler_RequestResponse(r => r.RequestUri != new Uri("https://api.watttime.org/login") && r.Headers.Authorization?.ToString() == $"Bearer {validToken}", System.Net.HttpStatusCode.OK, responseContent); + AddHandler_RequestResponse(r => r.RequestUri != new Uri(_BASE_WATTTIME_LOGIN_URL) && r.Headers.Authorization?.ToString() == $"Bearer {validToken}", System.Net.HttpStatusCode.OK, responseContent); } /** From daa4503679c42aa0873d70a8b5283ca332f064c0 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Thu, 20 Jun 2024 10:51:57 +1000 Subject: [PATCH 30/31] More cleanup of some of the strings More cleanup of some of the strings. Creating consistency for using the test data on parameters and not just reponses also. --- .../test/Client/WattTimeClientTests.cs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs index 2b15c016d..03557348d 100644 --- a/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs +++ b/src/CarbonAware.DataSources/CarbonAware.DataSources.WattTime/test/Client/WattTimeClientTests.cs @@ -122,7 +122,7 @@ public async Task GetDataAsync_DeserializesExpectedResponse() this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => { - return r.RequestUri!.ToString().Equals("https://api.watttime.org/v3/historical?region=region&start=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&end=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type=co2_moer") && r.Method == HttpMethod.Get; + return r.RequestUri!.ToString().Equals($"https://api.watttime.org/v3/historical?region={WattTimeTestData.Constants.Region}&start=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&end=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type=co2_moer") && r.Method == HttpMethod.Get; }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetGridDataResponseJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -175,7 +175,7 @@ public async Task GetCurrentForecastAsync_DeserializesExpectedResponse() this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => { - return r.RequestUri!.ToString().Equals(@"https://api.watttime.org/v3/forecast?region=region&signal_type=co2_moer") && r.Method == HttpMethod.Get; + return r.RequestUri!.ToString().Equals($"https://api.watttime.org/v3/forecast?region={WattTimeTestData.Constants.Region}&signal_type=co2_moer") && r.Method == HttpMethod.Get; }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetCurrentForecastJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); @@ -240,7 +240,7 @@ public async Task GetForecastOnDateAsync_DeserializesExpectedResponse() this.AddHandlers_Auth(); this.AddHandler_RequestResponse(r => { - return r.RequestUri!.ToString().Equals("https://api.watttime.org/v3/forecast/historical?region=region&start=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&end=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type=co2_moer") && r.Method == HttpMethod.Get; + return r.RequestUri!.ToString().Equals($"https://api.watttime.org/v3/forecast/historical?region={WattTimeTestData.Constants.Region}&start=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&end=2022-04-22T00%3a00%3a00.0000000%2b00%3a00&signal_type=co2_moer") && r.Method == HttpMethod.Get; }, System.Net.HttpStatusCode.OK, WattTimeTestData.GetHistoricalForecastDataJsonString()); var client = new WattTimeClient(this.HttpClientFactory, this.Options.Object, this.Log.Object, this.MemoryCache); From 903a58f7d698d833178b71f484abf61a721b4117 Mon Sep 17 00:00:00 2001 From: Vaughan Knight Date: Tue, 25 Jun 2024 12:11:30 +1000 Subject: [PATCH 31/31] Updates to documentation and changelog Updates to documentation and changelog --- CHANGELOG.md | 63 +++++++++++++++++++ .../decisions/0016-watt-time-v3.md | 36 +++++++++-- .../tutorial-extras/carbon-aware-library.md | 2 +- 3 files changed, 94 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d41501578..fbc2e082f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,69 @@ All notable changes to the Carbon Aware SDK will be documented in this file. +## [1.5.0] - 2024-05 + +This is the WattTime v3 update. Most notable changes that may require action are for deployment configuration, and these are minor. + +### Added + +WattTime v3 API support. This is an inplace upgrade for v2. + +### Removed + +WattTime v2 API support due to v3 in place replacement. + +### Fixed + +-[#535] [Bug]: Configuration for locations loads twice](https://github.com/Green-Software-Foundation/carbon-aware-sdk/issues/535) + +### Changed + +Updates for WattTime v3 API endpoint from v2, details in the [ADR for WattTime v3 changes](./casdk-docs/docs/architecture//decisions/0016-watt-time-v3.md). + +#### API + +No changes + +#### API Deployment + +Due to the change for WattTime v3, there is change to the configuration for WattTime users. + +With some of the changes to the code, some of the configuration will also needs to change. + +| Config (v2) | Config (v3) | Description | +|------------------------------------|----------------------------------|------------------------------| +| `BalancingAuthorityCacheTTL` | `RegionCacheTTL` | This is the cache for regions data in seconds, and has a default value of 1 day. This only needs updating if you set it | +| n/a | `AuthenticationBaseUrl` | **NEW** This is the base URL for the WattTime Authentication API and defaults to `https://api.watttime.org/` if not set. | + +Example below if set (note they do not have to be set) +```json +"wattTime_no-proxy": { + "Type": "WattTime", + "Username": "the_username", + "Password": "super_secret_secret", + "BaseURL": "https://api.watttime.org/v3/", + "AutenticationBaseURL": "https://api.watttime.org", // This is new but not mandatory in config + "RegionCacheTTL": 86400, // This is changed but not mandatory in config + "Proxy": { + "UseProxy": false + } +``` + +#### SDK + +No changes + + +#### Other + +No changes + + +For more details, checkout [https://github.com/Green-Software-Foundation/carbon-aware-sdk/issues/503](https://github.com/Green-Software-Foundation/carbon-aware-sdk/issues/503) + + + ## [1.4.0] - 2024-05 diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md index 3ec13fbea..a1fef0ab7 100644 --- a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md +++ b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md @@ -8,7 +8,7 @@ Proposed ## Context As part of the update to Watt Time v3 we are proposing the changes to the underlying API calls. This needs to be tracked so we understand the impacts, and if multiple options are available, which option was selected and why. -This wll impact the `CarbonAware.DataSources.WattTime` project primarily. +This wll impact the `CarbonAware.DataSources.WattTime` project primarily - however all test will need changing where there are downstream tests that are impacted, and for example, if any initialization needs reconfiguring it will impact dependency injection and likely `Program.cs` in the WattTime API projects. ## Decision @@ -19,8 +19,7 @@ The proposal is for the outlined WattTime API mapping and changes. The following document and guidelines was used to understand the impact to the Carbon Aware SDK for the WattTime v3 updates. https://docs.watttime.org/#tag/Transitioning-from-APIv2-to-APIv3 ### Base URL -The base URL will need to change. -> TODO: Add where this is configured +The base URL will need to change. This is configured in the `appsettings.config` and can be set from environment variables. |Base URL (v2) | Based URL (v3) | |---|---| @@ -32,13 +31,17 @@ The paths will also need to change. The following is configured at `CarbonAware.DataSources.WattTime/src/Constants/Paths.cs` +All response types for emission data include a response/data object, and a `meta` object which contains information such as `region`. As such historical data, forecast data, and historical forecast data objects will change significantly. These will be moved across to their own `...Response` record objects to abstract any future changes in the response type. Previously these objects had a lot of overlap so where the same class and this will cause significant rework of the code and tests, and breaking these out now will abstract them. + +The `Login` is now at a different base URL, and to avoid any future issues a different HTTP client will be used for authentication, and the existing HTTP client will be used for API interaction. These will still sit in the `WattTimeClient` and as such no impacts to the dependent classes/logic. + | API Endpoint | Description | Path (v2) | Path (v3) | Notes | |--------------|-------------|-----------|-----------|---| | Data | Get data | /data | /historical | _Request_
  • `starttime` is now `start` and mandatory
  • `endtime` is now `end` and mandatory
  • `ba` is now `region`
  • `signal_type` added
    _Response_
  • `signal_type` added -| Forecast | Get forecast| /forecast | /forecast | **TODO: CHECK IMPACT**
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added -| Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | **We need to validate why historical was being used for the API, and what historical used to be, and whether this should be the new /forecast/historical or not.** +| Forecast | Get forecast| /forecast | /forecast |
    No longer be used for historical data
    _Request_
  • `ba` is now `region`
  • `extended_forecast` removed
  • `horizon_hours` added
  • `signal_type` added
  • Historical forecasts are now at `/forecast/historical`
    _Response_
  • `signal_type` added +| Historical | Get historical forecast data | /historical (?) | /forecast/historical (?) | This changed signficantly.
    _Request_
  • `ba` is now `region`
  • `starttime` is now `start` and mandatory
  • `endtime` is now `end` and mandatory
  • `signal_type` added
    _Response_
  • `signal_type` added | Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc | Check if the CA SDK uses BA at all

    _Request_
  • `name` is now `region_full_name`
  • `abbrev` is now `region`
  • `signal_type` added
    _Response_
  • `id` removed
  • `signal_type` added | -| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being no longer related to the API version.

    NOTE: Updated in wattTime client to now have 2 HTTP clients to decouple versions from the login. +| Login | User login | https://api2.watttime.org/v2/login | https://api.watttime.org/login | Path has changed from being version specific to being no longer related to the API version.

    Updated in `WattTimeClient` to now have 2 HTTP clients to decouple versions from the login. ### Query Strings @@ -56,6 +59,27 @@ The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QuerySt | `longitude` | - | Longitude | | `username` | - | Username | +## Update Changes +With some of the changes to the code, some of the configuration will also needs to change. + +| Config (v2) | Config (v3) | Description | +|------------------------------------|----------------------------------|------------------------------| +| `BalancingAuthorityCacheTTL` | `RegionCacheTTL` | This is the cache for regions data in seconds, and has a default value of 1 day. | +| n/a | `AuthenticationBaseUrl` | **NEW** This is the base URL for the WattTime Authentication API and defaults to `https://api.watttime.org/` if not set. | + +Example below if set (note they do not have to be set) +```json +"wattTime_no-proxy": { + "Type": "WattTime", + "Username": "the_username", + "Password": "super_secret_secret", + "BaseURL": "https://api.watttime.org/v3/", + "AutenticationBaseURL": "https://api.watttime.org", // This is new but not mandatory + "RegionCacheTTL": 86400, // This is new but not mandatory + "Proxy": { + "UseProxy": false + } +``` ## Green Impact Neutral diff --git a/casdk-docs/docs/tutorial-extras/carbon-aware-library.md b/casdk-docs/docs/tutorial-extras/carbon-aware-library.md index b47813f3b..2a8be2f3b 100644 --- a/casdk-docs/docs/tutorial-extras/carbon-aware-library.md +++ b/casdk-docs/docs/tutorial-extras/carbon-aware-library.md @@ -423,7 +423,7 @@ EmissionsForecast() #### Locations Each WattTime emissions data point is associated with a particular named -balancing authority. For transparency, this value is also used in +region often referred to as a balancing authority. For transparency, this value is also used in `EmissionsData` response objects. It is not overwritten to match the named datacenter provided by any request.