Merge pull request #532 from Green-Software-Foundation/wattime-v3-sup…

…port Watt Time v3 Support
Green-Software-Foundation · Jun 28, 2024 · 55b9b10 · 55b9b10
2 parents 4f5dbd2 + 903a58f
commit 55b9b10
Show file tree

Hide file tree

Showing 43 changed files with 1,017 additions and 554 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,69 @@
 
 All notable changes to the Carbon Aware SDK will be documented in this file.
 
+## [1.5.0] - 2024-05
+
+This is the WattTime v3 update.  Most notable changes that may require action are for deployment configuration, and these are minor.
+
+### Added 
+
+WattTime v3 API support.  This is an inplace upgrade for v2.
+
+### Removed
+
+WattTime v2 API support due to v3 in place replacement.
+
+### Fixed
+
+-[#535] [Bug]: Configuration for locations loads twice](https://github.com/Green-Software-Foundation/carbon-aware-sdk/issues/535)
+
+### Changed
+
+Updates for WattTime v3 API endpoint from v2, details in the [ADR for WattTime v3 changes](./casdk-docs/docs/architecture//decisions/0016-watt-time-v3.md).
+
+#### API
+
+No changes
+
+#### API Deployment
+
+Due to the change for WattTime v3, there is change to the configuration for WattTime users.
+
+With some of the changes to the code, some of the configuration will also needs to change.
+
+| Config (v2)                   | Config (v3)                 | Description                  |
+|------------------------------------|----------------------------------|------------------------------|
+| `BalancingAuthorityCacheTTL`                                 | `RegionCacheTTL`                                 | This is the cache for regions data in seconds, and has a default value of 1 day.  This only needs updating if you set it |
+| n/a                        | `AuthenticationBaseUrl`                                | **NEW** This is the base URL for the WattTime Authentication API and defaults to `https://api.watttime.org/` if not set. |
+
+Example below if set (note they do not have to be set)
+```json
+"wattTime_no-proxy": {
+  "Type": "WattTime",
+  "Username": "the_username",
+  "Password": "super_secret_secret",
+  "BaseURL": "https://api.watttime.org/v3/",
+  "AutenticationBaseURL": "https://api.watttime.org", // This is new but not mandatory in config
+  "RegionCacheTTL": 86400, // This is changed but not mandatory in config
+  "Proxy": {
+    "UseProxy": false
+  }
+```
+
+#### SDK 
+
+No changes
+
+
+#### Other
+
+No changes
+
+
+For more details, checkout [https://github.com/Green-Software-Foundation/carbon-aware-sdk/issues/503](https://github.com/Green-Software-Foundation/carbon-aware-sdk/issues/503) 
+
+
+
 
 ## [1.4.0] - 2024-05
 

diff --git a/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md b/casdk-docs/docs/architecture/decisions/0016-watt-time-v3.md
@@ -0,0 +1,86 @@
+
+# 0015. WattTime v3 Changes
+
+## Status
+
+Proposed
+
+## Context
+As part of the update to Watt Time v3 we are proposing the changes to the underlying API calls.  This needs to be tracked so we understand the impacts, and if multiple options are available, which option was selected and why.
+
+This wll impact the `CarbonAware.DataSources.WattTime` project primarily - however all test will need changing where there are downstream tests that are impacted, and for example, if any initialization needs reconfiguring it will impact dependency injection and likely `Program.cs` in the WattTime API projects.
+
+## Decision
+
+The proposal is for the outlined WattTime API mapping and changes.
+
+## WattTime v2, v3 Mapping
+
+The following document and guidelines was used to understand the impact to the Carbon Aware SDK for the WattTime v3 updates.  https://docs.watttime.org/#tag/Transitioning-from-APIv2-to-APIv3 
+
+### Base URL
+The base URL will need to change.  This is configured in the `appsettings.config` and can be set from environment variables.
+
+|Base URL (v2) | Based URL (v3) | 
+|---|---|
+| /v2 | /v3 |
+
+
+### Paths
+The paths will also need to change. 
+
+The following is configured at  `CarbonAware.DataSources.WattTime/src/Constants/Paths.cs`
+
+All response types for emission data include a response/data object, and a `meta` object which contains information such as `region`.  As such historical data, forecast data, and historical forecast data objects will change significantly.  These will be moved across to their own `...Response` record objects to abstract any future changes in the response type.  Previously these objects had a lot of overlap so where the same class and this will cause significant rework of the code and tests, and breaking these out now will abstract them. 
+
+The `Login` is now at a different base URL, and to avoid any future issues a different HTTP client will be used for authentication, and the existing HTTP client will be used for API interaction.  These will still sit in the `WattTimeClient` and as such no impacts to the dependent classes/logic.
+
+| API Endpoint | Description | Path (v2) | Path (v3) | Notes |
+|--------------|-------------|-----------|-----------|---|
+| Data         | Get data    | /data     | /historical          |  _Request_ <li> `starttime` is now `start` and mandatory <li> `endtime` is now `end` and mandatory <li> `ba` is now `region` <li> `signal_type` added <br> _Response_ <li> `signal_type` added
+| Forecast     | Get forecast| /forecast | /forecast   | <br> No longer be used for historical data <br> _Request_ <li> `ba` is now `region` <li> `extended_forecast` removed <li> `horizon_hours` added  <li> `signal_type` added <li> Historical forecasts are now at `/forecast/historical` <br> _Response_ <li> `signal_type` added 
+| Historical   | Get historical forecast data | /historical (?) | /forecast/historical (?)           | This changed signficantly.  <br> _Request_ <li> `ba` is now `region` <li> `starttime` is now `start` and mandatory <li> `endtime` is now `end` and mandatory <li> `signal_type` added <br> _Response_ <li> `signal_type` added 
+| Balancing Authority From Location | Get balancing authority from location | /ba-from-loc | /region-from-loc          | Check if the CA SDK uses BA at all <br><br> _Request_ <li> `name` is now `region_full_name` <li> `abbrev` is now `region` <li> `signal_type` added <br> _Response_ <li> `id` removed <li> `signal_type` added  | 
+| Login        | User login  | https://api2.watttime.org/v2/login    | https://api.watttime.org/login | Path has changed from being version specific to being no longer related to the API version.  <br><br> Updated in `WattTimeClient` to now have 2 HTTP clients to decouple versions from the login. 
+
+### Query Strings
+
+#### Signal Type
+Everything call takes an optional `signal_type` parameter that defaults to `co2_moer`.  
+
+The following comes from `CarbonAware.DataSources.WattTime/src/Constants/QueryStrings.cs` and the changes are consistent with the discussion above.
+
+| Query String (v2)                   | Query String if Changed (v3)                 | Description                  |
+|------------------------------------|----------------------------------|------------------------------|
+| `ba`                                 | `region`                                 | Balancing Authority / Region |
+| `starttime`                          | `start`                                | Start Time                       |
+| `endtime`                            | `end`                                 | End Time                         |
+| `latitude`                           | -                                 | Latitude                         |
+| `longitude`                          | -                                 | Longitude                        |
+| `username`                           | -                                 | Username                         |
+
+## Update Changes
+With some of the changes to the code, some of the configuration will also needs to change.
+
+| Config (v2)                   | Config (v3)                 | Description                  |
+|------------------------------------|----------------------------------|------------------------------|
+| `BalancingAuthorityCacheTTL`                                 | `RegionCacheTTL`                                 | This is the cache for regions data in seconds, and has a default value of 1 day. |
+| n/a                        | `AuthenticationBaseUrl`                                | **NEW** This is the base URL for the WattTime Authentication API and defaults to `https://api.watttime.org/` if not set. |
+
+Example below if set (note they do not have to be set)
+```json
+"wattTime_no-proxy": {
+  "Type": "WattTime",
+  "Username": "the_username",
+  "Password": "super_secret_secret",
+  "BaseURL": "https://api.watttime.org/v3/",
+  "AutenticationBaseURL": "https://api.watttime.org", // This is new but not mandatory
+  "RegionCacheTTL": 86400, // This is new but not mandatory
+  "Proxy": {
+    "UseProxy": false
+  }
+```
+## Green Impact  
+
+Neutral
+
diff --git a/casdk-docs/docs/tutorial-extras/carbon-aware-library.md b/casdk-docs/docs/tutorial-extras/carbon-aware-library.md
@@ -423,7 +423,7 @@ EmissionsForecast()
 #### Locations
 
 Each WattTime emissions data point is associated with a particular named
-balancing authority. For transparency, this value is also used in
+region often referred to as a balancing authority. For transparency, this value is also used in
 `EmissionsData` response objects. It is not overwritten to match the named
 datacenter provided by any request.
 

diff --git a/...e.CLI/test/integrationTests/Commands/EmissionsForecasts/EmissionsForecastsCommandTests.cs b/...e.CLI/test/integrationTests/Commands/EmissionsForecasts/EmissionsForecastsCommandTests.cs
@@ -99,7 +99,7 @@ public async Task EmissionsForecasts_RequestedAtOptions_ReturnsExpectedData()
         IgnoreTestForDataSource("data source does not implement '--requested-at'", DataSourceType.ElectricityMaps);
 
         // Arrange
-        _dataSourceMocker.SetupBatchForecastMock();
+        _dataSourceMocker.SetupHistoricalBatchForecastMock();
 
         // Act
         var exitCode = await InvokeCliAsync($"emissions-forecasts -l eastus -r 2022-09-01");

diff --git a/...ataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs b/...ataSources/CarbonAware.DataSources.ElectricityMaps/mock/ElectricityMapDataSourceMocker.cs
@@ -126,7 +126,7 @@ public void SetupDataMock(DateTimeOffset start, DateTimeOffset end, string locat
         SetupResponseGivenGetRequest(Paths.PastRange, pastRange);
     }
 
-    public void SetupBatchForecastMock()
+    public void SetupHistoricalBatchForecastMock()
     {
         throw new NotImplementedException();
     }

diff --git a/...ware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs b/...ware.DataSources/CarbonAware.DataSources.ElectricityMaps/src/ElectricityMapsDataSource.cs
@@ -72,7 +72,7 @@ private static EmissionsForecast ToEmissionsForecast(Location location, Forecast
     }
 
     /// <inheritdoc />
-    public async Task<EmissionsForecast> GetCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt)
+    public async Task<EmissionsForecast> GetHistoricalCarbonIntensityForecastAsync(Location location, DateTimeOffset requestedAt)
     {
         await Task.Run(() => true);
         throw new NotImplementedException();

diff --git a/...s/CarbonAware.DataSources.ElectricityMapsFree/mock/ElectricityMapsFreeDataSourceMocker.cs b/...s/CarbonAware.DataSources.ElectricityMapsFree/mock/ElectricityMapsFreeDataSourceMocker.cs
@@ -49,7 +49,7 @@ public void SetupForecastMock()
         throw new NotImplementedException();
     }
 
-    public void SetupBatchForecastMock()
+    public void SetupHistoricalBatchForecastMock()
     {
         throw new NotImplementedException();
     }

diff --git a/src/CarbonAware.DataSources/CarbonAware.DataSources.Json/mock/JsonDataSourceMocker.cs b/src/CarbonAware.DataSources/CarbonAware.DataSources.Json/mock/JsonDataSourceMocker.cs
@@ -42,7 +42,7 @@ public void Initialize() { }
     public void Reset() { }
     public void Dispose() { }
 
-    public void SetupBatchForecastMock()
+    public void SetupHistoricalBatchForecastMock()
     {
         throw new NotImplementedException();
     }

diff --git a/...urces/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj b/...urces/CarbonAware.DataSources.WattTime/mock/CarbonAware.DataSources.WattTime.Mocks.csproj
@@ -13,6 +13,7 @@
 
   <ItemGroup>
     <ProjectReference Include="..\src\CarbonAware.DataSources.WattTime.csproj" />
+    <ProjectReference Include="..\test\CarbonAware.DataSources.WattTime.Tests.csproj" />
   </ItemGroup>
 
   <ItemGroup>