(v3.3.3) - Sinergym extra variables logging (#412)

* Created ReduceObservationWrapper in wrappers.py * Added this new wrapper fixture in conftest.py * Added tests for the new wrapper in test_wrappers.py * Updated Sinergym version from 3.3.2 to 3.3.3 * Added wrapper documentation section to explain ReduceObservationWrapper * Added documentation modules for API Reference * Added notebook example of using this new wrapper * Fix environments documentation section title
ugr-sail · May 6, 2024 · 0191376 · 0191376
1 parent 5318e48
commit 0191376
Show file tree

Hide file tree

Showing 12 changed files with 457 additions and 7 deletions.
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -94,6 +94,7 @@ If you use *Sinergym* in your work, please cite our `paper <https://dl.acm.org/d
    pages/notebooks/default_building_control.nblink
    pages/notebooks/wrappers_examples.nblink
    pages/notebooks/personalize_loggerwrapper.nblink
+   pages/notebooks/logging_unused_variables.nblink
    pages/notebooks/rule_controller_example.nblink
    pages/notebooks/drl.nblink
 

diff --git a/docs/source/pages/environments.rst b/docs/source/pages/environments.rst
@@ -250,8 +250,9 @@ and ``runperiod``, can be set as extra configurations. These configurations, whi
 are specified in the ``config_params`` argument, a Python Dictionary. For additional information 
 on extra configurations in *Sinergym*, refer to :ref:`Extra Configuration in Sinergym simulations`.
 
+*************************************
 Adding New Weathers for Environments
-====================================
+*************************************
 
 *Sinergym* provides a variety of weather files for diverse global climates to enhance experimental diversity.
 
@@ -266,8 +267,9 @@ To incorporate a **new weather**:
 Upon addition, *Sinergym* will automatically modify the ``SizingPeriod:DesignDays`` and ``Site:Location`` 
 fields in the building model file using the *DDY* file.
 
+***************************************
 Adding New Buildings for Environments
-=====================================
+***************************************
 
 Users can either modify existing environments or create new ones, incorporating new climates, 
 action, and observation spaces. They also have the option to use a different **building model** 

diff --git a/docs/source/pages/modules/sinergym.utils.wrappers.NormalizeObservation.rst b/docs/source/pages/modules/sinergym.utils.wrappers.NormalizeObservation.rst
@@ -1,4 +1,4 @@
-sinergym.utils.wrappers.NormalizeObservation
+sinergym.utils.wrappers.NormalizeObservation
 ============================================
 
 .. currentmodule:: sinergym.utils.wrappers

diff --git a/docs/source/pages/modules/sinergym.utils.wrappers.ReduceObservationWrapper.rst b/docs/source/pages/modules/sinergym.utils.wrappers.ReduceObservationWrapper.rst
@@ -0,0 +1,45 @@
+sinergym.utils.wrappers.ReduceObservationWrapper
+================================================
+
+.. currentmodule:: sinergym.utils.wrappers
+
+.. autoclass:: ReduceObservationWrapper
+   :members:                                                           
+   :undoc-members:               
+
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~ReduceObservationWrapper.__init__
+      ~ReduceObservationWrapper.class_name
+      ~ReduceObservationWrapper.close
+      ~ReduceObservationWrapper.get_wrapper_attr
+      ~ReduceObservationWrapper.render
+      ~ReduceObservationWrapper.reset
+      ~ReduceObservationWrapper.step
+      ~ReduceObservationWrapper.wrapper_spec
+
+
+
+
+
+   .. rubric:: Attributes
+
+   .. autosummary::
+
+      ~ReduceObservationWrapper.action_space
+      ~ReduceObservationWrapper.logger
+      ~ReduceObservationWrapper.metadata
+      ~ReduceObservationWrapper.np_random
+      ~ReduceObservationWrapper.observation_space
+      ~ReduceObservationWrapper.render_mode
+      ~ReduceObservationWrapper.reward_range
+      ~ReduceObservationWrapper.spec
+      ~ReduceObservationWrapper.unwrapped
+
+
diff --git a/docs/source/pages/modules/sinergym.utils.wrappers.rst b/docs/source/pages/modules/sinergym.utils.wrappers.rst
@@ -30,6 +30,7 @@
       NormalizeObservation
       OfficeGridStorageSmoothingActionConstraintsWrapper
       PreviousObservationWrapper
+      ReduceObservationWrapper
 
 
 

diff --git a/docs/source/pages/notebooks/logging_unused_variables.nblink b/docs/source/pages/notebooks/logging_unused_variables.nblink
@@ -0,0 +1,3 @@
+{
+    "path": "../../../../examples/logging_unused_variables.ipynb"
+}
diff --git a/docs/source/pages/wrappers.rst b/docs/source/pages/wrappers.rst
@@ -149,6 +149,21 @@ This is a wrapper for logging all interactions between the agent and the environ
 be selected in the constructor if a different type of logging is required. For more information about the 
 *Sinergym* Logger, refer to :ref:`Logger`.
 
+**************************
+ReduceObservationWrapper
+**************************
+
+This wrapper starts from the original observation space and reduces it by subtracting the variables 
+specified in a string list parameter. These removed variables are returned in the info dictionary 
+(under the key ``removed_variables``) and are not used in the agent optimization process.
+
+If combined with the :ref:`LoggerWrapper` in subsequent layers, the removed variables will be saved 
+in the output files, even if they are not "used". This makes it perfect for monitoring simulation 
+values that are not part of the problem to be solved.
+
+Similarly, any other wrapper applied in layers prior to this one will affect the removed variables, 
+which can be observed in the info dictionary.
+
 ***********************
 MultiObsWrapper
 ***********************

diff --git a/examples/logging_unused_variables.ipynb b/examples/logging_unused_variables.ipynb
@@ -0,0 +1,235 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Logging unusued variables"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Until version v3.3.2 of *Sinergym*, all variables monitored using the *LoggerWrapper* had to be present in the observation space. This presents a drawback when we want to monitor certain aspects of the simulation that are not used in the optimization process (in other words, that are not present in the environment's observation space), it would be impossible.\n",
+    "\n",
+    "Including extra variables that are not directly part of the observation space requires certain internal changes that break the minimalist structure of the classes that make up the tool.\n",
+    "\n",
+    "This notebook explains the correct way to do it, which is available from *Sinergym* v3.3.3. It involves the use of [ReduceObservationWrapper](https://ugr-sail.github.io/sinergym/compilation/main/pages/wrappers.html#reduceobservationwrapper) in combination with [LoggerWrapper](https://ugr-sail.github.io/sinergym/compilation/main/pages/wrappers.html#loggerwrapper).\n",
+    "\n",
+    "The idea is to define all the variables we want, whether they are part of the final observation space or not, and monitor everything with the LoggerWrapper, to later add a layer that removes the desired variables from the observation space (when they would already be monitored, which is our goal)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "#==============================================================================================#\n",
+      "\u001b[38;20m[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-continuous-stochastic-v1]\u001b[0m\n",
+      "#==============================================================================================#\n",
+      "\u001b[38;20m[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res2]\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : Model Config is correct.\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : Updated building model with whole Output:Variable available names\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : Updated building model with whole Output:Meter available names\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 0, 'n_steps_per_hour': 4}\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : Episode length (seconds): 31536000.0\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : timestep size (seconds): 900.0\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : timesteps per episode: 35040\u001b[0m\n",
+      "\u001b[38;20m[REWARD] (INFO) : Reward function initialized.\u001b[0m\n",
+      "\u001b[38;20m[ENVIRONMENT] (INFO) : Environment 5zone-hot-continuous-stochastic-v1 created successfully.\u001b[0m\n",
+      "\u001b[38;20m[WRAPPER NormalizeAction] (INFO) : New normalized action Space: Box(-1.0, 1.0, (2,), float32)\u001b[0m\n",
+      "\u001b[38;20m[WRAPPER NormalizeAction] (INFO) : Wrapper initialized\u001b[0m\n",
+      "\u001b[38;20m[WRAPPER NormalizeObservation] (INFO) : Wrapper initialized.\u001b[0m\n",
+      "\u001b[38;20m[WRAPPER LoggerWrapper] (INFO) : Wrapper initialized.\u001b[0m\n",
+      "###########################################################################\n",
+      "Old observation space shape:  17\n",
+      "Old observation variables:  ['month', 'day_of_month', 'hour', 'outdoor_temperature', 'outdoor_humidity', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'htg_setpoint', 'clg_setpoint', 'air_temperature', 'air_humidity', 'people_occupant', 'co2_emission', 'HVAC_electricity_demand_rate', 'total_electricity_HVAC']\n",
+      "###########################################################################\n",
+      "\u001b[38;20m[WRAPPER ReduceObservationWrapper] (INFO) : Wrapper initialized.\u001b[0m\n",
+      "###########################################################################\n",
+      "Wrapped observation space shape:  14\n",
+      "Wrapped observation variables:  ['month', 'day_of_month', 'hour', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'htg_setpoint', 'clg_setpoint', 'air_humidity', 'people_occupant', 'co2_emission', 'HVAC_electricity_demand_rate', 'total_electricity_HVAC']\n",
+      "###########################################################################\n"
+     ]
+    }
+   ],
+   "source": [
+    "import gymnasium as gym\n",
+    "import numpy as np\n",
+    "\n",
+    "import sinergym\n",
+    "from sinergym.utils.wrappers import (\n",
+    "    LoggerWrapper,\n",
+    "    NormalizeAction,\n",
+    "    NormalizeObservation,\n",
+    "    ReduceObservationWrapper)\n",
+    "\n",
+    "# Creating environment and applying wrappers for normalization and logging\n",
+    "env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')\n",
+    "env = NormalizeAction(env)\n",
+    "env = NormalizeObservation(env)\n",
+    "env = LoggerWrapper(env)\n",
+    "print('###########################################################################')\n",
+    "print('Old observation space shape: ',env.observation_space.shape[0])\n",
+    "print('Old observation variables: ',env.get_wrapper_attr('observation_variables'))\n",
+    "print('###########################################################################')\n",
+    "env = ReduceObservationWrapper(env, obs_reduction=['outdoor_temperature','outdoor_humidity','air_temperature'])\n",
+    "print('###########################################################################')\n",
+    "print('Wrapped observation space shape: ',env.observation_space.shape[0])\n",
+    "print('Wrapped observation variables: ',env.get_wrapper_attr('observation_variables'))\n",
+    "print('###########################################################################')\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The order of the wrappers is important. By applying normalization first, for example, we ensure that this transformation is subsequently monitored. As we apply the Logger before reducing the observation space, we also record these variables before they are removed from the observations. If we left the logger for the end, these variables would not be recorded. This can be verified by reviewing the generated `monitor.csv` files.\n",
+    "\n",
+    "Let's review the info dictionary to see these normalized variables that are no longer in `obs`:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\u001b[38;20m[WRAPPER LoggerWrapper] (INFO) : End of episode detected, recording summary (progress.csv) if logger is active\u001b[0m\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice.\n",
+      "  return _methods._mean(a, axis=axis, dtype=dtype,\n",
+      "/usr/local/lib/python3.10/dist-packages/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide\n",
+      "  ret = ret.dtype.type(ret / rcount)\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "#----------------------------------------------------------------------------------------------#\n",
+      "\u001b[38;20m[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-continuous-stochastic-v1] [Episode 2]\u001b[0m\n",
+      "#----------------------------------------------------------------------------------------------#\n",
+      "\u001b[38;20m[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res2/Eplus-env-sub_run2]\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.\u001b[0m\n",
+      "\u001b[38;20m[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]\u001b[0m\n",
+      "\u001b[38;20m[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res2/Eplus-env-sub_run2/output]\u001b[0m\n",
+      "\u001b[38;20m[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res2/Eplus-env-sub_run2/USA_AZ_Davis-Monthan.AFB.722745_TMY3_Random_1.0_0.0_0.001.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res2/Eplus-env-sub_run2/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res2/Eplus-env-sub_run2/5ZoneAutoDXVAV.epJSON']\u001b[0m\n",
+      "\u001b[38;20m[ENVIRONMENT] (INFO) : Episode 2 started.\u001b[0m\n",
+      "\u001b[38;20m[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [5zone-hot-continuous-stochastic-v1]\u001b[0m\n",
+      "\u001b[38;20m[WRAPPER LoggerWrapper] (INFO) : Creating monitor.csv for current episode (episode 2) if logger is active\u001b[0m\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.\n",
+      "  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(\n",
+      "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py:311: UserWarning: \u001b[33mWARN: env.name to get variables from other wrappers is deprecated and will be removed in v1.0, to get this variable you can do `env.unwrapped.name` for environment variables or `env.get_wrapper_attr('name')` that will search the reminding wrappers.\u001b[0m\n",
+      "  logger.warn(\n",
+      "/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: \u001b[33mWARN: Casting input x to numpy array.\u001b[0m\n",
+      "  gym.logger.warn(\"Casting input x to numpy array.\")\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "###########################################################################\n",
+      "Reset observation length:  14\n",
+      "Removed variables info:  {'outdoor_temperature': 0.9948461020148158, 'outdoor_humidity': -0.9935016762419474, 'air_temperature': -0.6942300884545833}\n",
+      "###########################################################################\n",
+      "\u001b[38;20m[SIMULATOR] (INFO) : handlers are ready.\u001b[0m\n",
+      "\u001b[38;20m[SIMULATOR] (INFO) : System is ready.\u001b[0m\n",
+      "###########################################################################\n",
+      "Reset observation length:  14\n",
+      "Removed variables info:  {'outdoor_temperature': -0.09010209798427844, 'outdoor_humidity': 0.7044684403339807, 'air_temperature': 0.8821502704407667}\n",
+      "###########################################################################\n"
+     ]
+    }
+   ],
+   "source": [
+    "obs, info = env.reset()\n",
+    "print('###########################################################################')\n",
+    "print('Reset observation length: ',len(obs))\n",
+    "print('Removed variables info: ',info['removed_observation'])\n",
+    "print('###########################################################################')\n",
+    "\n",
+    "a = env.action_space.sample()\n",
+    "obs, _, _, _, info = env.step(a)\n",
+    "print('###########################################################################')\n",
+    "print('Reset observation length: ',len(obs))\n",
+    "print('Removed variables info: ',info['removed_observation'])\n",
+    "print('###########################################################################')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that even if we remove a variable that is used in the reward function, as this value is used in the core of the environment (before any wrapper), it works perfectly."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\u001b[38;20m[WRAPPER LoggerWrapper] (INFO) : End of episode, recording summary (progress.csv) if logger is active\u001b[0m\n",
+      "\u001b[38;20m[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-continuous-stochastic-v1]\u001b[0m\n",
+      "\u001b[38;20m[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [5zone-hot-continuous-stochastic-v1]\u001b[0m\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Progress: |***************************************************************************************************| 99%\n"
+     ]
+    }
+   ],
+   "source": [
+    "env.close()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}