v0.20.2 (Bugfix) HOG Support 🐷 + Docker Root support + QoL + Code Quality Latest

• NetworkIoCgroupContainerProvider had a segementation error when using multiple container IDs

v0.20 - HOG Support 🐷 + Docker Root support + QoL + Code Quality

HOG Suport!!1 ❤️ 🐷

• You can now track your development energy cost on Mac in our first developer preview
• Details: https://github.com/green-coding-berlin/hog

QoL:

• venv is now mandatory and keeps your Python system packages clean
• Docker root is now supported! We still recommend it for production, but for dev you now do not need to modify your docker client anymore 🥳
• Warnings have been added for many providers when they actually cannot run. No more (well ... less :) ) cryptic errors!
• Docker build logs are now shown in the Logs pane in the Dashboard

What's Changed

• Client has now two distinguished sleep times by @ArneTR in #468
• Adding email blocking for requests to non-configured domain by @ArneTR in #469
• Activate all 400 errors in API by @ArneTR in #474
• Docker error now appear in logs in more detail by @ArneTR in #475
• Bump psycopg[binary] from 3.1.10 to 3.1.11 by @dependabot in #478
• Bump pandas from 2.1.0 to 2.1.1 by @dependabot in #471
• Bump pylint from 2.17.5 to 2.17.6 by @dependabot in #481
• Bump python from 3.11.4-slim-bookworm to 3.11.5-slim-bookworm in /docker by @dependabot in #480
• Bump scipy from 1.11.2 to 1.11.3 by @dependabot in #486
• Bump pydantic from 2.3.0 to 2.4.2 by @dependabot in #485
• 446 ci page UI ux overhauls by @dan-mm in #467
• Bump psycopg[binary] from 3.1.11 to 3.1.12 by @dependabot in #484
• Bump fastapi from 0.103.1 to 0.103.2 by @dependabot in #489
• initial hog database backend by @ribalba in #470
• Removing CpuFrequencySysfsCoreProvider from default providers by @ArneTR in #487
• Import refactoring by @ArneTR in #476
• Docker root mode + venv QOL by @ArneTR in #488

Full Changelog: v0.19.1...v0.20

v0.19.1 Timeline View & Visual Improvements in Dashboard & macOS providers

Versin v0.19 adds the Timeline View!

Example for Overview: https://metrics.green-coding.berlin/energy-timeline.html
Example for concrete Timeline: https://metrics.green-coding.berlin/timeline.html?uri=https://github.com/green-coding-berlin/bakerydemo-gold-benchmark&filename=usage_scenario_warm.yml&branch=&machine_id=7

Timeline View is our first step to create a historic view of a software product over many commits. Here you can see how a software changes over time given a certain use case and potentially find when a energy hungry feature was introduced and optimize on it!

Main Features

• Timeline View added
• macOS improvements for powermetrics. The provider is now gracefully waited for and measurements do not fail with plain error.
• Eco-CI statistical data added to dashboard
• More infos about the running cluster machine added like installed packages, docker daemon stats, etc.
• Frontend has been greatly refactored to handle differences for machines better, display more warnings in case of inconsistency and rework how graphs are created

What's Changed

• Bump fastapi from 0.101.0 to 0.101.1 by @dependabot in #425
• Eco ci run stats by @dan-mm in #419
• Timeline comparison by @ArneTR in #420
• Bump orjson from 3.9.4 to 3.9.5 by @dependabot in #431
• Bump scipy from 1.11.1 to 1.11.2 by @dependabot in #433
• Bump pydantic from 2.1.1 to 2.3.0 by @dependabot in #436
• Bump fastapi from 0.101.1 to 0.103.0 by @dependabot in #437
• Network-proxy by @ribalba in #340
• Bump scipy from 1.11.1 to 1.11.2 by @dependabot in #441
• Bump orjson from 3.9.2 to 3.9.5 by @dependabot in #442
• Adds the stdbuf to disable buffering by @ribalba in #440
• Bump pandas from 2.0.3 to 2.1.0 by @dependabot in #443
• added workflow for building and pushing containers from docker/auxili… by @dan-mm in #394
• Added python and system packages listing by @ArneTR in #430
• Bump pytest from 7.4.0 to 7.4.1 by @dependabot in #449
• Bump fastapi from 0.103.0 to 0.103.1 by @dependabot in #448
• Schema checker improvements by @dan-mm in #395
• Frontend refactor by @ArneTR in #447
• Bump orjson from 3.9.5 to 3.9.6 by @dependabot in #456
• Bump orjson from 3.9.6 to 3.9.7 by @dependabot in #458
• Bump pytest from 7.4.1 to 7.4.2 by @dependabot in #455
• Runner now adds arguments to run by @ArneTR in #457
• added backwards compatability for cpu_util_avg stat, if its missing by @dan-mm in #428
• Bump docker/login-action from 2 to 3 by @dependabot in #459
• Bump docker/setup-qemu-action from 2 to 3 by @dependabot in #461
• Bump docker/setup-buildx-action from 2 to 3 by @dependabot in #460
• Powermetrics is now checked if it runs already on the system by @ArneTR in #452

Full Changelog: v0.18...v0.19

v0.18 SCI preview & PR energy for Eco-CI

In version 18 we are introducing the capability to output the Green Software Foundation's SCI with our tool

Please read up how to leverage that functionality in the docs

The other major upgrade is that Dashboard has been revamped and is now fully customizable. You can see any chart at any position you want by specifying which metrics shall be included.

Also if you are comparing different machines with different reporters you will get a warning if no matching reporter was found and no comparison can be made for a specific metric. Documentation for customization

Read the full changelog will all QoL, Bugfixes and Style improvements below.

What's Changed

• Dashboard customized charts by @ArneTR in #376
• Gmt commit hash by @ArneTR in #377
• Added min value by @ArneTR in #378
• Warnings in dashboard by @ArneTR in #379
• Warnings by @ArneTR in #380
• No notify admin for own project add by @ArneTR in #381
• eco-ci start measurement now also referencing v2 by @dan-mm in #389
• Bump pyyaml from 6.0 to 6.0.1 by @dependabot in #386
• Bump uvicorn[standard] from 0.22.0 to 0.23.0 by @dependabot in #383
• Bump orjson from 3.9.1 to 3.9.2 by @dependabot in #371
• Bump uvicorn[standard] from 0.23.0 to 0.23.1 by @dependabot in #391
• Postgres 15 by @ArneTR in #387
• Bump pydantic from 1.10.9 to 2.0 by @dependabot in #360
• Bump pydantic from 2.0 to 2.0.3 by @dependabot in #396
• Print name of service that had an issues with creating volumes by @djesic in #392
• Bump pydantic from 2.0.3 to 2.1.1 by @dependabot in #398
• Bump pylint from 2.17.4 to 2.17.5 by @dependabot in #400
• Safer docker prune for daily use by @ArneTR in #401
• Network interfaces hardware info by @ArneTR in #402
• Multi commit comparison by @ArneTR in #335
• Bump fastapi from 0.100.0 to 0.100.1 by @dependabot in #404
• Bump uvicorn[standard] from 0.23.1 to 0.23.2 by @dependabot in #405
• Gunicorn now logs access to stdout by @ArneTR in #406
• Index.js now can filter by repo and filename by @ArneTR in #408
• Fixes the metric output file becoming corrupted by @ribalba in #393
• Fixes the error on mac on which /tmp is a symlink to /private/tmp by @ribalba in #410
• Enables empty services in the usage_scenario by @ribalba in #409
• Fixes the github security errors by @ribalba in #411
• SCI metric by @ArneTR in #412
• eco-ci will now post the energy reading in the PR conversation by @dan-mm in #397
• Bump orjson from 3.9.2 to 3.9.3 by @dependabot in #415
• Bump fastapi from 0.100.1 to 0.101.0 by @dependabot in #414
• Bump psycopg[binary] from 3.1.9 to 3.1.10 by @dependabot in #413
• Bump orjson from 3.9.3 to 3.9.4 by @dependabot in #416
• Unit is now inside charts by @ArneTR in #417

Full Changelog: v0.17.1...v0.18

v0.17.1 Small Bugfix in Network-IO Display

The data for the network-IO were correctly measured, but displayed wrongly for the [Remove] phase.

For any new measurement with the tool the data will now show correctly. If you also want to bring your already stored measurements up to the correct display please:

• Run the migration python3 tools/import_data.py migrations/2023_07_10_indices.sql.
• Rebuild the phase_stats with the command: python3 tools/rebuild_phase_stats.py

Happy measuring with now correct Network-IO in the Remove phase :)

v0.17 Performance-, QoL-Updates and Bugfixes

This release changed the port-number of the Postgres DB and also some loading code for the OS filesystem mounts.

Please re-run the install script to bring these changes into effect.

Also some indices have been changed, so please run the migration 2023_07_08_indices.sql

For details see: https://docs.green-coding.berlin/docs/installation/updating/

What's Changed

• Bump fastapi from 0.97.0 to 0.98.0 by @dependabot in #341
• Bump scipy from 1.10.1 to 1.11.0 by @dependabot in #345
• Bump pytest from 7.3.2 to 7.4.0 by @dependabot in #346
• Bump pandas from 2.0.2 to 2.0.3 by @dependabot in #349
• Bump scipy from 1.11.0 to 1.11.1 by @dependabot in #350
• Port swap by @ArneTR in #357
• Bump fastapi from 0.98.0 to 0.99.1 by @dependabot in #359
• Removing version requirement by @ArneTR in #352
• First draft for fix regarding docker context subdir fix by @ArneTR in #353
• Errors are now all logged and excepted ony by one; Stderr does not fa… by @ArneTR in #358
• Docker version info by @ArneTR in #364
• Prune db by @ArneTR in #365
• Non blocking buffer reads by @ArneTR in #368
• No default shell by @ArneTR in #367
• added test cases for subdirectory volume loading by @dan-mm in #366
• Index page loads faster due to async DataTables and only on request by @ArneTR in #372
• Refactored notes to class by @ArneTR in #373
• tmpfs can now be set in install script by @ArneTR in #356

Full Changelog: v0.16...v0.17

v0.16 Eco-CI better stats & Gitlab integration & NOP Linux Measurement Cluster + many small updates

Eco-CI has gotten some major updates and a Gitlab integration and also our NOP Linux measurement cluster is now ready!

• NOP Linux Measurement Cluster installation added
• Performance has been greatly increased by using copy_from to insert phase_stats
• Schema of usage_scenario is now checked before run for bette errors and reproducibility
• API now uses HTTP 204 Status codes to allow proper caching
• Eco-CI frontend allows now for greater drilldown in metrics
• Stdout Logs display in Dashboard
• Runtime phase now supports sub-phases through multiple flows
• Various minor bugfixes

What's Changed

• Adds number of runs selected for comparison by @saiteja13427 in #295
• Rework check configuration for Machine energy by @djesic in #300
• Bump requests from 2.30.0 to 2.31.0 by @dependabot in #302
• Bump pandas from 2.0.1 to 2.0.2 by @dependabot in #309
• Bump requests from 2.30.0 to 2.31.0 by @dependabot in #307
• Bump pydantic from 1.10.7 to 1.10.8 by @dependabot in #308
• Move gunicorn Docker image dependencies to requirements.txt by @djesic in #305
• Refactor phase state to use copy_from by @djesic in #282
• Multi image fix by @ArneTR in #310
• removal of ghost folders in install script by @ArneTR in #312
• Refactor API to use proper HTTP status codes by @djesic in #311
• Eco ci frontend changes by @dan-mm in #296
• 184 schema checker by @dan-mm in #256
• Refactor how form data is sanitized by @djesic in #306
• stats.html detailed metrics table headings reorder by @djesic in #301
• Relative path for usage scenario by @djesic in #303
• Bump pandas from 2.0.1 to 2.0.2 by @dependabot in #317
• Bump orjson from 3.8.12 to 3.9.0 by @dependabot in #316
• Bump pydantic from 1.10.8 to 1.10.9 by @dependabot in #320
• Bump fastapi from 0.95.2 to 0.96.0 by @dependabot in #315
• Bump orjson from 3.9.0 to 3.9.1 by @dependabot in #321
• Bump pytest from 7.3.1 to 7.3.2 by @dependabot in #322
• Bump fastapi from 0.96.0 to 0.97.0 by @dependabot in #323
• Escape single and double quotes by @djesic in #324
• Hotfix for NULL values in SDIA reporter and enforcing in DB by @ArneTR in #331
• Default network creation by @ArneTR in #328
• Stdout Logs display in Dashboard by @ArneTR in #314
• First draft of the runtime-sub-phases by @ArneTR in #319
• Hotfix image creation by @ArneTR in #334
• Eco ci gitlab by @dan-mm in #332
• Check configuration will raise ValueError on fail by @djesic in #336
• Data tables for dashboard by @ArneTR in #343
• Hotfix ipmi by @ArneTR in #333
• Adds support for a distributed cluster setup. by @ribalba in #210

Full Changelog: v0.15...v0.16

v0.15 🔥 Compare Feature & Eco-CI Upgrades

This is a BIG release for the Green Metrics Tool 🔥

We have adapted the measurement workflow to incorporate phases (Baseline, Idle, Installation, Boot, Runtime, Remove) which makes our tool compatible with the requirements specified by the Blue Angel for Software.

Also it is our first step towards a tooling for Software Lifecycle Analysis.

See the comparison in action here, where we compare the influence of changing the database in a Nextcloud installation (swapping MariaDB for SQLite)

2. Also our Eco-CI Tooling has been greatly upgraded and allows now for drilldown towards different CPUs, Labels and timeframes. Check out our Eco-CI feature demo.

3. Contributors
   Big thx also to @saiteja13427 and @shekhar-chauhan for making contributions to the Green Metrics Tool ❤️

What's Changed

• Dev by @dan-mm in #179
• Bugfix: copying of the badge by @djesic in #181
• Adds include functinality to usage_scenario.yml by @ribalba in #157
• 160 cli tests by @dan-mm in #183
• 161 eco ci badges by @dan-mm in #177
• Graph for badges by @djesic in #186
• Refactoring ipmi reporter by @djesic in #194
• Success message at the end of the install script by @djesic in #200
• added config options test for start idle, end idle, and process time;… by @dan-mm in #201
• 127 ci caching by @dan-mm in #195
• Change domain to .internal by @djesic in #206
• Raise value error on invalid note timestamp by @djesic in #199
• Usage scenario now displayed as yaml by @djesic in #196
• Commit URL by @djesic in #198
• Changes volume mounts from -v to --mount bind by @ribalba in #216
• Bump pylint from 2.15.10 to 2.17.1 by @dependabot in #225
• Bump pydantic from 1.10.2 to 1.10.7 by @dependabot in #224
• 207 eco ci rewrite by @dan-mm in #226
• Adds docker build capabilities by @ribalba in #234
• Move psycopg2 to psycopg3 by @djesic in #231
• Create dependabot PRs on dev branch by @djesic in #232
• Bump requests from 2.25.1 to 2.28.2 by @dependabot in #222
• Bump pytest from 7.2.0 to 7.2.2 by @dependabot in #223
• added tests and fixtures for volume loading; refactored test function… by @dan-mm in #197
• Bump pylint from 2.17.1 to 2.17.2 by @dependabot in #237
• Adds the = notatio for env vars by @ribalba in #236
• Convert IPMI reporter to Joules by @djesic in #246
• Stacked bar charts by @djesic in #227
• 242 sql for eco ci by @dan-mm in #247
• Bump pytest from 7.2.2 to 7.3.1 by @dependabot in #248
• Bump pandas from 1.5.3 to 2.0.0 by @dependabot in #238
• Rewrite how we create folders by @djesic in #251
• Added description field to usage scenarios by @djesic in #254
• Compare Functionality by @ArneTR in #241
• Configuration check by @djesic in #253
• Adds option to change units to metric by @saiteja13427 in #263
• XSS in frontend by @djesic in #214
• Blue angel test fixes by @dan-mm in #275
• Extend the error helper to log to file by @saiteja13427 in #264
• Bump pylint from 2.17.2 to 2.17.4 by @dependabot in #285
• Bump requests from 2.28.2 to 2.30.0 by @dependabot in #283
• Bump psycopg[binary] from 3.1.8 to 3.1.9 by @dependabot in #284
• Shift click to select multiple projects to compare by @djesic in #293
• Project info for compare runs by @djesic in #297

New Contributors

• @saiteja13427 made their first contribution in #263

v0.14 macOS support + IPMI + configurable Domain + QOL

• macOS (darwin) as an architecture is now natively supported (Silicon Macs and Intel Macs)
  • Installation on macOS
  • powermetrics provider on macOS
• Domain is now configurable. Was fixed to be https://metrics.green-coding.berlin before
• Fedora support added
• QOL: Charts can now be moved and expanded in width
• More unit-tests
• Bugfixes & Refactoring

What's Changed

• Usage scenario testing 114 by @dan-mm in #154
• Implement IPMI power sensor by @djesic in #148
• 146 port the gmt to mac by @ribalba in #152
• 164 tests update for refactor by @dan-mm in #167
• Configurable domain by @djesic in #168
• 163 jobs update and tests by @dan-mm in #170
• Do not write to /etc/hosts if domain is not local by @djesic in #174
• Charts icons rework by @djesic in #172

Full Changelog: v0.13...v0.14

v0.13 OpenEnergyBadge & Eco-CI + QOL improvements

What has Changed?

• OpenEnergyBadge
• All energy based metric-providers now report in mJ
• Newest XGBoost Model integrated which uses HardwareAvailability and CPUThreads as input variables for more accuarcy
• API now reports referrer if available
• Tests use Eco CI energy estimation in VMs and als eco-ci activity checker Eco CI
• Removed unused fonts and IE icons
• GMT can now ignore errors from processes if specified
• Filename can now be changed from usage_scenario.yml
• GMT can now clone only the branch if requested
• New tests integrated
• Non-Compliant measurements can now be marked
• UI can now move charts and also expand them to full-width
• All python libraries are now installed via the requirements.txt file(s)

Bugfixes

• CO2 calculation contained an error Commit
• Process listing in hardware_info contained only user processes.
• API now dumps request object correctly in error mail
• LmSensors metric provider had typo in class name
• Powerspy2 was not calculating energy correctly in certain cases

Auto Release Notes

• :flame: Forgot the not when refactoring by @ribalba in #113
• Tests minor cleanups by @ArneTR in #119
• -added test output to github summary step by @dan-mm in #129
• Dev by @dan-mm in #130
• 126 - Domain and port changes by @djesic in #131
• Specify branch by @djesic in #135
• added second service and network to connect them; - this tests the ne… by @dan-mm in #133
• Makes the hardware_info script run certain parts as root by @ribalba in #108
• DB tests, a little test re-organization by @dan-mm in #128
• Added branch to projects list by @djesic in #141
• Invalid project flag by @djesic in #132
• 123 invalid measurement by @ArneTR in #143
• • give 'o' flag to grep so that only short test summary is displayed by @dan-mm in #144
• UI charts movement by @djesic in #145
• Changes the powerspy provider to return mJ by @ribalba in #134

Full Changelog: v0.12...v0.13