Help with understanding Impact Framework SCI score

[jjmax75]

Hi folks,

Could I get some help understanding the output from the Sci plugin please?

I'm putting together a presentation for the Green Software Ireland meetup and want to be sure I can explain the numbers I am using and getting back.

I've picked a functional unit - db read/write operations and I have a data set which is divided into 5 minute blocks. A typical input looks like this

timestamp: 2024-05-01T17:00:00.000Z
duration: 300
cpu/utilization: 5.123
readwrite: 3000

I've set my functional-unit to readwrite and my functional-unit-time to 1 week
The result for Sci is something like 0.25.

What does this value represent? The confusing part for me is the time element - is it saying 0.25 gCO2e/kWh for each readwrite operation?

[jmcook1186]

Hello,

if you set your functional-unit to readwrite and your functional-unit-time to 1-week then your SCI will be expressed in a unit of gCO2eq/readwrite/week.

Appreciate this separation into two component units is a bit confusing - we have this issue being worked on this week to make the plugin logic simpler - the issue description contains some detail that might be helpful to better understand how it works now and how we are simplifying it:
Green-Software-Foundation/if#705

[jjmax75]

@jmcook1186 Thanks for your reply and all the info.

Does this mean I can currently remove functional-unit-time and get a result of x gCO2e/readwrite?

If I do that the sci score for a typical input looks something like 4.12334567e-7 (which looks maybe right)

[jmcook1186]

yes, functional-unit-time is optional, you can just pass no value if you don't also want to normalize to some time unit.
Assuming you do that, your SCI value will be gCO2eq per functional-unit, in your case gCO2/readwrite.

Hard to assess whether your values are right without seeing a manifest file, but seems plausible that the carbon consumed for a single read/write operation would be very small.

[jjmax75]

Thanks @jmcook1186 this has been very helpful, I'll mark your first reply as the answer

[jjmax75]

Hi @jmcook1186,
I have another question on this. I'm not sure if this is an issue or if it's down to my understanding of the way Sci and IF works.

Here's an example of one of my manifest file with one input and its output

name: test-carbon-calc
description: Test to demo incorrect calculation of carbon
tags: null
aggregation: null
initialize:
  plugins:
    teads-aws:
      path: '@grnsft/if-unofficial-plugins'
      method: TeadsAWS
      global-config:
        interpolation: linear
    sci:
      path: '@grnsft/if-plugins'
      method: Sci
      global-config:
        functional-unit: ticket-journeys
    sci-o:
      path: '@grnsft/if-plugins'
      method: SciO
  outputs:
    - yaml
if-version: v0.3.2
tree:
  children:
    child:
      pipeline:
        - teads-aws
        - sci-o
        - sci
      defaults:
        cloud/instance-type: m5.xlarge
        grid/carbon-intensity: 118.5
        ticket-journeys: 446.9
      inputs:
        - timestamp: 2024-05-21T09:57:00.000Z
          duration: 60
          cpu/utilization: 9.595
      outputs:
        - timestamp: 2024-05-21T09:57:00.000Z
          duration: 60
          cpu/utilization: 9.595
          cloud/instance-type: m5.xlarge
          grid/carbon-intensity: 118.5
          ticket-journeys: 446.9
          energy: 0.00022982333333333336
          carbon-embodied: 0.031923634893455094
          carbon-operational: 0.027234065000000002
          carbon: 0.0009859616648909182
          sci: 0.0000022062243564352614

For this example I've chosen ticket-journeys as the functional unit and defaulted it to 446.9 - the number of ticket journeys handled by our system in one minute. (a ticket journey is a user purchasing a ticket, registering an account, and using the ticket).

The carbon-embodied and carbon-operational values are generated by the Teads-AWS and Sci-O plugins respectively.

My understanding is these are expressed as gCO2eq and are the values for the duration of the inout (60 seconds in this case)

The carbon value is produced by the sci plugin and is the sum of these two values divided by the duration of the input.
This yields a value expressed as gCO2eq/second

The sci value is the carbon value divided by the functional unit: tickets.

I'm not sure what the sci value is telling me here. Is it gCO2eq/second/ticket? I'm also not sure if this is expected.

[jjmax75]

I've tried a few things. Here's a new demo manifest

name: test-carbon-calc
description: Test to demo incorrect calculation of carbon
tags: null
aggregation:
  metrics:
    - carbon
    - energy
    - total-carbon
  type: both
initialize:
  plugins:
    teads-aws:
      path: '@grnsft/if-unofficial-plugins'
      method: TeadsAWS
      global-config:
        interpolation: linear
    sci:
      path: '@grnsft/if-plugins'
      method: Sci
      global-config:
        functional-unit: ticket-journeys
    sci-o:
      path: '@grnsft/if-plugins'
      method: SciO
    sum:
      path: '@grnsft/if-plugins'
      method: Sum
      global-config:
        input-parameters:
          - carbon-operational
          - carbon-embodied
        output-parameter: total-carbon
    divide:
      path: '@grnsft/if-plugins'
      method: Divide
      global-config:
        numerator: total-carbon
        denominator: ticket-journeys
        output: total-sci
  outputs:
    - yaml
if-version: v0.3.2
tree:
  children:
    child:
      pipeline:
        - teads-aws
        - sci-o
        - sum
        - divide
        - sci
      defaults:
        cloud/instance-type: m5.xlarge
        grid/carbon-intensity: 118.5
        ticket-journeys: 446.9
      inputs:
        - timestamp: 2024-05-21T09:57:00.000Z
          duration: 60
          cpu/utilization: 9.595
      outputs:
        - timestamp: 2024-05-21T09:57:00.000Z
          duration: 60
          cpu/utilization: 9.595
          cloud/instance-type: m5.xlarge
          grid/carbon-intensity: 118.5
          ticket-journeys: 446.9
          energy: 0.00022982333333333336
          carbon-embodied: 0.031923634893455094
          carbon-operational: 0.027234065000000002
          total-carbon: 0.059157699893455096
          total-sci: 0.00013237346138611568
          carbon: 0.0009859616648909182
          sci: 0.0000022062243564352614
      aggregated:
        carbon: 0.0009859616648909182
        energy: 0.00022982333333333336
        total-carbon: 0.059157699893455096
  outputs:
    - carbon: 0.0009859616648909182
      energy: 0.00022982333333333336
      total-carbon: 0.059157699893455096
      timestamp: 2024-05-21T09:57:00.000Z
      duration: 60
  aggregated:
    carbon: 0.0009859616648909182
    energy: 0.00022982333333333336
    total-carbon: 0.059157699893455096

This is using the sum and divide modules to calculate Sci plus I added an aggregation section to calculate the total values of carbon from the Sci plugin and the manual approach.

With the Sci plugin the aggregation result is the total of all the Sci carbon values added together. As Sci is giving carbon/second - should this value be multiplied by the duration (60)?

[jmcook1186]

Thanks for digging into this. Yes, in the current implementation, unless you provide a functional-unit-time the value gets normalized to per second, so to have a SCI value expressed as gCO2eq/ticket-journey you would need to make the functional-unit-time match your duration.

e.g.

name: sci
description: successful path
tags:
initialize:
  plugins:
    sci:
      kind: plugin
      method: Sci
      path: "builtin"
tree:
  children:
    child:
      pipeline:
        - sci
      config:
        sci:
          functional-unit-time: 1 min
          functional-unit: ticket-journeys 
      inputs:
        - timestamp: 2023-07-06T00:00
          duration: 60
          energy: 5
          carbon: 10
          ticket-journeys: 10

Appreciate this isn't the most intuitive way to handle this - we are refactoring the plugin as we speak to straighten it all out and make it simpler to use.

[jjmax75]

THanks @jmcook1186 that's very helpful and fixes my problem