Attempt to Build Findings for Sustainability Considerations

[StSchnell]

In the context of a sustainability analysis experiences are described here, based on the answers to a specific question. Furthermore an attempt will be made, to build findings for sustainability considerations. This bases of an analysis of a JavaScript library, which is mainly server-side used in business.

Sustainability Comparison of forEach and for-each-in Iterators of the Rhino JavaScript Engine

The Mozilla Rhino engine offers various options for working with indexed collections. One is the Array.prototype.forEach loop, known from the ECMAScript language specification. Another is the for-each-in statement from the ECMAScript for XML (E4X) specification. The Rhino engine is used in many business and enterprise products. It can be assumed that these iterators are often used in customer programming. From this perspective a consideration of performance and energy consumption is certainly a valuable aspect. The question is therefore asked: Which of the iterators of the Rhino engine is the more sustainable approach, forEach or for-each-in?

The following JavaScript test program was used to compare the two approaches:

var arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
  16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
  31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45,
  46, 47, 48, 49, 50];

function dummy() {
  arr.forEach( function(num) { } );
  for each (var num in arr) { }
}

function _forEach() {
  arr.forEach( function(num) { } );
}

function _for_each() {
  for each (var num in arr) { }
}

function main() {

  dummy();

  for (var i = 1; i <= 10000000; i++) {
    _forEach();
    _for_each();
  }

}

// Main
main();

There are three functions that are called by main function. The dummy function is only used for initialization. During my tests I noticed that the function called first always shows significantly poorer performance. To compensate this behavior the dummy function is called first. Their consumption has no effect on the following considerations.

Then there is the function _forEach, which contains Array.prototype.forEach, and _for_each, which contains the for-each-in statement. Both process an array of 50 numbers 10 million times. The high number of iterations is really necessary to get meaningful results. If the number is too low, the analysis tools may not recognize the execution of the functions.

Four approaches were used for this analysis:

Java Profiler

The first analysis was done with a Java profiler.

	Samples	Standard Deviation
compareForEach._c__for_each_5 (compareForEach.js:19)	653	107
compareForEach._c__forEach_3 (compareForEach.js:15)	151	16

We see here that the for-each-in statement appears approximately 4.31 times more frequently in the samples.

JavaScript Profiler

The second analysis was done with a Rhino JavaScript profiler, which bases on the Java Profiler. This profiler was specially developed for use in platforms that use the Rhino Engine as the basis for customer coding.

	Samples	Standard Deviation
compareForEach_jsProfiler._c__for_each_19	18490	746
compareForEach_jsProfiler._c__forEach_17	5093	42

Here is the for-each-in statement approximately 3.63 times more frequently in the samples. This result is similar to that of the Java Profiler, if a blind eye is turned.

Time Consumption

The third analysis was used to measure the time consumption of the functions.

	Time	Standard Deviation
_for_each	11080 ms	1942
_forEach	3754 ms	486

Here needs the for-each-in statement approximately 2.95 times more time.

Energy Consumption

The fourth analysis was done with jPowerMonitor, a library for measuring energy consumption.

	Energy	Standard Deviation	CO2
compareForEach._c__for_each_5	314.71661 J	64.20133	0.05223 g
compareForEach._c__forEach_3	0.93786 J	0.53404	0.00011 g

Here we can see that the for-each-in statement also consumes significantly more energy and therefore generates more CO2.

Conclusion

From these four analyses we can summarize that Array.prototype.forEach is in this case more performant than the for-each-in statement. In addition Array.prototype.forEach consumes significantly less energy than the for-each-in statement. A recommendation can be derived from this, to avoid the using the for-each-in statement if possible.

Analysis Basics

Measurement Methodology

The profiler measurements were made with Bellsoft JDK 22 in Windows 10. The energy consumption measurement were made with Bellsoft JDK 21.0.1 in Windows 10. Rhino Engine 1.7.14 was used in all measurements. No manual activities were executed on the system during the measurements. Background activities were possible, but were monitored with the Task Manager and none were detected. This means that a constant measuring environment can be assumed. All measurements were made ten times and then the average is here as an example quoted.

Code Analysis

The Mozilla Rhino scripting engine transforms the JavaScript code into Java bytecode, which will then be executed by Java. The Java source code can only be generated by decompiling. This automatically generated source code is not easy to read. But it can be very informative to see how the JavaScript code is transformed into Java code.

Experiences and Findings of Sustainability

1. To derive recommendations the source code must be known, this means that this kind of analysis can only be white box testing, as far as possible.
2. To get reliable statements it is necessary to run as many iterations as possible.
3. The level of abstraction of the Rhino engine to the Java execution environment complicates this kind of analysis. The software stack used should be kept as small as possible.
4. The results of the different analyses via stack trace sample counting by the profilers, the time and energy consumption measurement shows a loose correlation. This provides indications, but no definitive statements can be derived from this.
5. In addition to the detailed analysis of the functions, the behavior of the complete program execution should also be considered. For example, initialization routines can be detected on this way, which can significantly influence the measurement result.
6. Administrative access to the system is required for energy consumption measurement. This could be problematic in some environments, so that reference systems will have to be used.
7. Platforms that use the Rhino engine in interpretive mode, profilers do not provide a useful result because the JavaScript functions are hidden behind other calls. In this case the measurements must be run on reference systems.

Conclusion

The question raised, which of the iteration methods forEach or for-each-in was the more sustainable, could be answered.

In any case there have been some findings. It is always preferable to use several approaches in a sustainability analysis, to build a balance. Because there were no direct interdependencies visible between the analysis approaches, so that clear conclusions cannot be made from their relations. The use of a reference system can probably not often be avoided.

It can be assumed that libraries, that work in a similar way like the Rhino engine, can also be tested for sustainability using the same experiences and findings described here.

[p-bakker]

a Rhino JavaScript profiler, which bases on the Java Profiler. This profiler was specially developed for use in platforms that use the Rhino Engine as the basis for customer coding

Which profiler is this?

[StSchnell]

Hello @p-bakker,
in this example I used as JavaScript Profiler an implementation of the Tiny Profiler in JavaScript.

// Begin ---------------------------------------------------------------

/**
 * JavaScript implementation of Tiny Profiler to analyze which methods
 * consume the most resources. This approach is a statistical profiler.
 * It samples the call stack periodically and collects the results.
 *
 * @function profiler
 * @param {Any} in_functionToProfile
 * @param {number} in_sampleInterval
 * @param {boolean} in_showAll
 */
function profiler(in_functionToProfile, in_sampleInterval, in_showAll) {

  var __stop = false;
  var __totalSampleCount = 0;
  var __samples = [];

  var Profiler = function(interval) {
    this.interval = interval;
  };

  Profiler.newInstance = function(interval) {
    var profiler = new Profiler(interval);
    java.lang.Runtime.getRuntime().addShutdownHook(
      java.lang.Thread(profiler.onEnd)
    );
    return profiler;
   };
 
  Profiler.prototype = {

    addSample : function(stackTraceElements) {
      stackTraceElements.forEach( function(stackTraceElement) {
        var sample = stackTraceElement.getClassName();
        sample += ".";
        sample += stackTraceElement.getMethodName();
        __samples.push(sample);
      } );
      __totalSampleCount++;
    },

    sample : function() {
      try {
        var allStackTraces = java.lang.Thread.getAllStackTraces();
        var iterator = allStackTraces.entrySet().iterator();
        while (iterator.hasNext()) {
          var entry = iterator.next();
          if (!entry.getKey().isDaemon()) {
            this.addSample(entry.getValue());
          }
        }

      } catch (exception) {
        print("Error: " + exception);
      }
    },

    sleep : function(duration) {
      if (duration.isNegative() || duration.isZero()) {
        return;
      }
      java.lang.Thread.sleep(
        duration.toMillis(),
        duration.toNanosPart() % 1000000
      );
    },

    onEnd : function() {
      __stop = true;
      while (__stop);
    }

  };

  /**
   * Executes the profiler to analyze a function.
   *
   * @function executeProfiler
   * @param {function} functionToProfile
   * @param {number} sampleInterval
   * @param {boolean} showAll
   */
  function executeProfiler(functionToProfile, sampleInterval, showAll) {

    if (typeof functionToProfile !== "function") {
      throw new Error("functionToProfile argument must be function");
    }

    var _sampleInterval = java.time.Duration.ofMillis(10);
    if (
      typeof sampleInterval === "number" &&
      sampleInterval > 0
    ) {
      _sampleInterval = java.time.Duration.ofMillis(sampleInterval);
    }

    var _showAll = true;
    if (typeof showAll === "boolean") {
      _showAll = showAll;
    }

    var profiler = Profiler.newInstance(_sampleInterval);

    var run = function() {
      var start = java.time.Duration.ofNanos(java.lang.System.nanoTime());
      while (!__stop) {
        profiler.sample();
        var duration = java.time.Duration.ofNanos(
          java.lang.System.nanoTime()
        ).minus(start);
        var sleep = profiler.interval.minus(duration);
        try {
          profiler.sleep(sleep);
        } catch (exception) {
          print("Warn: " + exception);
          break;
        }
      }
      __stop = false;
    };

    // Start thread to record the stacktrace
    var thread = new java.lang.Thread(run);
    thread.setDaemon(true);
    thread.setName("Profiler");
    thread.start();

    // Profile function
    functionToProfile();

    var __ssamples = __samples.slice();
    // Processing of the results
    print("Total samples: " + __totalSampleCount);
    print("===== Method Table ======");
    var count = {};
    __ssamples.sort().forEach( function(el) {
      count[el] = count[el] + 1 || 1;
    });

    // Put result into an array
    var counter = [];
    for (var key in count) {
      var cnt = {
        "key" : key,
        "value" : count[key]
      };
      counter.push(cnt);
    }

    // Sort the result by samples descending
    counter.sort(function(a, b) {
      if (a.value > b.value) return -1;
      if (a.value < b.value) return 1;
      return 0;
    });

    // Output of the result
    counter.forEach( function(item) {
      var methodName =
        JSON.stringify(item.key).replace(/\"/g, "").substring(0, 100);
      if (_showAll) {
        print(
          methodName + Array((100 - methodName.length) + 1).join(" ") +
          " " +
          JSON.stringify(item.value)
        );
      } else {
        if (methodName.indexOf("jsProfiler") !== -1) {
          print(
            methodName + Array((100 - methodName.length) + 1).join(" ") +
            " " +
            JSON.stringify(item.value)
          );
        }
      }
    });

  }

  executeProfiler(in_functionToProfile, in_sampleInterval, in_showAll);

}

// End -----------------------------------------------------------------

var functionToProfile = function compareForEach() {

  var arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
    16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
    31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45,
    46, 47, 48, 49, 50];

  function dummy() {
    arr.forEach( function(num) { });
    for each (var num in arr) { }
  }

  function _forEach() {
    arr.forEach( function(num) { });
  }

  function _for_each() {
    for each (var num in arr) { }
  }

  dummy();

  for (var i = 1; i <= 1000000; i++) {
    _forEach();
    _for_each();
  }

};

// Main
profiler(functionToProfile, 10, false);

Best regards
Stefan

[p-bakker]

Ah, you're using a Java Profiler from JavaScript to profile the Java code that resulted from compiling JavaScript code to Java when using Rhino in compiled mode (optLevel >= 0)

Thought you were suggesting you use a real JavaScript profiler (that would also work in interpreted mode for example)

[p-bakker]

Would bed interested in comparing .for each against a regular for loop as well as a for-of loop

[StSchnell]

Hello @p-bakker,
here my test program with additional for, for-of and while loop.

var functionToProfile = function compareForEach() {

  var arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
    16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
    31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45,
    46, 47, 48, 49, 50];

  function dummy() {
    arr.forEach( function(num) { });
    for each (var num in arr) { }
    for(var i = 0; i < arr.length; i++) { }
  }

  function _forEach() {
    arr.forEach( function(num) { });
  }

  function _for_each() {
    for each (var num in arr) { }
  }

  function _for() {
    for(var i = 0; i < arr.length; i++) { }
  }

  function _for_of() {
    for(var i of arr) { }
  }

  function _while() {
    var i = 0;
    while(i < arr.length) {
      i++;
    }
  }

  dummy();

  for (var i = 1; i <= 1000000; i++) {
    _forEach();
    _for_each();
    _for();
    _for_of();
    _while();
  }

};

And here my results.

Total samples: 82935
===== Method Table ======
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1.call                         494469
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c_executeProfiler_9         82566
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c_profiler_1                82566
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c_script_0                  82566
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1.exec                         82566
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c_compareForEach_14         82566
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c__for_of_21                47223
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c__for_20                   9694
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c__for_each_19              9563
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c__while_22                 9541
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._c__forEach_17               5562
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1.<init>                       234
org.mozilla.javascript.gen.compareForEach_jsProfiler_001_js_1._i18                         224

Best regards
Stefan