From ca4656198febc4590dd7de54b651f82c7550ddc0 Mon Sep 17 00:00:00 2001 From: "Vincent A. Cicirello" Date: Wed, 29 May 2024 16:01:06 -0400 Subject: [PATCH] Added experiments (#142) * timing experiment Java 17+ * Update pom.xml * updated benchmarks * Update TimeZigguratVersusJavaBuiltin.java * change benchmark mode, etc * Create results.txt * moved directories * update experiments * renamed directory * data from runs of new version * deternine which gaussian alg for all PRNGs * Create README.md * Create README.md * Create README.md * Update README.md * Update README.md * Update README.md --- .gitignore | 1 + README.md | 31 ++- experiment/README.md | 20 ++ experiment/properties/README.md | 29 ++ experiment/properties/pom.xml | 175 ++++++++++++ .../whatgaussian/WhatGaussian.java | 72 +++++ .../what-gaussian-implementation.txt | 15 + experiment/timing17/README.md | 25 ++ experiment/timing17/pom.xml | 259 ++++++++++++++++++ experiment/timing17/results17.txt | 254 +++++++++++++++++ .../TimeZigguratVersusJavaBuiltin.java | 174 ++++++++++++ 11 files changed, 1051 insertions(+), 4 deletions(-) create mode 100644 experiment/README.md create mode 100644 experiment/properties/README.md create mode 100644 experiment/properties/pom.xml create mode 100644 experiment/properties/src/main/java/org/cicirello/experiments/whatgaussian/WhatGaussian.java create mode 100644 experiment/properties/what-gaussian-implementation.txt create mode 100644 experiment/timing17/README.md create mode 100644 experiment/timing17/pom.xml create mode 100644 experiment/timing17/results17.txt create mode 100644 experiment/timing17/src/main/java/org/cicirello/experiments/ziggurat/TimeZigguratVersusJavaBuiltin.java diff --git a/.gitignore b/.gitignore index 2f7896d..8b8c81d 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ target/ +dependency-reduced-pom.xml diff --git a/README.md b/README.md index 04441aa..1b1c79d 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [Ziggurat Gaussian - Fast Gaussian distributed pseudorandom number generation in Java via the Ziggurat algorithm](#ziggurat-gaussian) -Copyright (C) 2015, 2017-2023 [Vincent A. Cicirello](https://www.cicirello.org/). +Copyright (C) 2015, 2017-2024 [Vincent A. Cicirello](https://www.cicirello.org/). | __Packages and Releases__ | [![Maven Central](https://img.shields.io/maven-central/v/org.cicirello/ziggurat.svg?label=Maven%20Central&logo=apachemaven)](https://central.sonatype.com/artifact/org.cicirello/ziggurat/) [![GitHub release (latest by date)](https://img.shields.io/github/v/release/cicirello/ZigguratGaussian?logo=GitHub)](https://github.com/cicirello/ZigguratGaussian/releases) | | :--- | :--- | @@ -35,12 +35,35 @@ Java API is the polar method (nextGaussian method of the Random and ThreadLocalRandom classes, however the polar method is quite slow relative to other newer available alternatives, such as the Ziggurat method. -You can find some experimental data comparing the performance of a sequential +## When to Use This Library + +The following table summarizes when this library will speed up Gaussian random +number generation relative to Java's builtin functionality: + +| Java Version | Significantly Faster For | +| --- | --- | +| Java 11 ≤ version < Java 17 | `Random`, `SecureRandom`, `SplittableRandom`,`ThreadLocalRandom` | +| Java version ≥ Java 17 | `Random`, `SecureRandom`, `ThreadLocalRandom` | + +Source code and data of experiments with Java 17 can be found in +the [experiment](experiment) directory. The following paper discusses +experiments using the library with Java 17. In Java 17, several enhancements +to random number generation were introduced, including several new random number +geenrators a RandomGenerator interface, and a modified ziggurat for the new random +number generators as well as for `SplittableRandom`. The other legacy generators +still use the slow polar method, however. This report explores how and where our +ziggurat library is still relevant in Java 17. + +> Vincent A. Cicirello. 2024. [Fast Gaussian Distributed Pseudorandom Number Generation in Java via the Ziggurat Algorithm](https://reports.cicirello.org/24/009/). Technical Report ALG-24-009, Cicirello.org, May 2024. [[PDF]](https://reports.cicirello.org/24/009/ALG-24-009.pdf) + +You can find some additional experimental data comparing the performance of a sequential genetic algorithm (GA) using this implementation of the Ziggurat method for Gaussian mutation vs using the more common polar method, as well as experimental data for the same comparison but with a PGA, in the following paper: -> V. A. Cicirello. [Impact of Random Number Generation on Parallel Genetic Algorithms](https://www.cicirello.org/publications/cicirello2018flairs.html). *Proceedings of the Thirty-First International Florida Artificial Intelligence Research Society Conference*, pages 2-7. AAAI Press, May 2018. +> V. A. Cicirello. [Impact of Random Number Generation on Parallel Genetic Algorithms](https://www.cicirello.org/publications/cicirello2018flairs.html). *Proceedings of the Thirty-First International Florida Artificial Intelligence Research Society Conference*, pages 2-7. AAAI Press, May 2018. [[PDF]](https://www.cicirello.org/publications/cicirello-flairs2018.pdf). + +## Background on the Algorithm Itself See the following articles for detailed description of the Ziggurat algorithm itself, as well as additional experimental data: @@ -83,7 +106,7 @@ the version number with the version that you want to use. org.cicirello ziggurat - 1.0.4 + 1.0.5 ``` diff --git a/experiment/README.md b/experiment/README.md new file mode 100644 index 0000000..d3f172e --- /dev/null +++ b/experiment/README.md @@ -0,0 +1,20 @@ +# Experiments, Sample Code, and Other Misc + +This directory contains source code of experiments with the +library, sample code, and other misc items, as follows. + +## Experiments with the Ziggurat Library and Java 17 + +The code in the directory [timing17](timing17) can be used to reproduce the experiments of the following paper, which among other things explores the relevance of the library for Java 17+: + +> Vincent A. Cicirello. 2024. [Fast Gaussian Distributed Pseudorandom Number Generation in Java via the Ziggurat Algorithm](https://reports.cicirello.org/24/009/). Technical Report ALG-24-009, Cicirello.org, May 2024. [[PDF]](https://reports.cicirello.org/24/009/ALG-24-009.pdf) + +## Which Gaussian Algorithm Does Java Use + +The code in the directory [properties](properties) extracts a list of all of the +random number generators for Java 17, and determines what implementation of +`nextGaussian` each uses. Classes that inherit the default in Java 17 use a modified +ziggurat, which is faster than the original ziggurat algorithm. The `Random` class, +and the classes that extend it override the default with the slow polar method. Those +are cases where our implementation of the original ziggurat algorithm is still relevant +for Java 17+ applications. \ No newline at end of file diff --git a/experiment/properties/README.md b/experiment/properties/README.md new file mode 100644 index 0000000..aceef3d --- /dev/null +++ b/experiment/properties/README.md @@ -0,0 +1,29 @@ +# Which Gaussian Algorithm Does Java Use + +The code in this directory extracts a list of all of the random number generators +for Java 17, and determines what implementation of `nextGaussian` each uses. Classes +that inherit the default in Java 17 use a modified ziggurat, which is faster than +the original ziggurat algorithm. The `Random` class, and the classes that extend it +override the default with the slow polar method. Those are cases where our implementation +of the original ziggurat algorithm is still relevant for Java 17+ applications. + +## Building with Maven + +To compile, execute the following within this directory: + +```Shell +mvn clean package +``` + +## Running + +To run, execute the following within this directory: + +```Shell +java -cp target/what-gaussian-1.0.0.jar org.cicirello.experiments.whatgaussian.WhatGaussian +``` + +## Output + +If you just want to inspect the output from my run, see the +file [what-gaussian-implementation.txt](what-gaussian-implementation.txt). \ No newline at end of file diff --git a/experiment/properties/pom.xml b/experiment/properties/pom.xml new file mode 100644 index 0000000..5f5f89d --- /dev/null +++ b/experiment/properties/pom.xml @@ -0,0 +1,175 @@ + + 4.0.0 + + org.cicirello + what-gaussian + 1.0.0 + jar + + Check which RandomGenerators use default Gaussian algorithm. + + Determines which of Java's builtin pseudorandom number generators + use the RandomGenerator interface's default algorithm for nextGaussian, which + override it. + + + https://github.com/cicirello/ZigguratGaussian + + + + GPL-3.0-or-later + https://www.gnu.org/licenses/gpl-3.0.en.html + repo + + Check which RandomGenerators use default Gaussian algorithm. + Copyright (C) 2024 Vincent A. Cicirello. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see https://www.gnu.org/licenses/. + + + + + + + Vincent A Cicirello + development@cicirello.org + https://www.cicirello.org/ + Cicirello.org + https://www.cicirello.org/ + + + + + Cicirello.org + https://www.cicirello.org/ + + + + + analysis + + + + com.github.spotbugs + spotbugs-maven-plugin + 4.8.1.0 + + true + ${session.executionRootDirectory}/spotbugs-exclude.xml + + + com.h3xstream.findsecbugs + findsecbugs-plugin + 1.12.0 + + + + + + test + + spotbugs + + + + + + + + + + + github + https://github.com/cicirello/ZigguratGaussian/issues + + + + + org.cicirello + ziggurat + 1.0.5 + + + + + UTF-8 + 17 + true + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.11.0 + + 17 + + + + org.apache.maven.plugins + maven-source-plugin + 3.3.0 + + + attach-sources + + jar + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.6.2 + + + attach-javadocs + + jar + + + + + false + true + Check which RandomGenerators use default Gaussian algorithm + Check which RandomGenerators use default Gaussian algorithm + true + false + true + true + Vincent A. Cicirello. All rights reserved.]]> + + + + com.spotify.fmt + fmt-maven-plugin + 2.21.1 + + + + format + + + + + + + \ No newline at end of file diff --git a/experiment/properties/src/main/java/org/cicirello/experiments/whatgaussian/WhatGaussian.java b/experiment/properties/src/main/java/org/cicirello/experiments/whatgaussian/WhatGaussian.java new file mode 100644 index 0000000..d965f15 --- /dev/null +++ b/experiment/properties/src/main/java/org/cicirello/experiments/whatgaussian/WhatGaussian.java @@ -0,0 +1,72 @@ +/* + * Check which RandomGenerators use default Gaussian algorithm. + * Copyright (C) 2024 Vincent A. Cicirello + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.cicirello.experiments.whatgaussian; + +import java.util.concurrent.ThreadLocalRandom; +import java.util.random.RandomGenerator; +import java.util.random.RandomGeneratorFactory; + +/** + * Determines which of Java's builtin pseudorandom number generators use the RandomGenerator + * interface's default algorithm for nextGaussian, which override it. + * + * @author Vincent A. Cicirello, https://www.cicirello.org/ + */ +public final class WhatGaussian { + + /** + * Entrypoint for program. + * + * @param args unused + */ + public static void main(String[] args) { + + System.out.printf("%10s %21s %s\n", "Group", "RandomGenerator", "Where nextGaussian declared"); + RandomGeneratorFactory.all() + .sorted((f1, f2) -> f1.group().compareTo(f2.group())) + .forEach( + generatorFactory -> { + String name = generatorFactory.name(); + String group = generatorFactory.group(); + RandomGenerator r = generatorFactory.create(); + try { + String classThatDeclaresNextGaussian = + r.getClass().getMethod("nextGaussian").getDeclaringClass().getName(); + System.out.printf("%10s %21s %s\n", group, name, classThatDeclaresNextGaussian); + } catch (NoSuchMethodException exception) { + System.out.println( + "Something highly unexpexpected occured. A method with a default not found."); + } + }); + try { + String classThatDeclaresNextGaussian = + ThreadLocalRandom.current() + .getClass() + .getMethod("nextGaussian") + .getDeclaringClass() + .getName(); + System.out.printf( + "%10s %21s %s\n", "Legacy", "ThreadLocalRandom", classThatDeclaresNextGaussian); + } catch (NoSuchMethodException exception) { + System.out.println( + "Something highly unexpexpected occured. A method with a default not found."); + } + } +} diff --git a/experiment/properties/what-gaussian-implementation.txt b/experiment/properties/what-gaussian-implementation.txt new file mode 100644 index 0000000..10737b0 --- /dev/null +++ b/experiment/properties/what-gaussian-implementation.txt @@ -0,0 +1,15 @@ + Group RandomGenerator Where nextGaussian declared + LXM L32X64MixRandom java.util.random.RandomGenerator + LXM L128X128MixRandom java.util.random.RandomGenerator + LXM L64X128MixRandom java.util.random.RandomGenerator + LXM L128X1024MixRandom java.util.random.RandomGenerator + LXM L64X128StarStarRandom java.util.random.RandomGenerator + LXM L64X256MixRandom java.util.random.RandomGenerator + LXM L128X256MixRandom java.util.random.RandomGenerator + LXM L64X1024MixRandom java.util.random.RandomGenerator + Legacy SecureRandom java.util.Random + Legacy Random java.util.Random + Legacy SplittableRandom java.util.random.RandomGenerator + Xoroshiro Xoroshiro128PlusPlus java.util.random.RandomGenerator + Xoshiro Xoshiro256PlusPlus java.util.random.RandomGenerator + Legacy ThreadLocalRandom java.util.Random diff --git a/experiment/timing17/README.md b/experiment/timing17/README.md new file mode 100644 index 0000000..cfc3ddb --- /dev/null +++ b/experiment/timing17/README.md @@ -0,0 +1,25 @@ +# Experiments with the Ziggurat Library and Java 17 + +The code in this directory can be used to reproduce the experiments of the following paper: + +> Vincent A. Cicirello. 2024. [Fast Gaussian Distributed Pseudorandom Number Generation in Java via the Ziggurat Algorithm](https://reports.cicirello.org/24/009/). Technical Report ALG-24-009, Cicirello.org, May 2024. [[PDF]](https://reports.cicirello.org/24/009/ALG-24-009.pdf) + +## Building with Maven + +To compile the experiments, execute the following within this directory: + +```Shell +mvn clean package +``` + +## Running + +The above builds a `jar-with-dependencies` that is executable, and which will be found within the target directory, which is created if it doesn't already exist. You can then run it with the following: + +```Shell +java -jar target/ziggurat-timing-1.0.0-jar-with-dependencies.jar +``` + +## Data + +If you just want to inspect the results from my run, you can find the data in [results17.txt](results17.txt). \ No newline at end of file diff --git a/experiment/timing17/pom.xml b/experiment/timing17/pom.xml new file mode 100644 index 0000000..e920702 --- /dev/null +++ b/experiment/timing17/pom.xml @@ -0,0 +1,259 @@ + + 4.0.0 + + org.cicirello + ziggurat-timing + 1.0.0 + jar + + Experiment comparing org.cicirello.ziggurat versus Java builtin implementation + + Compares Java's builtin Gaussian implementation with this Ziggurat impementation. Beginning with Java 17, the Java API's default Gaussian + implementation is based on the Ziggurat algorithm (except for the Random class, + which still uses the Polar Method). Thus, for Java 17+, it doesn't seem like there + should still be any benefit from my implementation of the Ziggurat algorithm. + + The aim of this program is to empirically verify that Java's builtin Gaussian + method is at least as good as my implementation for Java 17+. If you run this + on earlier versions of Java, the Java builtin implementation should be slower. + But on newer, it will probably be faster. + + + https://github.com/cicirello/ZigguratGaussian + + + + GPL-3.0-or-later + https://www.gnu.org/licenses/gpl-3.0.en.html + repo + + Experiment comparing org.cicirello.ziggurat versus Java builtin implementation. + Copyright (C) 2023-2024 Vincent A. Cicirello. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see https://www.gnu.org/licenses/. + + + + + + + Vincent A Cicirello + development@cicirello.org + https://www.cicirello.org/ + Cicirello.org + https://www.cicirello.org/ + + + + + Cicirello.org + https://www.cicirello.org/ + + + + + analysis + + + + com.github.spotbugs + spotbugs-maven-plugin + 4.8.1.0 + + true + ${session.executionRootDirectory}/spotbugs-exclude.xml + + + com.h3xstream.findsecbugs + findsecbugs-plugin + 1.12.0 + + + + + + test + + spotbugs + + + + + + + + + + + github + https://github.com/cicirello/ZigguratGaussian/issues + + + + + org.junit.jupiter + junit-jupiter + 5.10.1 + test + + + org.cicirello + ziggurat + 1.0.5 + + + org.openjdk.jmh + jmh-core + 1.37 + + + org.openjdk.jmh + jmh-generator-annprocess + 1.37 + + + + + UTF-8 + 17 + true + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.11.0 + + 17 + + + + org.apache.maven.plugins + maven-source-plugin + 3.3.0 + + + attach-sources + + jar + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.6.2 + + + attach-javadocs + + jar + + + + + false + true + Experiment comparing org.cicirello.ziggurat versus Java builtin implementation + Experiment comparing org.cicirello.ziggurat versus Java builtin implementation + true + false + true + true + Vincent A. Cicirello. All rights reserved.]]> + + + + com.spotify.fmt + fmt-maven-plugin + 2.21.1 + + + + format + + + + + + org.apache.maven.plugins + maven-shade-plugin + 3.5.3 + + + + org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin + + + true + jar-with-dependencies + + + *:* + + module-info.class + LICENSE + THIRD-PARTY + + + + org.cicirello:ziggurat + + META-INF/MANIFEST.MF + + + + org.openjdk.jmh:jmh-generator-annprocess + + META-INF/MANIFEST.MF + + + + org.openjdk.jmh:jmh-core + + META-INF/MANIFEST.MF + + + + net.sf.jopt-simple:jopt-simple + + META-INF/MANIFEST.MF + + + + org.apache.commons:commons-math3 + + META-INF/MANIFEST.MF + + + + + + + package + + shade + + + + + + + \ No newline at end of file diff --git a/experiment/timing17/results17.txt b/experiment/timing17/results17.txt new file mode 100644 index 0000000..971142e --- /dev/null +++ b/experiment/timing17/results17.txt @@ -0,0 +1,254 @@ + +Experiment comparing org.cicirello.ziggurat versus Java builtin implementation. +Copyright (C) 2023-2024 Vincent A. Cicirello +This program comes with ABSOLUTELY NO WARRANTY. This is free +software, and you are welcome to redistribute it under certain +conditions. See the GNU General Public License for more +details: https://www.gnu.org/licenses/gpl-3.0.html + +# JMH version: 1.37 +# VM version: JDK 17.0.2, OpenJDK 64-Bit Server VM, 17.0.2+8 +# VM invoker: D:\InstalledPrograms\Adoptium\jdk-17.0.2.8-hotspot\bin\java.exe +# VM options: +# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) +# Warmup: 5 iterations, 10 s each +# Measurement: 5 iterations, 10 s each +# Timeout: 10 min per iteration +# Threads: 1 thread, will synchronize iterations +# Benchmark mode: Average time, time/op +# Benchmark: org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.builtinRandom + +# Run progress: 0.00% complete, ETA 00:11:40 +# Fork: 1 of 1 +# Warmup Iteration 1: 107.678 ns/op +# Warmup Iteration 2: 106.841 ns/op +# Warmup Iteration 3: 102.926 ns/op +# Warmup Iteration 4: 103.072 ns/op +# Warmup Iteration 5: 102.974 ns/op +Iteration 1: 102.820 ns/op +Iteration 2: 103.388 ns/op +Iteration 3: 103.066 ns/op +Iteration 4: 102.784 ns/op +Iteration 5: 103.129 ns/op + + +Result "org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.builtinRandom": + 103.037 ±(99.9%) 0.951 ns/op [Average] + (min, avg, max) = (102.784, 103.037, 103.388), stdev = 0.247 + CI (99.9%): [102.086, 103.988] (assumes normal distribution) + + +# JMH version: 1.37 +# VM version: JDK 17.0.2, OpenJDK 64-Bit Server VM, 17.0.2+8 +# VM invoker: D:\InstalledPrograms\Adoptium\jdk-17.0.2.8-hotspot\bin\java.exe +# VM options: +# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) +# Warmup: 5 iterations, 10 s each +# Measurement: 5 iterations, 10 s each +# Timeout: 10 min per iteration +# Threads: 1 thread, will synchronize iterations +# Benchmark mode: Average time, time/op +# Benchmark: org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.builtinSplittableRandom + +# Run progress: 14.29% complete, ETA 00:10:04 +# Fork: 1 of 1 +# Warmup Iteration 1: 10.203 ns/op +# Warmup Iteration 2: 9.905 ns/op +# Warmup Iteration 3: 8.924 ns/op +# Warmup Iteration 4: 8.924 ns/op +# Warmup Iteration 5: 8.922 ns/op +Iteration 1: 8.919 ns/op +Iteration 2: 8.922 ns/op +Iteration 3: 8.931 ns/op +Iteration 4: 8.927 ns/op +Iteration 5: 8.936 ns/op + + +Result "org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.builtinSplittableRandom": + 8.927 ±(99.9%) 0.026 ns/op [Average] + (min, avg, max) = (8.919, 8.927, 8.936), stdev = 0.007 + CI (99.9%): [8.901, 8.953] (assumes normal distribution) + + +# JMH version: 1.37 +# VM version: JDK 17.0.2, OpenJDK 64-Bit Server VM, 17.0.2+8 +# VM invoker: D:\InstalledPrograms\Adoptium\jdk-17.0.2.8-hotspot\bin\java.exe +# VM options: +# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) +# Warmup: 5 iterations, 10 s each +# Measurement: 5 iterations, 10 s each +# Timeout: 10 min per iteration +# Threads: 1 thread, will synchronize iterations +# Benchmark mode: Average time, time/op +# Benchmark: org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.builtinThreadLocalRandom + +# Run progress: 28.57% complete, ETA 00:08:23 +# Fork: 1 of 1 +# Warmup Iteration 1: 90.549 ns/op +# Warmup Iteration 2: 90.284 ns/op +# Warmup Iteration 3: 88.341 ns/op +# Warmup Iteration 4: 88.344 ns/op +# Warmup Iteration 5: 88.320 ns/op +Iteration 1: 89.693 ns/op +Iteration 2: 88.742 ns/op +Iteration 3: 88.323 ns/op +Iteration 4: 88.695 ns/op +Iteration 5: 88.414 ns/op + + +Result "org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.builtinThreadLocalRandom": + 88.774 ±(99.9%) 2.096 ns/op [Average] + (min, avg, max) = (88.323, 88.774, 89.693), stdev = 0.544 + CI (99.9%): [86.678, 90.870] (assumes normal distribution) + + +# JMH version: 1.37 +# VM version: JDK 17.0.2, OpenJDK 64-Bit Server VM, 17.0.2+8 +# VM invoker: D:\InstalledPrograms\Adoptium\jdk-17.0.2.8-hotspot\bin\java.exe +# VM options: +# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) +# Warmup: 5 iterations, 10 s each +# Measurement: 5 iterations, 10 s each +# Timeout: 10 min per iteration +# Threads: 1 thread, will synchronize iterations +# Benchmark mode: Average time, time/op +# Benchmark: org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.wrappedThreadLocalRandom + +# Run progress: 42.86% complete, ETA 00:06:42 +# Fork: 1 of 1 +# Warmup Iteration 1: 11.577 ns/op +# Warmup Iteration 2: 11.821 ns/op +# Warmup Iteration 3: 10.427 ns/op +# Warmup Iteration 4: 10.372 ns/op +# Warmup Iteration 5: 10.384 ns/op +Iteration 1: 10.370 ns/op +Iteration 2: 10.444 ns/op +Iteration 3: 10.395 ns/op +Iteration 4: 10.380 ns/op +Iteration 5: 10.368 ns/op + + +Result "org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.wrappedThreadLocalRandom": + 10.392 ±(99.9%) 0.121 ns/op [Average] + (min, avg, max) = (10.368, 10.392, 10.444), stdev = 0.031 + CI (99.9%): [10.271, 10.512] (assumes normal distribution) + + +# JMH version: 1.37 +# VM version: JDK 17.0.2, OpenJDK 64-Bit Server VM, 17.0.2+8 +# VM invoker: D:\InstalledPrograms\Adoptium\jdk-17.0.2.8-hotspot\bin\java.exe +# VM options: +# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) +# Warmup: 5 iterations, 10 s each +# Measurement: 5 iterations, 10 s each +# Timeout: 10 min per iteration +# Threads: 1 thread, will synchronize iterations +# Benchmark mode: Average time, time/op +# Benchmark: org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.zigguratRandom + +# Run progress: 57.14% complete, ETA 00:05:01 +# Fork: 1 of 1 +# Warmup Iteration 1: 19.023 ns/op +# Warmup Iteration 2: 19.227 ns/op +# Warmup Iteration 3: 17.438 ns/op +# Warmup Iteration 4: 17.355 ns/op +# Warmup Iteration 5: 17.330 ns/op +Iteration 1: 17.305 ns/op +Iteration 2: 17.405 ns/op +Iteration 3: 17.501 ns/op +Iteration 4: 17.331 ns/op +Iteration 5: 17.422 ns/op + + +Result "org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.zigguratRandom": + 17.393 ±(99.9%) 0.300 ns/op [Average] + (min, avg, max) = (17.305, 17.393, 17.501), stdev = 0.078 + CI (99.9%): [17.092, 17.693] (assumes normal distribution) + + +# JMH version: 1.37 +# VM version: JDK 17.0.2, OpenJDK 64-Bit Server VM, 17.0.2+8 +# VM invoker: D:\InstalledPrograms\Adoptium\jdk-17.0.2.8-hotspot\bin\java.exe +# VM options: +# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) +# Warmup: 5 iterations, 10 s each +# Measurement: 5 iterations, 10 s each +# Timeout: 10 min per iteration +# Threads: 1 thread, will synchronize iterations +# Benchmark mode: Average time, time/op +# Benchmark: org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.zigguratSplittableRandom + +# Run progress: 71.43% complete, ETA 00:03:21 +# Fork: 1 of 1 +# Warmup Iteration 1: 10.731 ns/op +# Warmup Iteration 2: 9.703 ns/op +# Warmup Iteration 3: 10.069 ns/op +# Warmup Iteration 4: 10.094 ns/op +# Warmup Iteration 5: 10.067 ns/op +Iteration 1: 10.073 ns/op +Iteration 2: 10.073 ns/op +Iteration 3: 10.153 ns/op +Iteration 4: 10.076 ns/op +Iteration 5: 10.069 ns/op + + +Result "org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.zigguratSplittableRandom": + 10.089 ±(99.9%) 0.139 ns/op [Average] + (min, avg, max) = (10.069, 10.089, 10.153), stdev = 0.036 + CI (99.9%): [9.950, 10.228] (assumes normal distribution) + + +# JMH version: 1.37 +# VM version: JDK 17.0.2, OpenJDK 64-Bit Server VM, 17.0.2+8 +# VM invoker: D:\InstalledPrograms\Adoptium\jdk-17.0.2.8-hotspot\bin\java.exe +# VM options: +# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) +# Warmup: 5 iterations, 10 s each +# Measurement: 5 iterations, 10 s each +# Timeout: 10 min per iteration +# Threads: 1 thread, will synchronize iterations +# Benchmark mode: Average time, time/op +# Benchmark: org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.zigguratThreadLocalRandom + +# Run progress: 85.71% complete, ETA 00:01:40 +# Fork: 1 of 1 +# Warmup Iteration 1: 11.098 ns/op +# Warmup Iteration 2: 10.803 ns/op +# Warmup Iteration 3: 10.905 ns/op +# Warmup Iteration 4: 10.892 ns/op +# Warmup Iteration 5: 10.894 ns/op +Iteration 1: 10.905 ns/op +Iteration 2: 10.912 ns/op +Iteration 3: 10.899 ns/op +Iteration 4: 10.895 ns/op +Iteration 5: 10.892 ns/op + + +Result "org.cicirello.experiments.ziggurat.TimeZigguratVersusJavaBuiltin.zigguratThreadLocalRandom": + 10.901 ±(99.9%) 0.032 ns/op [Average] + (min, avg, max) = (10.892, 10.901, 10.912), stdev = 0.008 + CI (99.9%): [10.869, 10.932] (assumes normal distribution) + + +# Run complete. Total time: 00:11:44 + +REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on +why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial +experiments, perform baseline and negative tests that provide experimental control, make sure +the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts. +Do not assume the numbers tell you what you want them to tell. + +NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise +extra caution when trusting the results, look into the generated code to check the benchmark still +works, and factor in a small probability of new VM bugs. Additionally, while comparisons between +different JVMs are already problematic, the performance difference caused by different Blackhole +modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons. + +Benchmark Mode Cnt Score Error Units +TimeZigguratVersusJavaBuiltin.builtinRandom avgt 5 103.037 ± 0.951 ns/op +TimeZigguratVersusJavaBuiltin.builtinSplittableRandom avgt 5 8.927 ± 0.026 ns/op +TimeZigguratVersusJavaBuiltin.builtinThreadLocalRandom avgt 5 88.774 ± 2.096 ns/op +TimeZigguratVersusJavaBuiltin.wrappedThreadLocalRandom avgt 5 10.392 ± 0.121 ns/op +TimeZigguratVersusJavaBuiltin.zigguratRandom avgt 5 17.393 ± 0.300 ns/op +TimeZigguratVersusJavaBuiltin.zigguratSplittableRandom avgt 5 10.089 ± 0.139 ns/op +TimeZigguratVersusJavaBuiltin.zigguratThreadLocalRandom avgt 5 10.901 ± 0.032 ns/op diff --git a/experiment/timing17/src/main/java/org/cicirello/experiments/ziggurat/TimeZigguratVersusJavaBuiltin.java b/experiment/timing17/src/main/java/org/cicirello/experiments/ziggurat/TimeZigguratVersusJavaBuiltin.java new file mode 100644 index 0000000..e1b932e --- /dev/null +++ b/experiment/timing17/src/main/java/org/cicirello/experiments/ziggurat/TimeZigguratVersusJavaBuiltin.java @@ -0,0 +1,174 @@ +/* + * Experiment comparing org.cicirello.ziggurat versus Java builtin implementation. + * Copyright (C) 2023-2024 Vincent A. Cicirello + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.cicirello.experiments.ziggurat; + +import java.io.IOException; +import java.util.Random; +import java.util.SplittableRandom; +import java.util.concurrent.ThreadLocalRandom; +import java.util.concurrent.TimeUnit; +import java.util.random.RandomGenerator; +import org.cicirello.math.rand.ZigguratGaussian; +import org.openjdk.jmh.annotations.Benchmark; +import org.openjdk.jmh.annotations.BenchmarkMode; +import org.openjdk.jmh.annotations.Fork; +import org.openjdk.jmh.annotations.Mode; +import org.openjdk.jmh.annotations.OutputTimeUnit; + +/** + * Compares Java's builtin Gaussian implementation with this Ziggurat impementation. Beginning with + * Java 17, the Java API's default Gaussian implementation is based on the Ziggurat algorithm + * (except for the Random class, which still uses the Polar Method). Thus, for Java 17+, it doesn't + * seem like there should still be any benefit from my implementation of the Ziggurat algorithm. + * + *

The aim of this program is to empirically verify that Java's builtin Gaussian method is at + * least as good as my implementation for Java 17+. If you run this on earlier versions of Java, the + * Java builtin implementation should be slower. But on newer, it will probably be faster. + * + * @author Vincent A. Cicirello, https://www.cicirello.org/ + */ +public class TimeZigguratVersusJavaBuiltin { + + private static final Random random = new Random(42); + private static final SplittableRandom splittable = new SplittableRandom(42); + + private static final RandomGenerator wrappedThreadLocalRandom = + new RandomGenerator() { + @Override + public long nextLong() { + return ThreadLocalRandom.current().nextLong(); + } + }; + + /** + * Benchmark the Java API's ThreadLocalRandom class's builtin implementation of nextGaussian. + * + * @return the next Gaussian into JMH's blackhole + */ + @Benchmark + @Fork(value = 1) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + @BenchmarkMode(Mode.AverageTime) + public double builtinThreadLocalRandom() { + return ThreadLocalRandom.current().nextGaussian(); + } + + /** + * Benchmark the Java API's SplittableRandom class's builtin implementation of nextGaussian. + * + * @return the next Gaussian into JMH's blackhole + */ + @Benchmark + @Fork(value = 1) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + @BenchmarkMode(Mode.AverageTime) + public double builtinSplittableRandom() { + return splittable.nextGaussian(); + } + + /** + * Benchmark the Java API's Random class's builtin implementation of nextGaussian. + * + * @return the next Gaussian into JMH's blackhole + */ + @Benchmark + @Fork(value = 1) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + @BenchmarkMode(Mode.AverageTime) + public double builtinRandom() { + return random.nextGaussian(); + } + + /** + * Benchmark my implementation of ZigguratGaussian using ThreadLocalRandom. + * + * @return the next Gaussian into JMH's blackhole + */ + @Benchmark + @Fork(value = 1) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + @BenchmarkMode(Mode.AverageTime) + public double zigguratThreadLocalRandom() { + return ZigguratGaussian.nextGaussian(); + } + + /** + * Benchmark my implementation of ZigguratGaussian using SplittableRandom. + * + * @return the next Gaussian into JMH's blackhole + */ + @Benchmark + @Fork(value = 1) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + @BenchmarkMode(Mode.AverageTime) + public double zigguratSplittableRandom() { + return ZigguratGaussian.nextGaussian(splittable); + } + + /** + * Benchmark my implementation of ZigguratGaussian using Random. + * + * @return the next Gaussian into JMH's blackhole + */ + @Benchmark + @Fork(value = 1) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + @BenchmarkMode(Mode.AverageTime) + public double zigguratRandom() { + return ZigguratGaussian.nextGaussian(random); + } + + /** + * Coerce Java to use the RandomGenerator interface's implementation of the ziggurat algorithm + * with the ThreadLocalRandom class. + * + * @return the next Gaussian into JMH's blackhole + */ + @Benchmark + @Fork(value = 1) + @OutputTimeUnit(TimeUnit.NANOSECONDS) + @BenchmarkMode(Mode.AverageTime) + public double wrappedThreadLocalRandom() { + return wrappedThreadLocalRandom.nextGaussian(); + } + + /** + * Runs the experiments. + * + * @param args Ignored, no command line arguments. + */ + public static void main(String[] args) throws IOException { + printCopyrightAndLicense(); + org.openjdk.jmh.Main.main(args); + } + + /** Prints copyright and license notices. */ + private static void printCopyrightAndLicense() { + System.out.println(); + System.out.println( + "Experiment comparing org.cicirello.ziggurat versus Java builtin implementation."); + System.out.println("Copyright (C) 2023-2024 Vincent A. Cicirello"); + System.out.println("This program comes with ABSOLUTELY NO WARRANTY. This is free"); + System.out.println("software, and you are welcome to redistribute it under certain"); + System.out.println("conditions. See the GNU General Public License for more"); + System.out.println("details: https://www.gnu.org/licenses/gpl-3.0.html"); + System.out.println(); + } +}