Skip to content

Releases: fryguybob/ghc-stm-benchmarks

PPoPP2019 Benchmarks

23 Nov 23:33
@fryguybob fryguybob
PPoPP2019-v1
ab44bfc
Compare
Choose a tag to compare
PPoPP2019 Benchmarks Latest
Latest

PPoPP 2019 Hybrid TM, Mutable Fields GHC Benchmarks

All the resources needed to build and run our benchmarks are
included here.

Installing:

This has been tested on a fresh OS install (Ubuntu 18.04.1 LTS) on a machine
that supports Intel TSX.

$ uname -a
Linux d14005 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.1 LTS
Release:        18.04
Codename:       bionic

Prerequisites

We need gcc, g++, make, gmp, and ncurses:

$ sudo apt install gcc g++ make libgmp-dev libncurses-dev

Make an environment variable GHC_COMPILERS that has a path to where the
compilers will be installed. We used $HOME/ghc-8 as our path:

$ mkdir $HOME/ghc-8
$ export GHC_COMPILERS=$HOME/ghc-8

Installing the Binary Distribution:

Unpack and configure the binary distribution for the hybrid compiler:

$ cd $GHC_COMPILERS
... get the binary distribution file for the hybrid compiler
$ tar -xvf ghc-8.0.2-x86_64-mutable-fields-hybrid.tar.xz
$ cd ghc-hybrid
$ ./configure --prefix $GHC_COMPILERS/bin-hybrid
$ make install

Repeat for the compiler with the fine-grain based TM runtime:

$ cd $GHC_COMPILERS
... get the binary distribution file for the fine-grain compiler
$ tar -xvf ghc-8.0.2-x86_64-mutable-fields-fine.tar.xz
$ cd ghc-fine
$ ./configure --prefix $GHC_COMPILERS/bin-fine
$ make install

GHC should now be installed to $GHC_COMPILERS/bin-hybrid/bin/ghc
and $GHC_COMPILERS/bin-fine/bin/ghc.

Building Compiler from Source:

The GHC sources are available on GitHub:

They are much easier to build using the source distribution packaging.

You can skip this step, but you will not get the early subscription
TM version for comparison to late subscription.

We will build the compiler from source using the binary distribution's hybrid
GHC. It should be buildable with any GHC close to version 8.0.2, just point to
the desired compiler in the --with-ghc configure field.

Configure and build the hybrid TM compiler (GHC takes a long tome to build):

$ cd $GHC_COMPILERS
... get the source distribution file for the hybrid compiler
$ tar -xvf ghc-8.0.2-mutable-fields-hybrid-src.tar.xz
$ cd src-hybrid
$ ./configure --prefix $GHC_COMPILERS/build-hybrid --with-ghc=$GHC_COMPILERS/bin-hybrid/bin/ghc
$ make
$ make install

Configure and build the hybrid-early TM compiler (following these steps it will only
have to rebuild the rts):

$ cd $GHC_COMPILERS/src-hybrid
$ cd src-hybrid
$ sed -i 's/#define HTM_LATE_LOCK_SUBSCRIPTION/\/\/#define HTM_LATE_LOCK_SUBSCRIPTION/' rts/STM.c
$ ./configure --prefix $GHC_COMPILERS/build-hybrid-early --with-ghc=$GHC_COMPILERS/bin-hybrid/bin/ghc
$ make
$ make install

Configure and build the fine TM compiler (GHC takes a long tome to build):

$ cd $GHC_COMPILERS
... get the source distribution file for the fine-grain compiler
$ tar -xvf ghc-8.0.2-mutable-fields-fine-src.tar.xz
$ cd src-hybrid
$ ./configure --prefix $GHC_COMPILERS/build-fine --with-ghc=$GHC_COMPILERS/bin-hybrid/bin/ghc
$ make
$ make install

Benchmarks

Benchmarks are packaged so that only our GHC is needed for building. No cabal
is needed and required packages are all included with minimal modifications to
remove irrelevant dependencies.

Picking the compilers to use

Use links to set the compilers to use as the hybrid, hybrid-early, and fine
compilers.

$ cd $GHC_COMPILERS
$ ln -s build-hybrid hybrid
$ ln -s build-hybrid-early hybrid-early
$ ln -s build-fine fine

Building the Benchmarks

Get and build the benchmarks (start in the directory of your
choice, we will use $HOME):

$ cd $HOME
... get the benchmark source distribution
$ tar -xvf PPoPP2019-mutable-fields-benchmarks.tar.xz
$ cd PPoPP2019
$ ./build.sh

This should build 24 benchmark binaries in the bin child directory:

$ ls bin
HAMT-MUT-fine              HAMT-TVAR-hybrid-early       RBTREE-TVAR-hybrid
HAMT-MUT-hybrid            RBTREE-MUT-fine              RBTREE-TVAR-hybrid-early
HAMT-MUT-hybrid-early      RBTREE-MUT-hybrid            TREAP-MUT-fine
HAMT-TSTRUCT-fine          RBTREE-MUT-hybrid-early      TREAP-MUT-hybrid
HAMT-TSTRUCT-hybrid        RBTREE-TSTRUCT-fine          TREAP-MUT-hybrid-early
HAMT-TSTRUCT-hybrid-early  RBTREE-TSTRUCT-hybrid        TREAP-TVAR-fine
HAMT-TVAR-fine             RBTREE-TSTRUCT-hybrid-early  TREAP-TVAR-hybrid
HAMT-TVAR-hybrid           RBTREE-TVAR-fine             TREAP-TVAR-hybrid-early

Running the Benchmarks

The results for each figure are run and plotted with their own script.
For figures 1a, 1b, and 1c there are a total of 24*threads individual
runs of around 1.3 seconds each. On our 72 thread machine that comes to
around 40 minutes for the first three figures.

$ ./fig-1.sh

A plot of each figure will be in the output child directory:

$ ls output/
fig-1a.html  fig-1b.html  fig-1c.html

The figures can be viewed with a browser that supports javascript. These plots
should match the findings in the paper.

The second set of figures should be quicker:

$ ./fig-2a.sh
$ ./fig-2bc.sh

Producing:

$ ls output/
fig-1a.html  fig-1b.html  fig-1c.html  fig-2a.html  fig-2b.html  fig-2c.html

Troubleshooting

One difficult to predict aspect of these benchmarks is the layout of the machine.
We have guessed some common architectures, but they not match what you actually
have on your machine. The files topo-cores-sockets-threads-* in the benchmark
suite are a configuration file for the Haskell runtime system's +RTS -qa flag.
Each line is the processor to bind each GHC capability's thread affinity. We
attempt to fill the cores (without hyperthreads) on the first socket, then
subsequent sockets. Once all the cores are used we allocate hyperthreads. The
tool mktopo.hs can be used to produce different layouts.

Assets 7
Loading