Add concurrency to Exponential Histogram

[lenin-jaganathan]

Guard the entire write operation externally for exponential histogram.

TODO:

• Add concurrency tests (I will try to come to this next week)

Fixes #5740

[lenin-jaganathan]

I initially thought that using a ReadWriteLock would provide a better performance than normal synchronization. But, after running the benchmarks, that is not the case. The overhead of ReadLock seems to be higher compared to the simpler math operations (like incrementing values) that are performed inside the lock. Considering the code complexity it brings it seems that not a good option. I am sharing benchmark results from my local PC (M1 10 cores),

Existing Implementation

1 - thread
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  96.115          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  85.981          ns/op

2 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  80.613          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  60.498          ns/op

4 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  211.090          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  179.947          ns/op

8 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  468.598          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  281.201          ns/op

Full Synchronization (the current state of PR)

1 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  96.945          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  85.520          ns/op

2 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  424.293          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  324.563          ns/op

4 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  922.629          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  853.906          ns/op

8 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  1832.407          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  1662.399          ns/op

Full Synchronization with AtomicLongArray (we have been using a AtomicLongArray until now)

1 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  94.078          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  84.147          ns/op

2 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  428.927          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  336.852          ns/op

4 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  865.197          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  817.838          ns/op

8 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  1708.773          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  1622.357          ns/op

ReadWrite Locks (Implementation in the first commit of this PR)

1 - thread
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  107.122          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2   90.638          ns/op

2 - threads
Benchmark                                                     
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  481.020          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  261.227          ns/op

4 - threads
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  946.514          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  888.778          ns/op

8 - threads                                                
CompareOTLPHistograms.oltpCumulativeExponentialHistogramTimer  avgt    2  4375.336          ns/op
CompareOTLPHistograms.oltpDeltaExponentialHistogramTimer       avgt    2  4036.524          ns/op

I will leave the ReadWriteLock implementation in the commit history so if someone has a look at it, then can run the tests as well (or correct me if I am doing something wrong there). I add a second commit to drop those changes and add synchronization to the writes.

[lenin-jaganathan]

JCStressTest Concurrency result

Before fix,

                                                                                                RESULT      SAMPLES     FREQ      EXPECT  DESCRIPTION
                                                 class java.lang.ArrayIndexOutOfBoundsException, 20, 0          747   <0.01%   Forbidden  Exception in recording thread
                                                 class java.lang.ArrayIndexOutOfBoundsException, 20, 1          164   <0.01%   Forbidden  Exception in recording thread
  class java.lang.ArrayIndexOutOfBoundsException, class java.lang.ArrayIndexOutOfBoundsException, null           21   <0.01%   Forbidden  Exception in both reading and writing threads
                                            null, class java.lang.ArrayIndexOutOfBoundsException, null       21,671   <0.01%   Forbidden  Exception in reading thread
                                                                                           null, 20, 0   62,561,872    3.72%  Acceptable  Read before write
                                                                                           null, 20, 1  768,594,621   45.70%  Acceptable  Reading after single write
                                                                                            null, 5, 2  850,341,011   50.56%  Acceptable  Read after all writes
                                                                                           null, 20, 2      283,601    0.02%     Unknown  
                                                                                            null, 5, 1        7,350   <0.01%     Unknown  
                                                null, class java.lang.NegativeArraySizeException, null          345   <0.01%     Unknown  
      class java.lang.ArrayIndexOutOfBoundsException, class java.lang.NegativeArraySizeException, null          240   <0.01%     Unknown  
                                                                                           null, -6, 1            2   <0.01%     Unknown  
                                                                                           null, -6, 2           85   <0.01%     Unknown  

With current changes,

                                                                                                RESULT        SAMPLES     FREQ      EXPECT  DESCRIPTION
                                                 class java.lang.ArrayIndexOutOfBoundsException, 20, 0              0    0.00%   Forbidden  Exception in recording thread
                                                 class java.lang.ArrayIndexOutOfBoundsException, 20, 1              0    0.00%   Forbidden  Exception in recording thread
  class java.lang.ArrayIndexOutOfBoundsException, class java.lang.ArrayIndexOutOfBoundsException, null              0    0.00%   Forbidden  Exception in both reading and writing threads
                                                                                           null, 20, 0    348,591,988   19.38%  Acceptable  Read before write
                                                                                           null, 20, 1    432,725,443   24.05%  Acceptable  Reading after single write
                                                                                            null, 5, 2  1,017,765,093   56.57%  Acceptable  Read after all writes
                                            null, class java.lang.ArrayIndexOutOfBoundsException, null              0    0.00%   Forbidden  Exception in reading thread

[shakuzen]

Thanks for all the work and testing on this. Looks good to me. I'll merge now so we can get some snapshots published for easier testing, but if @jonatan-ivanov has any feedback, we can address it post-merge.