Skip to content

Commit

Permalink
add style to images in conclusion section
Browse files Browse the repository at this point in the history
  • Loading branch information
Dominic Garguilo authored and Dominic Garguilo committed Apr 9, 2024
1 parent 0f07976 commit 3a48ead
Showing 1 changed file with 16 additions and 8 deletions.
24 changes: 16 additions & 8 deletions _posts/blog/2024-04-08-does-a-compactor-return-memory-to-OS.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ There could be a scenario where the amount of memory on a machine limits the num

If the compactor process does not return the memory to the OS, then we are stuck with only using the following number of compactor processes:
`(total memory)/(memory per compactor)`.
If the compactor processes return the memory to the OS, i.e. does not stay at the maximum 6G once they reach it, then we can oversubscribe the memory allowing us to run more compactor processes on that machine.
If the compactor processes return the memory to the OS, i.e. does not stay at the maximum 6G once they reach it, then we can oversubscribe the memory allowing us to run more compactor processes on that machine.

It should be noted that there is an inherent risk when oversubscribing processes that the user must be willing to accept if they choose to do oversubscribe. In this case, there is the possibility that all compactors run at the same time which might use all the memory on the machine. This could cause one or more of the compactor processes to be killed by the OOM killer.

## Test Setup

Expand Down Expand Up @@ -233,19 +235,25 @@ All Experiments Performed:
[![]({{site.baseurl}}/images/blog/202404_compactor_memory/java_21_shenandoah_x2_s256_UseShenandoah_VM.png){:width="500px"}]({{site.baseurl}}/images/blog/202404_compactor_memory/java_21_shenandoah_x2_s256_UseShenandoah_VM.png)

## Conclusion
All the garbage collectors tested (G1 GC, Shenandoah GC, and ZGC) and all the Java versions tested (11, 17, 21) will release memory that is no longer used by a compactor, back to the OS\*. Regardless of which GC is used, after an external compaction is done, most (but usually not all) memory is eventually released back to the OS and all memory is released back to the JVM. Although a comparable amount of memory is returned to the OS in each case, the amount of time it takes for the memory to be returned and the amount of memory used during a compaction depends on which garbage collector is used and which parameters are set for the java process.
All the garbage collectors tested (G1 GC, Shenandoah GC, and ZGC) and all the Java versions tested (11, 17, 21) will release memory that is no longer used by a compactor, back to the OS. Regardless of which GC is used, after an external compaction is done, most (but usually not all) memory is eventually released back to the OS and all memory is released back to the JVM. Although a comparable amount of memory is returned to the OS in each case, the amount of time it takes for the memory to be returned and the amount of memory used during a compaction depends on which garbage collector is used and which parameters are set for the java process.

[![]({{site.baseurl}}/images/blog/202404_compactor_memory/java_17_G1_x1_s256_periodic60000_OS.png){:width="800px"}]({{site.baseurl}}/images/blog/202404_compactor_memory/java_17_G1_x1_s256_periodic60000_OS.png)
The amount that is never released back to the OS appears to be minimal and may only be present with G1 GC and Shenandoah GC. In the following graph with Java 17 using G1 GC, we see that the baseline OS memory usage before any compactions are done is a bit less than 400mb. We see that after a compaction is done and the garbage collection runs, this baseline settles at about 500mb.

The amount that is never released back to the OS appears to be minimal and may only be present with G1 GC and Shenandoah GC. For example, from a Java 17 example using G1 GC pictured above, we see that the baseline OS memory usage before any compactions are done is a bit less than 400mb. We see that after a compaction is done and the garbage collection runs, this baseline settles at about 500mb.
<a class="p-3 border rounded d-block" href="{{ site.baseurl }}/images/blog/202404_compactor_memory/java_17_G1_x1_s256_periodic60000_OS.png">
<img src="{{ site.baseurl }}/images/blog/202404_compactor_memory/java_17_G1_x1_s256_periodic60000_OS.png" class="img-fluid rounded" alt="Graph showing memory usage from the OS perspective"/>
</a>

[![]({{site.baseurl}}/images/blog/202404_compactor_memory/java_17_G1_x1_s256_periodic60000_VM.png){:width="800px"}]({{site.baseurl}}/images/blog/202404_compactor_memory/java_17_G1_x1_s256_periodic60000_VM.png)
On the same test run, the JVM perspective (pictured in the graph below) shows that all memory is returned (memory usage drops back down to Xms=256mb after garbage collection occurs).

From the JVM perspective (pictured above), all memory is returned (memory usage drops back down to Xms=256mb after garbage collection occurs).
<a class="p-3 border rounded d-block" href="{{ site.baseurl }}/images/blog/202404_compactor_memory/java_17_G1_x1_s256_periodic60000_VM.png">
<img src="{{ site.baseurl }}/images/blog/202404_compactor_memory/java_17_G1_x1_s256_periodic60000_VM.png" class="img-fluid rounded" alt="Graph showing memory usage from the JVM perspective"/>
</a>

[![]({{site.baseurl}}/images/blog/202404_compactor_memory/java_21_ZGC_x2_s256_UseZGC_generational_uncommit_OS.png){:width="800px"}]({{site.baseurl}}/images/blog/202404_compactor_memory/java_21_ZGC_x2_s256_UseZGC_generational_uncommit_OS.png)
The roughly 100mb of unreturned memory is also present with Shenandoah GC in Java 17 and Java 21 but does not appear to be present with Java 11. With ZGC, however, we see several runs where nearly all the memory used during a compaction is returned to the OS (the graph below was from a run using ZGC with Java 21). These findings regarding the unreturned memory may or may not be significant. They may also be the result of variance between runs. More testing would need to be done to confirm or deny these claims.

The roughly 100mb of unreturned memory is also present with Shenandoah GC in Java 17 and Java 21 but does not appear to be present with Java 11. With ZGC, however, we see several runs where nearly all the memory used during a compaction is returned to the OS (the example above was using ZGC with Java 21). These findings regarding the unreturned memory may or may not be significant. They may also be the result of variance between runs. More testing would need to be done to confirm or deny these claims.
<a class="p-3 border rounded d-block" href="{{ site.baseurl }}/images/blog/202404_compactor_memory/java_21_ZGC_x2_s256_UseZGC_generational_uncommit_OS.png">
<img src="{{ site.baseurl }}/images/blog/202404_compactor_memory/java_21_ZGC_x2_s256_UseZGC_generational_uncommit_OS.png" class="img-fluid rounded" alt="Graph showing memory usage from the OS perspective"/>
</a>

Another interesting finding was that the processes use more memory when more is allocated. These results were obtained from initiating a compaction of 700mb of data (see experiment.jsh script). For example, setting 2gb versus 1gb of max heap for the compactor process results in a higher peak memory usage. During a compaction, when only allocated 1gb of heap space, the max heap space is not completely utilized. When allocated 2gb, compactions exceed 1gb of heap space used. It appears that G1 GC and ZGC use the least amount of heap space during a compaction (maxing out around 1.5gb and when using ZGC with ZGeneration in Java 21, this maxes out around 1.7gb). Shenandoah GC appears to use the most heap space during a compaction with a max heap space around 1.9gb (for Java 11, 17, and 21). However, these differences might be due to differences between outside factors during runs and more testing may need to be done to confirm or deny these claims.

Expand Down

0 comments on commit 3a48ead

Please sign in to comment.