Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize ConcurrentLru read throughput (volatile) #643

Closed
wants to merge 5 commits into from

Conversation

bitfaster
Copy link
Owner

@bitfaster bitfaster commented Nov 19, 2024

Improve read throughput by ~150% by avoiding volatile writes (store-load memory barrier).

Before

Results_Read_500_base

After

Results_Read_500

TODO: investigate read latency/code size. I would assume impact is negligible and this is a good tradeoff.

@bitfaster bitfaster changed the title Optimize ConcurrentLru reads Optimize ConcurrentLru read throughput Nov 19, 2024
@coveralls
Copy link

coveralls commented Nov 19, 2024

Coverage Status

coverage: 99.218% (+0.07%) from 99.149%
when pulling 3f8eef4 on users/alexpeck/fastmark
into aeae236 on main.

@bitfaster
Copy link
Owner Author

Method Runtime Mean StdDev Ratio Code Size
ConcurrentDictionary .NET 6.0 7.549 ns 0.0378 ns 1.00 NA
FastConcurrentLru .NET 6.0 9.893 ns 0.1486 ns 1.31 7,124 B
ConcurrentLru .NET 6.0 15.437 ns 0.0481 ns 2.04 7,371 B
AtomicFastLru .NET 6.0 27.798 ns 0.1106 ns 3.68 NA
FastConcurrentTLru .NET 6.0 11.911 ns 0.0872 ns 1.58 7,560 B
FastConcLruAfterAccess .NET 6.0 12.419 ns 0.0656 ns 1.65 6,831 B
FastConcLruAfter .NET 6.0 14.761 ns 0.1330 ns 1.96 8,167 B
ConcurrentTLru .NET 6.0 21.596 ns 0.2259 ns 2.86 7,836 B
ConcurrentLfu .NET 6.0 29.011 ns 0.4737 ns 3.84 NA
ClassicLru .NET 6.0 44.402 ns 0.8194 ns 5.88 NA
RuntimeMemoryCacheGet .NET 6.0 116.210 ns 1.2554 ns 15.39 89 B
ExtensionsMemoryCacheGet .NET 6.0 49.176 ns 0.6670 ns 6.51 119 B
ConcurrentDictionary .NET Framework 4.8 13.933 ns 0.1054 ns 1.00 4,127 B
FastConcurrentLru .NET Framework 4.8 16.670 ns 0.1470 ns 1.20 27,400 B
ConcurrentLru .NET Framework 4.8 20.384 ns 0.0891 ns 1.46 27,704 B
AtomicFastLru .NET Framework 4.8 37.778 ns 0.1741 ns 2.71 358 B
FastConcurrentTLru .NET Framework 4.8 28.804 ns 0.2912 ns 2.07 27,584 B
FastConcLruAfterAccess .NET Framework 4.8 30.078 ns 0.1984 ns 2.16 358 B
FastConcLruAfter .NET Framework 4.8 33.320 ns 0.6004 ns 2.39 358 B
ConcurrentTLru .NET Framework 4.8 33.120 ns 0.1406 ns 2.38 27,936 B
ConcurrentLfu .NET Framework 4.8 52.242 ns 0.6760 ns 3.75 NA
ClassicLru .NET Framework 4.8 52.825 ns 0.2602 ns 3.79 NA
RuntimeMemoryCacheGet .NET Framework 4.8 309.169 ns 3.1850 ns 22.19 79 B
ExtensionsMemoryCacheGet .NET Framework 4.8 95.930 ns 0.8040 ns 6.89 129 B

@bitfaster
Copy link
Owner Author

This adds 4 instructions with 1 jump:

image

@bitfaster bitfaster changed the title Optimize ConcurrentLru read throughput Optimize ConcurrentLru read throughput (volatile) Nov 20, 2024
@bitfaster bitfaster closed this Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants