[actpool] impl a store to cache actions on disk #4362
Conversation
envestcc
requested review from
CoderZhi,
dustinxie,
Liuhaai and
a team
as code owners
| r.Equal(act, acts[0]) | ||
| } | ||
|
|
||
| func BenchmarkDatabase(b *testing.B) { |
There was a problem hiding this comment.
benchmark: around 60x faster than kv
goos: darwin
goarch: arm64
pkg: github.com/iotexproject/iotex-core/actpool
BenchmarkDatabase/billy-put-10 10000 138064 ns/op 140654 B/op 1 allocs/op
BenchmarkDatabase/pebbledb-put-10 158 6843763 ns/op 24024 B/op 27 allocs/op
BenchmarkDatabase/boltdb-put-10 120 8863447 ns/op 541060 B/op 64 allocs/op
PASS
ok github.com/iotexproject/iotex-core/actpool 5.885s
actpool/blobstore.go
Outdated
| } | ||
|
|
||
| func (s *blobStore) drop() { | ||
| for { |
There was a problem hiding this comment.
just lock the s.lookup map, and delete all items?
There was a problem hiding this comment.
lock/unlock should be called only in public methods
There was a problem hiding this comment.
then define a public method? the current implementation is actually not correct, access to s.lookup should be protected with a lock (just like in other funcs) right?
There was a problem hiding this comment.
drop() is called by Put(), should not be called by outside
There was a problem hiding this comment.
OK I see, add a TODO or file an issue for a better drop policy, for now it is simply deleting all items, the newly added item could also be deleted
There was a problem hiding this comment.
as discuss, no need for, just delete the first item returned from evict
actpool/blobstore.go
Outdated
| func (s *blobStore) Get(hash hash.Hash256) (*action.SealedEnvelope, error) { | ||
| s.lock.RLock() | ||
| defer s.lock.RUnlock() | ||
| if s.store == nil { |
There was a problem hiding this comment.
add a Readiness to s, and use s.IsReady()
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #4362 +/- ##
==========================================
- Coverage 75.86% 75.65% -0.21%
==========================================
Files 361 364 +3
Lines 30116 30407 +291
==========================================
+ Hits 22846 23005 +159
- Misses 6134 6246 +112
- Partials 1136 1156 +20 ☔ View full report in Codecov by Sentry. |
|
|
||
| // TODO: implement a proper eviction policy | ||
| func (s *blobStore) evict() (hash.Hash256, bool) { | ||
| for h := range s.lookup { |
There was a problem hiding this comment.
add a TODO, to find the worst action according to some policy
|
Description
blobstore is a customized, high-performance datastore for transient actions, which is backed by billy (a super simple datastore, also the blobpool storage option chosen by Ethereum). It will be used for cache actions so that can load from local when restart, avoiding large load on the network.
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Test Configuration:
Checklist: