Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop Celo L1 at a specific block #2322

Closed
wants to merge 21 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/geth/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ var (
utils.USBFlag,
// utils.SmartCardDaemonPathFlag,
utils.OverrideHForkFlag,
utils.L2MigrationBlockFlag,
utils.TxPoolLocalsFlag,
utils.TxPoolNoLocalsFlag,
utils.TxPoolJournalFlag,
Expand Down
8 changes: 8 additions & 0 deletions cmd/utils/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,11 @@ var (
Usage: "Manually specify the hfork block, overriding the bundled setting",
}

L2MigrationBlockFlag = cli.Uint64Flag{
Name: "l2migrationblock",
Usage: "Block number at which to halt the network for Celo L2 migration. This is the first block of Celo as an L2, and one after the last block of Celo as an L1. If unset or set to 0, no halt will occur.",
}

BloomFilterSizeFlag = cli.Uint64Flag{
Name: "bloomfilter.size",
Usage: "Megabytes of memory allocated to bloom-filter for pruning",
Expand Down Expand Up @@ -1723,6 +1728,9 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) {
log.Debug("Sanitizing Go's GC trigger", "percent", int(gogc))
godebug.SetGCPercent(int(gogc))

if ctx.GlobalIsSet(L2MigrationBlockFlag.Name) {
cfg.L2MigrationBlock = new(big.Int).SetUint64(ctx.GlobalUint64(L2MigrationBlockFlag.Name))
}
if ctx.GlobalIsSet(SyncModeFlag.Name) {
cfg.SyncMode = *GlobalTextMarshaler(ctx, SyncModeFlag.Name).(*downloader.SyncMode)
}
Expand Down
6 changes: 6 additions & 0 deletions consensus/istanbul/backend/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -554,6 +554,12 @@ func (sb *Backend) Commit(proposal istanbul.Proposal, aggregatedSeal types.Istan
}
}
sb.onNewConsensusBlock(block, result.Receipts, result.Logs, result.State)

nextBlockNum := new(big.Int).Add(block.Number(), big.NewInt(1))
if sb.chain.Config().IsL2Migration(nextBlockNum) {
sb.logger.Info("The next block is the L2 migration block, stopping announce protocol and closing istanbul backend", "currentBlock", block.NumberU64(), "hash", block.Hash(), "nextBlock", nextBlockNum)
sb.StopAnnouncing()
}
return nil
}

Expand Down
20 changes: 19 additions & 1 deletion core/blockchain.go
Original file line number Diff line number Diff line change
Expand Up @@ -811,6 +811,14 @@ func (bc *BlockChain) ExportN(w io.Writer, first uint64, last uint64) error {
//
// Note, this function assumes that the `mu` mutex is held!
func (bc *BlockChain) writeHeadBlock(block *types.Block) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's also core.HeaderChain.writeHeaders where rawdb.WriteCanonicalHash is called. I'm just wondering if this could cause any issues. For example we may sync a header later than the stop block.

Copy link
Contributor Author

@alecps alecps Sep 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that by calling bc.StopInsert() we prevent the HeaderChain code from continuing to update the head header. writeHeaders checks something called procInterrupt(), which wraps a call to insertStopped(), which is injected when NewHeaderChain() is called. When we call StopInsert(), it makes procInterrupt() return true in writeHeaders before it updates the head header references etc.

// Normally the check at the end of the function will pass first, but if a node is restarted
// with the same l2-migration-block configured after already reaching and stopping on l2-migration-block,
// this check will pass first and log an error.
if bc.Config().IsL2Migration(block.Number()) {
log.Error("Attempt to insert block number >= l2MigrationBlock, stopping block insertion", "block", block.NumberU64(), "hash", block.Hash())
bc.StopInsert()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should need to call StopInsert here since this check will effectively prevent any additional blocks being added.

Copy link
Contributor Author

@alecps alecps Sep 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think StopInsert() is actually preventing some threads from inserting blocks, as removing it below makes the test fail. Specifically, the writeHeaders function you asked about above checks something called procInterrupt(), which wraps a call to insertStopped(), which is injected when NewHeaderChain() is called. When we call StopInsert(), it makes procInterrupt() return true in writeHeaders before it updates the head header references etc.
This check was in place to fix an error that shows up via command line testing when a node is restarted with the same l2-migration-block number configured after already reaching it. I'll test whether removing this call to StopInsert re-introduces that error.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure why we need changes to writeHeadBlock at all. So I put a panic in for the IsL2Migration migration case to see when the check is relevant. The panic was triggered by the block fetcher.

It might be more elegant to discard the blocks when the block fetcher receives than rather then preventing the write here. Suggested commit: 90dbc6c

I still don't have the full picture, so let me know if I'm missing something.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@karlb that looks pretty neat to me, and the tests seem to work, so I'd be in favour of this approach.

Copy link
Contributor

@piersy piersy Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I've pushed a couple of updates to karlb/stop and removed basically everything except the extra flag, the rejection in the validate function and the e2e test. Tests seem to be working, so I think that would be the way to go now. - https://github.com/celo-org/celo-blockchain/pull/2330/files

return
}
// If the block is on a side chain or an unknown one, force other heads onto it too
updateHeads := rawdb.ReadCanonicalHash(bc.db, block.NumberU64()) != block.Hash()

Expand All @@ -837,6 +845,13 @@ func (bc *BlockChain) writeHeadBlock(block *types.Block) {
}
bc.currentBlock.Store(block)
headBlockGauge.Update(int64(block.NumberU64()))

nextBlockNum := new(big.Int).Add(block.Number(), big.NewInt(1))
if bc.Config().IsL2Migration(nextBlockNum) {
log.Info("The next block is the L2 migration block, stopping block insertion", "currentBlock", block.NumberU64(), "hash", block.Hash(), "nextBlock", nextBlockNum.Uint64())
bc.StopInsert()
return
}
Comment on lines +848 to +854
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this, the check up front should be sufficient. Sopping insertion isn't really important unless there is an attempt to insert the next block.

Copy link
Contributor Author

@alecps alecps Sep 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I remove this the long test actually starts failing. I was under the impression bc.StopInsert() was essential to call based on what I saw during testing. The function flips an atomic value that is checked at the beginning of numerous functions related to block insertion, including those in HeaderChain like writeHeaders which you asked about above.

}

// Genesis retrieves the chain's genesis block.
Expand Down Expand Up @@ -1085,7 +1100,7 @@ func (bc *BlockChain) Stop() {
triedb := bc.stateCache.TrieDB()
triedb.SaveCache(bc.cacheConfig.TrieCleanJournal)
}
log.Info("Blockchain stopped")
log.Info("Blockchain stopped", "number", bc.CurrentBlock().NumberU64(), "hash", bc.CurrentBlock().Hash())
}

// StopInsert interrupts all insertion methods, causing them to return
Expand Down Expand Up @@ -1870,6 +1885,9 @@ func (bc *BlockChain) insertChain(chain types.Blocks, verifySeals bool) (int, er
// If the chain is terminating, stop processing blocks
if bc.insertStopped() {
log.Debug("Abort during block processing")
if bc.Config().IsL2Migration(block.Number()) {
err = errInsertionInterrupted
}
break
}
// If the header is a banned one, straight out abort
Expand Down
23 changes: 23 additions & 0 deletions core/blockchain_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ import (
"github.com/celo-org/celo-blockchain/ethdb"
"github.com/celo-org/celo-blockchain/params"
"github.com/celo-org/celo-blockchain/trie"
"github.com/stretchr/testify/require"
)

// So we can deterministically seed different blockchains
Expand Down Expand Up @@ -205,6 +206,28 @@ func TestLastBlock(t *testing.T) {
}
}

func TestNoInsertPastL2MigrationBlock(t *testing.T) {
_, blockchain, err := newCanonical(mockEngine.NewFaker(), 0, true)
if err != nil {
t.Fatalf("failed to create pristine chain: %v", err)
}
defer blockchain.Stop()

blockchain.chainConfig = blockchain.chainConfig.DeepCopy()
migrationBlock := 2
blockchain.chainConfig.L2MigrationBlock = big.NewInt(int64(migrationBlock))

blocks := makeBlockChain(blockchain.CurrentBlock(), 100000, mockEngine.NewFullFaker(), blockchain.db, 0)
failedBlock, err := blockchain.InsertChain(blocks)
require.EqualError(t, err, errInsertionInterrupted.Error())
// Compare with migrationBlock-1 because failedBlock is the index of the failed block in the blocks[] array, not in the actual blockchain.
require.EqualValues(t, migrationBlock-1, failedBlock)
// Only the first block in blocks[] should be inserted
if blocks[migrationBlock-2].Hash() != rawdb.ReadHeadBlockHash(blockchain.db) {
t.Fatalf("Write/Get HeadBlockHash failed")
}
}

// Tests that given a starting canonical chain of a given size, it can be extended
// with various length chains.
func TestExtendCanonicalHeaders(t *testing.T) { testExtendCanonical(t, false) }
Expand Down
4 changes: 4 additions & 0 deletions core/forkid/forkid.go
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,10 @@ func gatherForks(config *params.ChainConfig) []uint64 {
if !strings.HasSuffix(field.Name, "Block") {
continue
}
// Do not include L2MigrationBlock in forkid as doing so will prevent syncing with nodes that have not set the L2MigrationBlock flag
if field.Name == "L2MigrationBlock" {
continue
}
if field.Type != reflect.TypeOf(new(big.Int)) {
continue
}
Expand Down
4 changes: 2 additions & 2 deletions e2e_test/e2e_bench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ func BenchmarkNet100EmptyBlocks(b *testing.B) {
for i := 0; i < b.N; i++ {
ac := test.AccountConfig(n, 0)
gingerbreadBlock := common.Big0
gc, ec, err := test.BuildConfig(ac, gingerbreadBlock)
gc, ec, err := test.BuildConfig(ac, gingerbreadBlock, nil)
require.NoError(b, err)
network, shutdown, err := test.NewNetwork(ac, gc, ec)
require.NoError(b, err)
Expand All @@ -45,7 +45,7 @@ func BenchmarkNet1000Txs(b *testing.B) {

ac := test.AccountConfig(n, n)
gingerbreadBlock := common.Big0
gc, ec, err := test.BuildConfig(ac, gingerbreadBlock)
gc, ec, err := test.BuildConfig(ac, gingerbreadBlock, nil)
require.NoError(b, err)
accounts := test.Accounts(ac.DeveloperAccounts(), gc.ChainConfig())
network, shutdown, err := test.NewNetwork(ac, gc, ec)
Expand Down
Loading
Loading