Add new getcfilters message for utreexo nodes

[alainjr10]

Add support for new getcfilters message called UtreexoCFilter
When a new block is added, a new UtreexoCFilter filter is created and persisted on the node.
This filter is created by serializing the contents of the roots at that height.
When a node receives a getcfilters message, the logic doesn't change much, it handles it almost identical to how it deals with basic filters.

[kcalvinalvin]

Still crashing for me on the latest commit.

[I] calvin@nixos ~/b/u/utreexod (sob-getcfilters-fix)> ./utreexod --signet --datadir=. --logdir=. --flatutreexoproofindex --utreexocfilters
2024-08-05 22:58:31.706 [INF] BTCD: Version 0.3.1-beta
2024-08-05 22:58:31.706 [INF] BTCD: Loading block database from 'signet/blocks_ffldb'
2024-08-05 22:58:31.728 [INF] BTCD: Block database loaded
2024-08-05 22:58:31.731 [INF] INDX: Utreexo C filter index enabled
2024-08-05 22:58:31.731 [INF] INDX: Flat Utreexo Proof index is enabled
2024-08-05 22:58:31.731 [INF] INDX: Initializing Utreexo state from 'signet/utreexostate_flat'
2024-08-05 22:58:31.777 [INF] INDX: Utreexo state loaded
2024-08-05 22:58:31.778 [INF] CHAN: Pre-alloacting for 251 MiB: 
2024-08-05 22:58:31.781 [INF] INDX: Catching up indexes from height -1 to 0
2024-08-05 22:58:31.781 [INF] BTCD: Gracefully shutting down the database...
2024-08-05 22:58:31.811 [INF] BTCD: Shutdown complete
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x6f1d73]
                                                                                       
goroutine 1 [running]:
github.com/utreexo/utreexod/database/ffldb.rollbackOnPanic(0xc0003dec30)
        /home/calvin/bitcoin-projects/utreexod/utreexod/database/ffldb/db.go:2221 +0x56 
panic({0x98a940?, 0xf389b0?})         
        /nix/store/zg65r8ys8y5865lcwmmybrq5gn30n1az-go-1.21.6/share/go/src/runtime/panic.go:914 +0x21f
github.com/utreexo/utreexod/blockchain.(*BlockChain).IsUtreexoViewActive(0x0)
        /home/calvin/bitcoin-projects/utreexod/utreexod/blockchain/utreexoviewpoint.go:849 +0x13
github.com/utreexo/utreexod/blockchain/indexers.(*UtreexoCFIndex).fetchUtreexoRoots(0xc00026a420, {0xbefd50, 0xc0003dec30}, 0xc0001e86b0?)
        /home/calvin/bitcoin-projects/utreexod/utreexod/blockchain/indexers/utreexocfindex.go:184 +0x32
github.com/utreexo/utreexod/blockchain/indexers.(*UtreexoCFIndex).ConnectBlock(0xbefd50?, {0xbefd50, 0xc0003dec30}, 0xc0001d8510, {0x1a?, 0x10?, 0xc000100c00?})
        /home/calvin/bitcoin-projects/utreexod/utreexod/blockchain/indexers/utreexocfindex.go:159 +0x5c
github.com/utreexo/utreexod/blockchain/indexers.dbIndexConnectBlock({0xbefd50, 0xc0003dec30}, {0xbee400, 0xc00026a420}, 0xc0001d8510, {0x0, 0x0, 0x0})
        /home/calvin/bitcoin-projects/utreexod/utreexod/blockchain/indexers/manager.go:123 +0x11a
github.com/utreexo/utreexod/blockchain/indexers.(*Manager).Init.func6({0xbefd50?, 0xc0003dec30?})
        /home/calvin/bitcoin-projects/utreexod/utreexod/blockchain/indexers/manager.go:509 +0x3b
github.com/utreexo/utreexod/database/ffldb.(*db).Update(0xc00011a200?, 0xc0000d86a0)
        /home/calvin/bitcoin-projects/utreexod/utreexod/database/ffldb/db.go:2278 +0x7e 
github.com/utreexo/utreexod/blockchain/indexers.(*Manager).Init(0xc0001fee10, 0xc0001b2580?, 0xc00011a200?)
        /home/calvin/bitcoin-projects/utreexod/utreexod/blockchain/indexers/manager.go:508 +0xdf7
github.com/utreexo/utreexod/blockchain.New(0xc0001e96e0)
        /home/calvin/bitcoin-projects/utreexod/utreexod/blockchain/chain.go:2525 +0x8c2 
main.newServer({0xc0002997d0, 0x1, 0x1}, {0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, {0xbed300, ...}, ...)
        /home/calvin/bitcoin-projects/utreexod/utreexod/server.go:3485 +0x1945     
main.btcdMain(0x0)                           
        /home/calvin/bitcoin-projects/utreexod/utreexod/utreexod.go:406 +0xea7
main.main()
        /home/calvin/bitcoin-projects/utreexod/utreexod/utreexod.go:592 +0x96

[kcalvinalvin]

The node is crashing when the indexes are being initialized and the genesis block is processed. It's happening because during initialization you aren't setting the chain for the index. The below code will fix that.

diff --git a/blockchain/indexers/utreexocfindex.go b/blockchain/indexers/utreexocfindex.go
index c0ac2910..d7db4bbc 100644
--- a/blockchain/indexers/utreexocfindex.go
+++ b/blockchain/indexers/utreexocfindex.go
@@ -72,7 +72,8 @@ func (idx *UtreexoCFIndex) NeedsInputs() bool {
 
 // Init initializes the utreexo cf index. This is part of the Indexer
 // interface.
-func (idx *UtreexoCFIndex) Init(_ *blockchain.BlockChain) error {
+func (idx *UtreexoCFIndex) Init(chain *blockchain.BlockChain) error {
+       idx.chain = chain
        return nil // Nothing to do.
 }

Though it still doesn't sync and there's an infinite loop happening somewhere...

[kcalvinalvin]

Could you make the node error out early if you give --noutreexo and --utreexocfilters together? You can look at the current un-allowed flag combos here:

utreexod/config.go

Lines 1217 to 1222 in f68f79d

	if cfg.WatchOnlyWallet && cfg.NoUtreexo {
	err := fmt.Errorf("%s: the --watchonlywallet requires the --noutreexo option off", funcName)
	fmt.Fprintln(os.Stderr, err)
	fmt.Fprintln(os.Stderr, usageMessage)
	return nil, nil, err
	}

[alainjr10]

Could you make the node error out early if you give --noutreexo and --utreexocfilters together? You can look at the current un-allowed flag combos here:

utreexod/config.go

Lines 1217 to 1222 in f68f79d

	if cfg.WatchOnlyWallet && cfg.NoUtreexo {
	err := fmt.Errorf("%s: the --watchonlywallet requires the --noutreexo option off", funcName)
	fmt.Fprintln(os.Stderr, err)
	fmt.Fprintln(os.Stderr, usageMessage)
	return nil, nil, err
	}

Sure. This will be done now.

[kcalvinalvin]

CI seems to be failing on the latest commit

[I] calvin@nixos ~/b/u/utreexod ((5cb8c305)) [1]> sh goclean.sh
+ go test -short -race -tags=rpctest -tags=bdkwallet ./...
?       github.com/utreexo/utreexod/bdkwallet/bdkgo     [no test files]
?       github.com/utreexo/utreexod/cmd/addblock        [no test files]
?       github.com/utreexo/utreexod/cmd/findcheckpoint  [no test files]
?       github.com/utreexo/utreexod/cmd/gencerts        [no test files]
?       github.com/utreexo/utreexod/cmd/utreexoctl      [no test files]
?       github.com/utreexo/utreexod/database/cmd/dbtool [no test files]
?       github.com/utreexo/utreexod/electrum    [no test files]
?       github.com/utreexo/utreexod/integration/rpctest [no test files]
?       github.com/utreexo/utreexod/limits      [no test files]
--- FAIL: TestHelp (0.00s)
    rpcserverhelp_test.go:16: RPC handler defined for method 'getutreexocfilterheader' without also specifying result types
    rpcserverhelp_test.go:16: RPC handler defined for method 'getutreexocfilter' without also specifying result types
    rpcserverhelp_test.go:34: Failed to generate one-line usage: "getutreexocfilter" is not registered
FAIL
FAIL    github.com/utreexo/utreexod     0.016s
ok      github.com/utreexo/utreexod/addrmgr     (cached)
ok      github.com/utreexo/utreexod/bdkwallet   (cached)
?       github.com/utreexo/utreexod/mining/cpuminer     [no test files]
?       github.com/utreexo/utreexod/netsync     [no test files]
?       github.com/utreexo/utreexod/rpcclient/examples/bitcoincorehttp  [no test files]
?       github.com/utreexo/utreexod/rpcclient/examples/bitcoincorehttpbulk      [no test files]
?       github.com/utreexo/utreexod/rpcclient/examples/btcdwebsockets   [no test files]
?       github.com/utreexo/utreexod/rpcclient/examples/btcwalletwebsockets      [no test files]
?       github.com/utreexo/utreexod/rpcclient/examples/customcommand    [no test files]
ok      github.com/utreexo/utreexod/blockchain  48.777s
ok      github.com/utreexo/utreexod/blockchain/fullblocktests   (cached)
ok      github.com/utreexo/utreexod/blockchain/indexers 29.978s
ok      github.com/utreexo/utreexod/blockchain/internal/sizehelper      (cached)
ok      github.com/utreexo/utreexod/blockchain/internal/utreexobackends (cached)
ok      github.com/utreexo/utreexod/btcjson     (cached)
ok      github.com/utreexo/utreexod/btcutil     (cached)
ok      github.com/utreexo/utreexod/btcutil/base58      (cached)
ok      github.com/utreexo/utreexod/btcutil/bech32      (cached)
ok      github.com/utreexo/utreexod/btcutil/bloom       (cached)
ok      github.com/utreexo/utreexod/btcutil/coinset     (cached)
ok      github.com/utreexo/utreexod/btcutil/gcs (cached)
ok      github.com/utreexo/utreexod/btcutil/gcs/builder (cached)
ok      github.com/utreexo/utreexod/btcutil/txsort      (cached)
ok      github.com/utreexo/utreexod/chaincfg    (cached)
ok      github.com/utreexo/utreexod/chaincfg/chainhash  (cached)
ok      github.com/utreexo/utreexod/connmgr     (cached)
ok      github.com/utreexo/utreexod/database    (cached)
ok      github.com/utreexo/utreexod/database/ffldb      (cached)
ok      github.com/utreexo/utreexod/database/internal/treap     (cached)
ok      github.com/utreexo/utreexod/integration (cached)
ok      github.com/utreexo/utreexod/mempool     (cached)
ok      github.com/utreexo/utreexod/mining      (cached)
ok      github.com/utreexo/utreexod/peer        (cached)
ok      github.com/utreexo/utreexod/rpcclient   (cached)
ok      github.com/utreexo/utreexod/txscript    (cached)
ok      github.com/utreexo/utreexod/wallet      (cached)
--- FAIL: TestServiceFlagStringer (0.00s)
    protocol_test.go:29: Running 12 tests
    protocol_test.go:33: String #11
         got: SFNodeNetwork|SFNodeNetworkLimited|SFNodeGetUTXO|SFNodeBloom|SFNodeWitness|SFNodeXthin|SFNodeBit5|SFNodeCF|SFNode2X|SFNodeUtreexo|SFNodeUtreexoCF|0xfefffa00 want: SFNodeNetwork|SFNodeNetworkLimited|SFNodeGetUTXO|SFNodeBloom|SFNodeWitness|SFNodeXthin|SFNodeBit5|SFNodeCF|SFNode2X|SFNodeUtreexo|0xfefffb00
FAIL
FAIL    github.com/utreexo/utreexod/wire        9.423s
FAIL
[I] calvin@nixos ~/b/u/utreexod ((5cb8c305)) [1]>