added a way to monitor and sanity-test localnet (#106)

* added a way to monitor and sanity-test localnet

* added a way to display a refreshing table of localnet data
* added a way to test localnet in CI, dumping the data and testing against that.  removed network test in favor of this.

* updated readme

* updated README

* upgrade go

* PR feedback

* pr feedback

* remove underscore

* Update e2e/monitor/main.go

Co-authored-by: Joshua Sing <joshua@bloq.com>

* Update e2e/monitor/main.go

Co-authored-by: Joshua Sing <joshua@bloq.com>

* Update e2e/monitor/main.go

Co-authored-by: Joshua Sing <joshua@bloq.com>

* Update e2e/monitor/main.go

Co-authored-by: Joshua Sing <joshua@bloq.com>

* pr feedback

* pr feedback

* pr feedback

* added some better docs

* fix spelling mistake

---------

Co-authored-by: Joshua Sing <joshua@bloq.com>

.github/workflows/network-test.yml → .github/workflows/localnet-test.yml

@@ -3,7 +3,7 @@
 # which can be found in the LICENSE file.
 
 # GitHub Actions workflow to run e2e network tests.
-name: "E2E Network Test"
+name: "Localnet Test"
 on:
   workflow_dispatch: # Manually triggered
 
@@ -35,5 +35,11 @@ jobs:
       - name: "Download and verify dependencies"
         run: make deps
 
-      - name: "make networktest"
-        run: make networktest
+      - name: "run localnet"
+        run: docker compose -f ./e2e/docker-compose.yml up -d
+
+      - name: "get localnet stats"
+        working-directory: ./e2e/monitor
+        # XXX should this be a make command?
+        run: go test -v ./...
+

Makefile

@@ -29,7 +29,7 @@ cmds = \
 	tbcd
 
 .PHONY: all clean clean-dist deps $(cmds) build install lint lint-deps tidy race test vulncheck \
-	vulncheck-deps dist archive sources checksums networktest
+	vulncheck-deps dist archive sources checksums
 
 all: lint tidy test build install
 
@@ -103,6 +103,3 @@ sources: dist
 checksums: dist
 	cd $(DIST) && shasum -a 256 * > $(project)_$(version)_checksums.txt
 
-networktest:
-	HEMI_RUN_NETWORK_TEST=1	\
-		go test -count=1 -v -run TestFullNetwork -v ./e2e/network_test.go

README.md

@@ -151,15 +151,6 @@ data and give you a fresh start
 docker compose -f ./e2e/docker-compose.yml down -v --remove-orphans
 ```
 
-
-
-### Running the full network tests
-
-This runs a test with an entirely local heminet, it uses bitcoind in regtest
-mode for the bitcoin chain
-
-Prerequisites: `docker`
-
-```
-make networktest
-```
+**NOTE:** The `--remove-orphans` flag should remove other containers not defined
+in the docker compose file. This is mainly here to help ensure you start with a
+clean environment.  It can be omitted.

e2e/monitor/README.md

@@ -0,0 +1,49 @@
+# Localnet Monitor
+
+The Localnet Monitor is a small program that simply polls localnet for values
+that we want to test against.  
+
+## Prerequisites
+
+* Go 1.22+
+* `docker` available in your cli
+
+## Running
+
+You can run the Localnet Monitor like so, this will read from localnet
+and print a table that refreshes every 1 second.
+
+Make sure you have localnet running:
+
+from the root of the repo:
+```
+docker compose -f ./e2e/docker-compose.yml down -v --remove-orphans
+docker compose -f ./e2e/docker-compose.yml up
+```
+
+**NOTE:** The `--remove-orphans` flag should remove other containers not defined
+in the docker compose file. This is mainly here to help ensure you start with a
+clean environment.  It can be omitted.
+
+from this directory:
+```
+$  go run ./... 
++--------------------------------+------------------------------------------------------------------------+
+| refreshing every 1 seconds     |                                                                        |
+| bitcoin block count            | 3007                                                                   |
+| poptxs mined                   | 11                                                                     |
+| first batcher publication hash | 0x2b86a72b48668b7a35dcab99166f9330c884c50d1b19847c3c0569a0d0806465,21  |
+| last batcher publication hash  | 0x0d2e805f1180f81dfb3abe97b9edb1894c110a6be58a7cfcd14de65807613670,108 |
+| batcher publication count      | 23                                                                     |
+| pop miner $HEMI balance        | 4000000000000000000                                                    |
++--------------------------------+------------------------------------------------------------------------+
+```
+
+If you would like to print the results as json, you can give the json 
+"snapshot" a delay via the env variable `HEMI_E2E_DUMP_JSON_AFTER_MS`, 
+after these milliseconds, values will be read and dumped.
+
+```
+$ HEMI_E2E_DUMP_JSON_AFTER_MS=10000 go run ./... 
+{"bitcoin_block_count":3011,"pop_tx_count":20,"first_batcher_publication_hash":"0x2b86a72b48668b7a35dcab99166f9330c884c50d1b19847c3c0569a0d0806465,21","last_batcher_publication_hash":"0x5ec52eeba46c300e98546de25991c1862ef8dd11c3ee3357ee2a717517e2fe8c,192","batcher_publication_count":34,"pop_miner_hemi_balance":"14000000000000000000"}
+```

e2e/monitor/go.mod

@@ -0,0 +1,30 @@
+module github.com/hemilabs/heminetwork/e2e/monitor
+
+go 1.22.2
+
+replace github.com/hemilabs/heminetwork => ../../
+
+require (
+	github.com/btcsuite/btcd v0.24.0
+	github.com/btcsuite/btcd/chaincfg/chainhash v1.1.0
+	github.com/gosuri/uilive v0.0.4
+	github.com/hemilabs/heminetwork v0.1.0
+	github.com/jedib0t/go-pretty/v6 v6.5.8
+)
+
+require (
+	github.com/btcsuite/btcd/btcec/v2 v2.3.2 // indirect
+	github.com/btcsuite/btcd/btcutil v1.1.5 // indirect
+	github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f // indirect
+	github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd // indirect
+	github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792 // indirect
+	github.com/decred/dcrd/crypto/blake256 v1.0.1 // indirect
+	github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
+	github.com/juju/loggo v1.0.0 // indirect
+	github.com/mattn/go-isatty v0.0.0-20160806122752-66b8e73f3f5c // indirect
+	github.com/mattn/go-runewidth v0.0.15 // indirect
+	github.com/rivo/uniseg v0.2.0 // indirect
+	golang.org/x/crypto v0.19.0 // indirect
+	golang.org/x/sys v0.17.0 // indirect
+)
+