Integration/v1 (#81)

* Chore: Update to Dusk-5 (#67) * working local updates for dusk-5 * revert rollup name in env * update remote config and sequencer internals for new protos * env var updates * Update go.mod * use astria-dusk-5 for all networks * Feature: store private key in keyring or keyfile (#64) * store private key in os keyring * --insecure flag for createaccount * improve secure flag * createaccount support keyfile * cleanup * working transfer with privkey, keyring, and keyfile * handle error * test file decryption * remove quotes that implied we aren't secure. was quoting because we're in the if branch that is ran when no `--insecure` flag exists, which means we're "secure" * capture all args for nicer just run usage * update just command comment * use astria-dusk-5 as chain id so remote will work * can only pass in one type of key flag * Feature: Replace `clean` commands with `reset` and `purge` commands (#76) * init doesn't overwrite existing files * add reset commands * add purge commands * update exec "cp" commands with io.copy * readme updates * update comments and descriptions * refactor helper functions and config * fix sequencer transfer description in readme * add cometbft rpc server ready check (#77) * feature: bridge commands (#66) * feat: Add init-bridge and bridge-lock commands * minor changes * store private key in os keyring * --insecure flag for createaccount * chore: initbridge response and command changes * improve secure flag * createaccount support keyfile * cleanup * working transfer with privkey, keyring, and keyfile * handle error * test file decryption * remove quotes that implied we aren't secure. was quoting because we're in the if branch that is ran when no `--insecure` flag exists, which means we're "secure" * capture all args for nicer just run usage * update just command comment * update go-sequencer-client and protos. * helpers. tests. * Update initbridge.go * noun verb. remove --privkey from usage * usage strings * consistent doc strings * reorg bridge commands into one file. make parent bridge command for proper help text. add missing fields to proto. * add chain, asset, and fee asset ids to init bridge opts --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> * Feature: Update services and CLI config (#79) * added network config default file creation * cleanup network config init * apply config overrides * flags and commands updated * update transfer command and integration tests * update invalid sequencer chain id errors * add default denom to network config and reset * move networks config from utils package to config package * readme updates * readme fmt * add base config and service log level flags * added debug logging for config * removed extra config struct nesting * remove .env files * refactor config dir * update reset commands * reset networks command comment and info update * move shared dev command flags to dev cmd root * command handler added for dev commands * update cli flag handler to have bindBoolFlag and rename handler file * update sequencer commands to use flagHandler * update flagHandler GetValue function to properly return differently typed flag values * update sequencer-chain-id and sequencer-url flags * GetEnvOverrides now updates ASTRIA_COMPOSER_ROLLUPS completely * update bridge commands to use flag handler * update load config names * instance flag help update * comments, typos, and small formatting fixes * refactor service log flag, remove need to SetLogLevel using PreRun for all commands * cobra init functions moved to after all command definitions * ValidateServiceLogLevelOrPanic moved to run.go * process runner opts config var renamed to env * move non global flags to local commands * use constants for defaults, fix initialisms * update bridge lock sequencer url description * fix string flag get value bug * typo fixes and small renamings * ConvertStructToEnvArray moved to ToSlice on BaseConfig * fix LoadNetworkConfigsOrPanic naming and CreateBaseConfig doc comment * update environment config flow * update GetEndpointOverrides to match full ws endpoint * doc comment and regex update for GetEndpointOverrides * Fix/bridge opts integration branch (#87) * added network config default file creation * cleanup network config init * apply config overrides * flags and commands updated * update transfer command and integration tests * update invalid sequencer chain id errors * add default denom to network config and reset * move networks config from utils package to config package * readme updates * readme fmt * add base config and service log level flags * added debug logging for config * removed extra config struct nesting * remove .env files * refactor config dir * update reset commands * reset networks command comment and info update * move shared dev command flags to dev cmd root * command handler added for dev commands * update cli flag handler to have bindBoolFlag and rename handler file * update sequencer commands to use flagHandler * update flagHandler GetValue function to properly return differently typed flag values * update sequencer-chain-id and sequencer-url flags * GetEnvOverrides now updates ASTRIA_COMPOSER_ROLLUPS completely * update bridge commands to use flag handler * update load config names * add fields to bridge lock opts * usage strings and doc comments * instance flag help update * update protos and go-sequencer-client versions * comments, typos, and small formatting fixes * refactor service log flag, remove need to SetLogLevel using PreRun for all commands * cobra init functions moved to after all command definitions * ValidateServiceLogLevelOrPanic moved to run.go * process runner opts config var renamed to env * move non global flags to local commands * update flag names * consistent naming * use constants for defaults, fix initialisms * update bridge lock sequencer url description * fix string flag get value bug * typo fixes and small renamings * ConvertStructToEnvArray moved to ToSlice on BaseConfig * fix LoadNetworkConfigsOrPanic naming and CreateBaseConfig doc comment * fix bridge lock flags * update environment config flow * update GetEndpointOverrides to match full ws endpoint --------- Co-authored-by: Sam Bukowski <sam.c.bukowski@gmail.com> --------- Co-authored-by: Sam Bukowski <sam.c.bukowski@gmail.com> Co-authored-by: quasystaty <121364949+quasystaty1@users.noreply.github.com>
astriaorg · May 23, 2024 · 3a634c4 · 3a634c4
1 parent 670cffa
commit 3a634c4
Show file tree

Hide file tree

Showing 51 changed files with 2,986 additions and 1,625 deletions.
diff --git a/README.md b/README.md
@@ -27,13 +27,20 @@ the Astria stack and interact with the Sequencer.
 | `dev`                     | Root command for cli development functionality.                                     |
 | `dev init`                | Downloads binaries and initializes the local environment.                           |
 | `dev run`                 | Runs a minimal, local Astria stack.                                                 |
-| `dev clean`               | Deletes the local data for the Astria stack.                                        |
-| `dev clean all`           | Deletes the local data, downloaded binaries, and config files for the Astria stack. |
+| `dev reset`               | The root command for resetting the local development instance data.                 |
+| `dev reset config`        | Reset config files.                                                                 |
+| `dev reset env`           | Reset environment variable files.                                                   |
+| `dev reset state`         | Reset Sequencer state.                                                              |
+| `dev purge binaries`      | Delete all binaries for a given instance.                                           |
+| `dev purge all`           | Delete all data for a given instance.                                               |
 | `sequencer balances`      | Get the balances of an account on the Sequencer.                                    |
 | `sequencer blockheight`   | Get the current block height of the Sequencer.                                      |
 | `sequencer createaccount` | Generate an account for the Sequencer.                                              |
 | `sequencer nonce`         | Get the current nonce for an account.                                               |
-| `sequencer transfer`      | Get the current block height of the Sequencer.                                      |
+| `sequencer transfer`      | Make a transfer on the Sequencer.                                      |
+
+> NOTE: Running a `dev purge` command requires you to run `dev init` afterwards to
+> reinitialize the deleted data.
 
 ## Installation
 
@@ -90,6 +97,8 @@ configuration files, and initializes CometBFT.
 
 The following files are generated:
 
+* In `~/.astria/<instance>`
+  * A `networks-config.toml` easily configuring different target networks
 * In `~/.astria/<instance>/config-local`:
   * A `.env` file for local configuration
   * `genesis.json` for the sequencer genesis
@@ -101,10 +110,10 @@ The following binaries are downloaded:
 
 | App              | Version |
 |------------------|---------|
-| Cometbft         | v0.37.4 |
-| Astria-Sequencer | v0.10.1 |
-| Astria-Conductor | v0.13.1 |
-| Astria-Composer  | v0.5.0  |
+| Cometbft         | v0.38.6 |
+| Astria-Sequencer | v0.11.0 |
+| Astria-Conductor | v0.14.0 |
+| Astria-Composer  | v0.6.0  |
 
 The `init` command will also run the initialization steps required by CometBFT,
 using the `genesis.json` and `priv_validator_key.json` files in the
@@ -135,20 +144,21 @@ dependencies.
 
 > NOTE: Running a local Sequencer is the default behavior of `dev run` command.
 > Thus, `astria-go dev run` is effectively an alias of
-> `astria-go dev run --local`.
+> `astria-go dev run --network local`.
 
 #### Run Against a Remote Sequencer
 
 If you want to run Composer and Conductor locally against a remote Astria
 Sequencer:
 
 ```bash
-astria-go dev run --remote
+# Run against the Astria Dusk dev net 
+astria-go dev run --network dusk
 ```
 
-Using the `--remote` flag, the cli will handle configuration of the components
-running on your local machine, but you will need to create an account on the
-remote sequencer. More details can be
+When using the `--network` flag to target a remote sequencer, the cli will
+handle configuration of the components running on your local machine, but you
+will need to create an account on the remote sequencer. More details can be
 [found here](https://docs.astria.org/developer/tutorials/1-using-astria-go-cli#setup-and-run-the-local-astria-components-to-communicate-with-the-remote-sequencer).
 
 ### Run Custom Binaries
@@ -161,7 +171,7 @@ cli to run your locally compiled Conductor with the other components using the
 `--conductor-path` flag:
 
 ```bash
-astria-go dev run --local \
+astria-go dev run --network local \
   --conductor-path <absolute path to the Astria mono repo>/target/debug/astria-conductor
 ```
 
@@ -170,7 +180,7 @@ binaries, while using a locally built version of the Conductor binary. You can
 swap out some or all binaries for the Astria stack with their appropriate flags:
 
 ```bash
-astria-go dev run --local \
+astria-go dev run --network local \
   --sequencer-path <sequencer bin path> \
   --cometbft-path <cometbft bin path> \
   --composer-path <composer bin path> \
@@ -183,11 +193,10 @@ If additional configuration is required, you can update the `.env` files in
 
 ## Instances
 
-The `dev init`, `dev run`, and `dev clean` commands all have an optional
-`--instance` flag. The value of this flag will be used as the directory name
-where the rollup data will be stored. Now you can run many rollups while keeping
-their configs and state data separate. If no value is provided, `default` is
-used, i.e. `~/.astria/default`.
+The `dev` commands all have an optional `--instance` flag. The value of this
+flag will be used as the directory name where the rollup data will be stored.
+Now you can run many rollups while keeping their configs and state data
+separate. If no value is provided, `default` is used, i.e. `~/.astria/default`.
 
 For example, if you run:
 

diff --git a/cmd/cliflaghandler.go b/cmd/cliflaghandler.go
@@ -0,0 +1,135 @@
+package cmd
+
+import (
+	"fmt"
+	"os"
+	"reflect"
+	"strings"
+
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
+)
+
+// CliFlagHandler is a struct that handles the binding of flags and the retrieval of the flag's string value
+type CliFlagHandler struct {
+	Cmd       *cobra.Command
+	EnvPrefix string
+}
+
+// CreateCliFlagHandler creates a new CliFlagHandler.
+func CreateCliFlagHandler(c *cobra.Command, envPrefix string) *CliFlagHandler {
+	return &CliFlagHandler{
+		Cmd:       c,
+		EnvPrefix: envPrefix,
+	}
+}
+
+// BindStringFlag binds a string flag to a cobra flag and viper env var handler for a
+// local command flag, and automatically creates the env var from the flag name.
+func (f *CliFlagHandler) BindStringFlag(name string, defaultValue string, usage string) {
+	envSuffix := strings.ToUpper(strings.ReplaceAll(name, "-", "_"))
+
+	f.Cmd.Flags().String(name, defaultValue, usage)
+	err := viper.BindPFlag(envSuffix, f.Cmd.Flags().Lookup(name))
+	if err != nil {
+		log.Fatalf("Error binding string flag: %s", err)
+	}
+}
+
+// BindStringPFlag binds a string flag to a cobra flag and viper env var handler for a
+// local command flag, and automatically creates the env var from the flag name.
+func (f *CliFlagHandler) BindStringPFlag(name string, shorthand string, defaultValue string, usage string) {
+	envSuffix := strings.ToUpper(strings.ReplaceAll(name, "-", "_"))
+
+	f.Cmd.Flags().StringP(name, shorthand, defaultValue, usage)
+	err := viper.BindPFlag(envSuffix, f.Cmd.Flags().Lookup(name))
+	if err != nil {
+		log.Fatalf("Error binding string flag: %s", err)
+	}
+}
+
+// BindBoolFlag binds a boolean flag to a cobra flag and viper env var handler for a
+// local command flag, and automatically creates the env var from the flag name.
+func (f *CliFlagHandler) BindBoolFlag(name string, defaultValue bool, usage string) {
+	envSuffix := strings.ToUpper(strings.ReplaceAll(name, "-", "_"))
+
+	f.Cmd.Flags().Bool(name, defaultValue, usage)
+	err := viper.BindPFlag(envSuffix, f.Cmd.Flags().Lookup(name))
+	if err != nil {
+		log.Fatalf("Error binding bool flag: %s", err)
+	}
+}
+
+// BindPersistentFlag binds a string flag to a cobra flag and viper env var
+// handler for a persistent command flag shared by a command and its
+// subcommands, and automatically creates the env var from the flag name.
+func (f *CliFlagHandler) BindPersistentFlag(name string, defaultValue string, usage string) {
+	envSuffix := strings.ToUpper(strings.ReplaceAll(name, "-", "_"))
+
+	f.Cmd.PersistentFlags().String(name, defaultValue, usage)
+	err := viper.BindPFlag(envSuffix, f.Cmd.PersistentFlags().Lookup(name))
+	if err != nil {
+		log.Fatalf("Error binding persistent flag: %s", err)
+	}
+}
+
+// toEnvVarName returns the full env var name for a given flag name.
+func (f *CliFlagHandler) toEnvVarName(flagName string) string {
+	envSuffix := strings.ToUpper(strings.ReplaceAll(flagName, "-", "_"))
+	fullEnvVar := strings.ToUpper(f.EnvPrefix) + "_" + strings.ToUpper(envSuffix)
+	return fullEnvVar
+}
+
+// GetValue returns the value of a flag as a string and logs the source of the
+// value. It will panic if the flag does not exist or if the flag cannot be read.
+func (f *CliFlagHandler) GetValue(flagName string) string {
+	envSuffix := strings.ToUpper(strings.ReplaceAll(flagName, "-", "_"))
+	// confirm the flag exists
+	flag := f.Cmd.Flags().Lookup(flagName)
+	if flag == nil {
+		log.Errorf("Flag '%s' doesn't exist. Has it been bound?", flagName)
+		panic(fmt.Sprintf("getValue: flag doesn't exist: %s", flagName))
+	}
+
+	// get the value from viper based on type
+	var value string
+	valueKind := reflect.TypeOf(flag.Value).Elem().Kind()
+	switch valueKind {
+	case reflect.Bool:
+		// we need to rebind the flag to viper when reading the value to
+		// ensure that it is read correctly. Otherwise, viper will always
+		// return the default value.
+		err := viper.BindPFlag(envSuffix, flag)
+		if err != nil {
+			log.Fatalf("getValue: Error rebinding bool flag for reading: %s", flagName)
+			panic(err)
+		}
+		value = fmt.Sprintf("%t", viper.GetBool(envSuffix))
+
+	case reflect.String:
+		err := viper.BindPFlag(envSuffix, flag)
+		if err != nil {
+			log.Fatalf("getValue: Error rebinding string flag for reading: %s", flagName)
+			panic(err)
+		}
+		value = viper.GetString(envSuffix)
+
+	default:
+		log.Errorf("Flag '%s' has an unsupported type: %s", flagName, valueKind)
+		panic(fmt.Sprintf("getValue: unsupported flag type: %s", valueKind))
+	}
+
+	if f.Cmd.Flag(flagName).Changed {
+		log.Debugf("%s flag is set with value: %s", flagName, value)
+		return value
+	}
+	_, envExists := os.LookupEnv(f.toEnvVarName(flagName))
+	if envExists {
+		log.Debugf("%s flag is set via env var to: %s", flagName, value)
+		return value
+	}
+
+	log.Debugf("%s flag is not set, using default: %s", flagName, value)
+	return value
+}
diff --git a/cmd/devtools/clean.go b/cmd/devtools/clean.go
diff --git a/cmd/devtools/binaries_config_darwin_amd64.go → ...ls/config/binaries_config_darwin_amd64.go b/cmd/devtools/binaries_config_darwin_amd64.go → ...ls/config/binaries_config_darwin_amd64.go
@@ -1,15 +1,15 @@
 //go:build darwin && amd64
 
-package devtools
+package config
 
 type Binary struct {
 	Name string
 	Url  string
 }
 
 var Binaries = []Binary{
-	{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.37.4/cometbft_0.37.4_darwin_amd64.tar.gz"},
-	{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.10.1/astria-sequencer-x86_64-apple-darwin.tar.gz"},
-	{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.5.0/astria-composer-x86_64-apple-darwin.tar.gz"},
-	{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.13.1/astria-conductor-x86_64-apple-darwin.tar.gz"},
+	{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.38.6/cometbft_0.38.6_darwin_amd64.tar.gz"},
+	{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.11.0/astria-sequencer-x86_64-apple-darwin.tar.gz"},
+	{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.6.0/astria-composer-x86_64-apple-darwin.tar.gz"},
+	{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.14.0/astria-conductor-x86_64-apple-darwin.tar.gz"},
 }
diff --git a/cmd/devtools/binaries_config_darwin_arm64.go → ...ls/config/binaries_config_darwin_arm64.go b/cmd/devtools/binaries_config_darwin_arm64.go → ...ls/config/binaries_config_darwin_arm64.go
@@ -1,15 +1,15 @@
 //go:build darwin && arm64
 
-package devtools
+package config
 
 type Binary struct {
 	Name string
 	Url  string
 }
 
 var Binaries = []Binary{
-	{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.37.4/cometbft_0.37.4_darwin_arm64.tar.gz"},
-	{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.10.1/astria-sequencer-aarch64-apple-darwin.tar.gz"},
-	{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.5.0/astria-composer-aarch64-apple-darwin.tar.gz"},
-	{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.13.1/astria-conductor-aarch64-apple-darwin.tar.gz"},
+	{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.38.6/cometbft_0.38.6_darwin_arm64.tar.gz"},
+	{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.11.0/astria-sequencer-aarch64-apple-darwin.tar.gz"},
+	{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.6.0/astria-composer-aarch64-apple-darwin.tar.gz"},
+	{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.14.0/astria-conductor-aarch64-apple-darwin.tar.gz"},
 }
diff --git a/cmd/devtools/binaries_config_linux_amd64.go → ...ols/config/binaries_config_linux_amd64.go b/cmd/devtools/binaries_config_linux_amd64.go → ...ols/config/binaries_config_linux_amd64.go
@@ -1,15 +1,15 @@
 //go:build linux && amd64
 
-package devtools
+package config
 
 type Binary struct {
 	Name string
 	Url  string
 }
 
 var Binaries = []Binary{
-	{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.37.4/cometbft_0.37.4_linux_amd64.tar.gz"},
-	{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.10.1/astria-sequencer-x86_64-unknown-linux-gnu.tar.gz"},
-	{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.5.0/astria-composer-x86_64-unknown-linux-gnu.tar.gz"},
-	{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.13.1/astria-conductor-x86_64-unknown-linux-gnu.tar.gz"},
+	{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.38.6/cometbft_0.38.6_linux_amd64.tar.gz"},
+	{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.11.0/astria-sequencer-x86_64-unknown-linux-gnu.tar.gz"},
+	{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.6.0/astria-composer-x86_64-unknown-linux-gnu.tar.gz"},
+	{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.14.0/astria-conductor-x86_64-unknown-linux-gnu.tar.gz"},
 }