Skip to content

Commit

Permalink
Zero params configuration and docs for configuration (#4)
Browse files Browse the repository at this point in the history
* fix: zero params configuration and docs for configuration
  • Loading branch information
dainiusjocas authored Mar 22, 2021
1 parent 2a6116f commit 56ca4d2
Show file tree
Hide file tree
Showing 2 changed files with 92 additions and 12 deletions.
78 changes: 78 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Configuration

`ket` is a CLI tool and can be configured in several ways:

- CLI arguments and flags
- configuration file;
- JQ overrides;
- environment variables.

The overall command format is: `[ENV_VARIABLES] ket [GLOBAL_FLAGS] [OPERATION] [OPERATION_FLAGS]`

## CLI arguments and flags

Every operation has a different set of supported flags and this list can be obtained by providing the `-h` or `--help` flag, e.g.:
```shell
./ket -h
```
or:
```shell
./ket reindex -h
```

## Configuration file

Configuration file is expected to be JSON.
To specify a configuration file use a CLI flag `-f` or `--config-file`, e.g.:
```shell
./ket operation -f config.json
```
or equivalent:
```shell
./ket operation -config-file=config.json
```

## JQ Overrides

JQ overrides are applied on the configuration read from the configuration file. E.g. configuration file `test-conf.json` contents are:
```json
{"max_docs": 1}
```
When we apply `jq` query `.max_docs=2` e.g.:
```shell
./ket reindex -f test-conf.json --override='.max_docs=2'
```
the configuration that will be passed to the `reindex` operation will look like this:
```json
{"max_docs":2}
```

Since the overrides are regular JQ scripts, it is possible to join configuration from different configuration files, e.g.:
```shell
./ket reindex \
-f test-conf.json \
--override='.source='$(jq -c '.source' examples/replay.json)'' \
--dry-run
```
And whatever is in the `examples/replay.json` file under `source` key, it will be included into the configuration for the current conmmand.

Another trick is that `ket` supports the `--dry-run` flag, which instructs `ket` to prepare configuration, and (instead of invoking the operation) to print configuration JSON to the STDOUT, e.g.:
```shell
./ket reindex \
-f test-conf.json \
--override='.source='$(jq -c '.source' examples/replay.json)'' \
--dry-run
```
You can inspect the config, or dump it to a file for later processing.

NOTE: this documents expects jq to be installed in the machine.

NOTE: if configuration file is not provided, then empty configuration `{}` is assumed, i.e. JQ overrides are applied on an empty hashmap.

## Environment variables

With environment variables only one aspect can be configured: logging level. E.g.:

```shell
ROOT_LOGGER_LEVEL=WARN ./ket operation -f config.json
```
26 changes: 14 additions & 12 deletions src/core.clj
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,16 @@
(first (filter (fn [op] (= (name operation-name) (:name op))) cli-operations)))

(defn read-config-file [config-file jq-overrides]
(if (and config-file (.exists (io/file config-file)))
(if (seq jq-overrides)
(let [^String file-contents (slurp config-file)]
(json/decode (jq/execute file-contents (str/join " | " jq-overrides))))
(json/read-file config-file))
(do
(when config-file
(log/warnf "Config file '%s' does not exists" config-file))
{})))
(let [combined-jq-overrides (str/join " | " jq-overrides)]
(if (and config-file (.exists (io/file config-file)))
(if (seq jq-overrides)
(let [^String file-contents (slurp config-file)]
(json/decode (jq/execute file-contents combined-jq-overrides)))
(json/read-file config-file))
(do
(if config-file
(throw (Exception. (format "Config file '%s' does not exists" config-file)))
(json/decode (jq/execute "{}" combined-jq-overrides)))))))

(defn execute-op [operation-name options cli-operations]
(if operation-name
Expand Down Expand Up @@ -64,8 +65,7 @@
(println (json/encode combined-conf))
(execute-op operation-name combined-conf cli-operations)))))))
(catch Exception e
(println (format "Failed to execute with exception:\n '%s'" e))
(.printStackTrace e))))
(log/errorf "Failed to execute with exception: '%s'" e))))

(def cli-operations
(concat ops/operations ops-overrides/cli))
Expand All @@ -74,7 +74,9 @@
(let [{:keys [options summary errors arguments] :as cli-opts} (cli/recursive-parse args cli-operations)]
(if errors
(println errors)
(if (or (get options :help) (and (empty? options) (empty? arguments)))
(if (or (get options :help)
(and (empty? options) (empty? arguments))
(empty? args))
(println summary)
(handle-subcommand cli-opts cli-operations)))))

Expand Down

0 comments on commit 56ca4d2

Please sign in to comment.