feat: config reloading #1771
Merged
feat: config reloading #1771
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
September 10, 2024 14:42
This commit provides a basic primitive for the thread safe hot swapping of an active http.Handler. It works by wrapping an atomic.Value within a struct implementing the http.Handler interface. Before serving each request it first makes a load call on the atomic.Value, returning the most recently stored http.Handler. The package is accompanied by two tests, one to test the basic store & load support. The second test is meant to be ran only when the race detector is present.
I've added LoadDirectory and LoadFile with comments explaining the behavior of each. LoadGlobalFromEnv simply calls loadGlobal which configures a *GlobalConfiguration from the processes current env. These will be used to compose a *GlobalConfiguration from the multiple configuration sources during process initialization and during live reloads.
A new optional flag (long: --watch-dir, short: -w) has been
added. When present any files with a ".env" suffix will be
loaded into the environment before the *GlobalConfiguration
is created, otherwise existing behavior is preserved.
In addition when the watch-dir flag is present a goroutine
will be started in serve_cmd.go and begin blocking on a call
to (*Reloader).Watch with a callback function that accepts a
*conf.GlobalConfiguration object. Each time this function is
called we create a new API object and store it within our
AtomicHandler, previously given as the root handler to the
*http.Server.
The Reloader uses some simple heuristics to deal with a few
edge cases, an overview:
- At most 1 configuration reload may happen per 10 seconds
with a +-1s margin of error.
- After a file within -watch-dir has changed the 10 second
grace period begins. After that it will reload the config.
- Config reloads first sort each file by name then processes
them in sequence.
- Directories within watch-dir are ignored during config
reloading.
- Implementation quirk: directory changes can trigger
a config reload, as I don't stat fsnotify events. This
and similar superfulous reloads could be easily fixed
by storing a snapshot of os.Environ() after successful
reloads to compare with the latest via slices.Equal()
before reloading.
- Files that do not end with a .env suffix are ignored.
- It handles the removal or renaming of the -watch-dir
during runtime, but an error message will be printed
every 10 seconds as long as it's missing.
- The config file passed with -c is only loaded once. Live
reloads only read the config dir. Meaning it would be
possible to create a config dir change that results in a
new final configuration on the next reload due to the
persistence of os.Environ().
|
@cstockton seems like some of the gosec tests are broken |
Pull Request Test Coverage Report for Build 10927565463Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
hf
approved these changes
Sep 18, 2024
hf
reviewed
Sep 18, 2024
hf
reviewed
Sep 18, 2024
hf
pushed a commit
that referenced
this pull request
Sep 24, 2024
🤖 I have created a release *beep* *boop* --- ## [2.161.0](v2.160.0...v2.161.0) (2024-09-24) ### Features * add `x-sb-error-code` header, show error code in logs ([#1765](#1765)) ([ed91c59](ed91c59)) * add webauthn configuration variables ([#1773](#1773)) ([77d5897](77d5897)) * config reloading ([#1771](#1771)) ([6ee0091](6ee0091)) ### Bug Fixes * add additional information around errors for missing content type header ([#1576](#1576)) ([c2b2f96](c2b2f96)) * add token to hook payload for non-secure email change ([#1763](#1763)) ([7e472ad](7e472ad)) * update aal requirements to update user ([#1766](#1766)) ([25d9874](25d9874)) * update mfa admin methods ([#1774](#1774)) ([567ea7e](567ea7e)) * user sanitization should clean up email change info too ([#1759](#1759)) ([9d419b4](9d419b4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
File based configuration reloading using fsnotify.
What is the current behavior?
Currently the Auth config is loaded once from the environment or file (-c flag) and persists until the service is restarted.
What is the new behavior?
A new optional flag (long:
--watch-dir, short:-w) has been added. When present any files with a ".env" suffix will be loaded into the environment before the*GlobalConfigurationis created, otherwise existing behavior is preserved.In addition when the watch-dir flag is present a goroutine will be started in serve_cmd.go and begin blocking on a call to
(*Reloader).Watchwith a callback function that accepts a*conf.GlobalConfiguration object. Each time this function is called we create a new*api.APIobject and store it within ourAtomicHandler, previously given as the root handler to the*http.Server.The Reloader uses some simple heuristics to deal with a few edge cases, an overview:
--watch-dirhas changed the 10 second grace period begins. After that it will reload the config.--watch-dirare ignored during config reloading.os.Environ()after successful reloads to compare with the latest viaslices.Equal()before reloading..envsuffix are ignored.-watch-dirduring runtime, but an error message will be printed every 10 seconds as long as it's missing.os.Environ().