MM-61311: Version-controlled config samples

[agarciamontoro]

Summary

This is something I've been wanting to do for a long while: having a set of config files that can be used as starter packs for other workflows. We already use two sets internally, which are the ones that I've uploaded here: the config files for release testing of the tool itself, and the config files for the performance comparisons of Mattermost new releases.

Some questions you may have:

• Why now? Because we've spent soooo much time these days trying to merge old config files with new fields for the latest release testing.
• And why TOML and not JSON? Because I want to start using it. The possibility of having comments in the files is incredibly useful, and just that outweighs other minor annoyances (e.g. TOML arrays, which are a bit weird). In any case, it's an experiment. If it doesn't work, we can revert back to JSON, no strings attached.
• And how are we going to keep these updated? Well, I'm going to add a step now in both the release testing and performance comparison playbooks to review these before we run the corresponding tests.

Ah! I've also added sample files in the config/ directory for the coordinator and comparison files in TOML, which we were missing. This is unrelated to the original goal of the PR.

Ticket Link

https://mattermost.atlassian.net/browse/MM-61311

[streamer45]

Looks good, some non-blocking questions:

• What's needed for the tool to pick up a .toml vs a .json?
• What happens if both files are present?
• Should we surface (e.g. log) which config file is loaded at any given time?

[agnivade]

Nice!

[agarciamontoro]

After double-checking with @fmartingr:

What's needed for the tool to pick up a .toml vs a .json?

One needs to pass its path as CLI flags. The flags are a bit eclectic: config for the deployer, ltconfig for config, comparison-config for the comparison and coordinator-config for the coordinator. I still need to test all this works.

What happens if both files are present?

By default it picks JSON. If the config flags are passed, those have priority.

Should we surface (e.g. log) which config file is loaded at any given time?

Maybe that's a good idea. Were you thinking of logging the whole thing? Just the path?

[streamer45]

After double-checking with @fmartingr:

What's needed for the tool to pick up a .toml vs a .json?

One needs to pass its path as CLI flags. The flags are a bit eclectic: config for the deployer, ltconfig for config, comparison-config for the comparison and coordinator-config for the coordinator. I still need to test all this works.

Thanks

What happens if both files are present?

By default it picks JSON. If the config flags are passed, those have priority.

👍

Should we surface (e.g. log) which config file is loaded at any given time?

Maybe that's a good idea. Were you thinking of logging the whole thing? Just the path?

I think the path would be sufficient.

[agarciamontoro]

I think the path would be sufficient.

Doing this in a future PR