Feature/nar 1545 app setup keygen

[mattschoch]

No description provided.

[linear]

NAR-1545 App Setup & keygen

[mattschoch]

@wcalderipe Pushed some updated

• encrypt and decrypt work correctly now.
• If you set env variables for KMS, it supports a AWS KMS keyring, otherwise it will generate a masterKey & encrypted based on a masterPassword config.
• I don't do ANYTHING related to api key anymore.
• I think the Engine & API key config or whatever should be set up elsewhere; I'm inserting to the engine table if needed but I believe the EncryptionService should only bootstrap the encryption encryption keyrings.

Other things to note / give feedback on

• I don't have tests on the KMS keyring...I tried it locally, but don't love the idea of using a real key in tests. wdyt? Maybe that'd be good, we can make a totally open key policy just for tests so we do actually do an e2e test on it? Should that go in e2e though or unit since everything else is the same as the unit test?
• Look at my jest.setup.ts, I removed the .env file completely rather than overwriting it; I really don't like that you have to look at multiple files to know the env of the tests, much prefer to have to explicitly define the whole environment.

LMK any thoughts you have, I believe this can be merged & is stable; it may continue to evolve a bit, but the core functions are working now & have basic tests. Do you see places you'd suggest additional tests?

[wcalderipe]

I wonder why not delete process.env directly.

[mattschoch]

I don't think you want/should delete the whole process.env object, not sure what happens there...tbh I just copy/pasted a snippet

[wcalderipe]

@mattschoch

I don't do ANYTHING related to api key anymore.
I think the Engine & API key config or whatever should be set up elsewhere; I'm inserting to the engine table if needed but I believe the EncryptionService should only bootstrap the encryption encryption keyrings.

Cool.

I've been thinking about the pattern of using the OnApplicationBootstrap interface on multiple services for provisioning setup, and I feel we could improve by centralizing the engine provisioning logic into a single service. This way, you'd know exactly where to look if you need to change the engine's provision.

The provision would involve other services like encryption:

• Provisioning the encryption keys
• Provisioning the admin API key
• Anything else needed (e.g., diagnostics on the network, checking if the database is using the latest version)

It doesn't have to be done in your PR; it's just an idea I'm exploring. What do you think?

I don't have tests on the KMS keyring...I tried it locally, but don't love the idea of using a real key in tests. wdyt? Maybe that'd be good, we can make a totally open key policy just for tests so we do actually do an e2e test on it? Should that go in e2e though or unit since everything else is the same as the unit test?

I believe we're good to merge. However, since KMS is critical to the application's logic and security model, I would definitely include an integration test for it.

Moreover, I think we should be able to run the application connected to any dependency to easily debug it. I've used LocalStack in the past to simulate AWS services (S3, DynamoDB, Kinesis, and Lambda). They offer a version of KMS, which you can find at LocalStack KMS documentation.

Aside from a standalone application, they also distribute it as a Docker image, available at LocalStack Docker images that you can simply drop in the docker-compose.yml or in the GitHub Action workflow definition.

Look at my jest.setup.ts, I removed the .env file completely rather than overwriting it; I really don't like that you have to look at multiple files to know the env of the tests, much prefer to have to explicitly define the whole environment.

Copy that. Do you think you can set up the Armory project to do the same? I'd prefer that we have a consistent way of handling cross-functional tasks like environment management.

[wcalderipe]

I left you a comment about using localstack #139 (comment) instead of connecting to AWS.

[mattschoch]

Re: testing KMS --
let me push that to "I'll add integration test on this separately, solving that problem on it's own"
I believe for KMS I probably prefer the solution of just actually using KMS directly, not localstack, since it's probably easier to have a single real key we're using. I just need to set up reasonable policies & credentials to do it, but we'd have to run A LOT of ci/cd to make it a cost-issue so I actually don't think there's a concern using KMS directly from local or ci/cd.

I definitely agree with your point that any dependency should be easy to set up / debug against. This has been a severe bottleneck in the past.