From 80544cb999891e0b1cf38a7b385f1b5e2855ed4a Mon Sep 17 00:00:00 2001 From: Andrew Nowak Date: Mon, 25 Nov 2024 13:50:17 +0000 Subject: [PATCH 1/2] pluto-message-ingestion: remove externals from ncc config --- pluto-message-ingestion/ncc.config.json | 4 ---- 1 file changed, 4 deletions(-) diff --git a/pluto-message-ingestion/ncc.config.json b/pluto-message-ingestion/ncc.config.json index 8f497cea0..3dc1021aa 100644 --- a/pluto-message-ingestion/ncc.config.json +++ b/pluto-message-ingestion/ncc.config.json @@ -1,7 +1,3 @@ { - "externals": { - "aws-sdk": "aws-sdk", - "/aws-sdk(/.*)/": "aws-sdk$1" - }, "minify": true } From 7ef76fc25c3f1b690da5fc471c49628c987cea0f Mon Sep 17 00:00:00 2001 From: Andrew Nowak Date: Mon, 25 Nov 2024 17:00:07 +0000 Subject: [PATCH 2/2] include brief testing instructions for the pluto message ingestion lambda --- pluto-message-ingestion/README.md | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/pluto-message-ingestion/README.md b/pluto-message-ingestion/README.md index 1498b3bb1..9888139ae 100644 --- a/pluto-message-ingestion/README.md +++ b/pluto-message-ingestion/README.md @@ -2,13 +2,32 @@ An AWS Lambda that provides integration with [Pluto](https://github.com/guardian/pluto) via a Kinesis stream. -Pluto and MAM do not communicate directly as the AWS VPCs are not setup. Instead, they communicate via a Kinesis stream. +Pluto and MAM do not communicate directly as the AWS VPCs are not setup. Instead, they communicate via a Kinesis stream. Pluto puts JSON messages on this stream and this Lambda picks them up. The Lambda switches on `message.type` and calls the relevant MAM end-point. ## Message types processed +> [!INFO] +> To test the lambda in CODE, instead of sending messages on the stream +> you can create a test event in the Lambda console. +> Start by creating a message in one of the formats below, then base64-encode +> it, then insert that encoded message into an event that looks like: +> ```json +> { +> "Records": [ +> { +> "kinesis": { +> "data": "[your encoded message]" +> } +> } +> ] +> } +> ``` +> Your fake Pluto project and commission should now be sent to MAM and stored +> in the `media-atom-maker-CODE-pluto-[projects/commissions]-table`s + ### `project-created` Pluto will send a message to Kinesis when a new Project has been created. MAM will then store a copy of the metadata in Dynamo. @@ -49,15 +68,15 @@ Pluto sends a JSON blob that looks like: ### Updating pre-existing projects If an old Pluto Project is updated, a `project-updated` message still gets put on the Kinesis stream. -As MAM has not seen this project before, it does not know what to update and thus the message is discarded, +As MAM has not seen this project before, it does not know what to update and thus the message is discarded, logging `attempted to update a project that does not exist`. -This means only projects created **after** the Pluto and MAM changes have gone out can be selected as a destination -for Direct Upload. +This means only projects created **after** the Pluto and MAM changes have gone out can be selected as a destination +for Direct Upload. ### Unknown message types If a message is put on the Kinesis stream that is not recognised, it will be logged and the lambda will succeed. -This is because Kinesis blocks on failure; succeeding takes the message off the stream so we can continue with +This is because Kinesis blocks on failure; succeeding takes the message off the stream so we can continue with remaining messages. See [http://docs.aws.amazon.com/lambda/latest/dg/retries-on-errors.html](http://docs.aws.amazon.com/lambda/latest/dg/retries-on-errors.html) for more info.