-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(blog): add blog on setting up a clojure lambda + calling it from…
… HTTP
- Loading branch information
1 parent
6dbba42
commit ed96ff5
Showing
1 changed file
with
184 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,184 @@ | ||
--- | ||
author: 'osm' | ||
layout: '../../layouts/BlogPost.astro' | ||
title: 'Fast clojure lambdas + calling from HTTP' | ||
description: '...' | ||
category: 'clojure' | ||
publishedDate: '2024-05-16' | ||
tags: | ||
- clojure | ||
- lambda | ||
- aws | ||
- 'api gateway' | ||
heroImage: 'xt24-stage.jpg' | ||
--- | ||
|
||
Have you heard the good news? | ||
[SnapStart](https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html) will make your clojure lambda's lightning quick! | ||
|
||
In this article we'll get a basic clojure lambda setup. | ||
|
||
We'll then talk about how to call the lambda via [function URLs](https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html), [CloudFront]() and finally [API Gateway](https://aws.amazon.com/api-gateway/). | ||
|
||
## Clojure lambda | ||
|
||
Let's get started with the code: | ||
|
||
```clojure | ||
(ns lambda | ||
(:gen-class | ||
:implements [com.amazonaws.services.lambda.runtime.RequestStreamHandler])) | ||
|
||
(defn -handleRequest [_ is os _context] | ||
(let [input (slurp is)] | ||
(if (re-find #"error" input) | ||
(throw (ex-info "test" {:my "error"})) | ||
(spit os "test")))) | ||
``` | ||
|
||
And that's really all you need to get going, (other than being able to [build a jar](https://clojure.org/guides/tools_build#_compiled_uberjar_application_build) and [deploy the lambda](https://docs.aws.amazon.com/lambda/latest/dg/java-package.html#java-package-console) but I'll leave that to you). | ||
|
||
{/* TODO: Add example project */} | ||
{/* TODO: Mention bumping the memory limit to increase the available CPU */} | ||
|
||
When I test the lambda in the console I get a duration of 1.76ms, not bad at all! | ||
|
||
To get it running this fast all I had to do was [turn on SnapStart](https://docs.aws.amazon.com/lambda/latest/dg/snapstart-activate.html) then [publish function version](https://docs.aws.amazon.com/lambda/latest/dg/configuration-versions.html#configuration-versions-config). Just make sure to test vs the version you created rather than the base lambda! Also note that the first request might be a tad slower for some reason 🤷. | ||
|
||
But once you have your lambda, you probably want to interact with it. | ||
|
||
## Function URLs | ||
|
||
A handy feature to quickly be able to interact with a lambda. | ||
|
||
For clojure because we need SnapStart, we have to for an alias that points to a version. | ||
|
||
To do this just [create an alias](https://docs.aws.amazon.com/lambda/latest/dg/configuration-aliases.html#configuration-aliases-config) pointing to the version you just created, then [create a function URL for that alias](https://docs.aws.amazon.com/lambda/latest/dg/urls-configuration.html#create-url-existing-alias). | ||
|
||
Once you have one you can just make requests against it like you would normally: | ||
|
||
```bash | ||
curl -XPOST --data 'some data' <your url> | ||
``` | ||
|
||
Function URLs will put a JSON formatted request into the input stream in [this format](https://docs.aws.amazon.com/lambda/latest/dg/urls-invocation.html#urls-request-payload). They also allow you to format your results into [this JSON format](https://docs.aws.amazon.com/lambda/latest/dg/urls-invocation.html#urls-response-payload), but if you don't use that format it will infer a format for you (including the response code being 200, and the body being JSON!). | ||
|
||
If you throw an uncaught exception it is translated into a 502 with `Internal server error` as the response body (I wasn't able to find a link to the docs for this, trying it out gave me this answer). | ||
|
||
But if you want a custom URL then this isn't enough, you need to put either CloudFront or API Gateway in front of the lambda. | ||
|
||
## CloudFront | ||
|
||
This is the thinnest option (i.e. least between your request and the lambda). | ||
|
||
I'll go ahead assuming that you've setup a [domain in Route 53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.html#domain-register-procedure-section) and a [certificate in ACM](https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html). | ||
|
||
First step is to create a Function URL [as above](#function-urls). Next you can [create a CloudFront distribution](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-creating-console.html) use the lambda URL as the origin (without the https:// bit), make sure to allow all the HTTP methods you want to support (it defaults to just GET & HEAD), decide if you want caching or WAF (it's fine to say no to both) and set the alternate domain (it can be a subdomain) w/ the associated certificate. After you've done this [create an A record alias](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-creating.html) in Route53 pointing to your new distribution's URL (check the 'alias' checkbox). Remember to wait until the distribution has finished deploying before testing! | ||
|
||
Note that because we're using a Function URL it has the same usage pattern, i.e. a JSON request will be provided in the specified format and you can specify the response in JSON. | ||
|
||
Also note that the lambda is the only thing on the other end of that particular domain. If you want to use a sub-path (say https://example.com/api/my-function) to call your function and free up the other sub paths for other lambdas or other services entirely you have two options: Code all that yourself (not a bad option) and API Gateway. | ||
|
||
## API Gateway | ||
|
||
API Gateway is a bit of a different beast as you'll soon see, but it's very flexible and can allow you to abstract some things like [Authentication](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-control-access-to-api.html) and [request validation](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-method-request-validation.html). | ||
|
||
The first thing to know is there are **three**(?) ways to setup a lambda: | ||
|
||
[With the HTTP API](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop.html) during setup point it at your lambda (no Function URL needed!) (also make sure to point it at your alias) with the method & resource path you want and off you go! Note that it **really** doesn't like the response not being json and will return a 500 with `{"message":"Internal Server Error"}. | ||
|
||
With a REST API first [create a new api](https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-rest-new-console.html) using the default lambda integration (i.e. don't check the `Lambda proxy integration` box during setup) (again make sure to use the alias). The thing to note here is that with the default integration, API Gateway is calling the raw `Invoke` request of the Lambda API. This invoke call will return a 200 response even if your lambda throws an uncaught exception(!) and so will your AWS Gateway method. Luckily API Gateway has a way around this, you can add a new `Method Response` with the status code you want to respond with (say `400` for invalid user input), next you add a new `Integration Response` (you can leave the default as it is) set the `Method response status code` to the one you just created and set the `Lambda error regex` to match the error message you want to catch (you can use `(\n|.)+` to match all errors). Note that this error regex will [match against the `errorMessage` key in the **JSON** response from your lambda](https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-method-settings-execution-console.html). Also note that if you throw an uncaught exception it will be formatted [like so](https://docs.aws.amazon.com/apigateway/latest/developerguide/handle-errors-in-lambda-integration.html) (which helpfully includes the `exceptionMessage` field). | ||
|
||
{/* TODO: mention that the above **also** really don't like non-json requests and will return a 400 with `{"message": "Could not parse request body into json:<snip>"}` */} | ||
{/* TODO: Remind people to deploy their API to a stage after making changes (no stage didn't work for me) */} | ||
|
||
And last but certainly not least is the same REST API as above but with the Lambda proxy integration. To create just follow the [same guide](https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-rest-new-console.html) but this time **do** check the `lambda proxy integration` box during setup. Now API Gateway will pass the input as a JSON object with [this format](https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-input-format) and expects a response with [this format](https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-output-format). Note that now the input **does not** have to be valid json as this will just be put in the request data. Also note that the output **does** have to be valid json, if it's not you'll get a 502 with `{"message": "Internal server error"}` | ||
|
||
{/* | ||
AWS Lambda will *always* return a 200 status code, even if an exception is thrown: | ||
https://docs.aws.amazon.com/lambda/latest/dg/java-exceptions.html | ||
(Well, except for rate limiting, invalid json etc.) | ||
So if that's fine then you can just: | ||
- Create a lambda function URL | ||
- Create a CloudFront distribution | ||
- Use the lambda URL as the origin (it'll remove the https:// bit for you) | ||
- Make sure to enable all the HTTP Methods you want (the others will give you errors) | ||
- Decide if you need caching or a WAF (I said no to both) | ||
- Setup the alternate domain w/ the certificate | ||
- Add an A record (using an alias) in Route53 pointing to your distribution's url | ||
Off you go! (I think) | ||
If you want more you need to look at API Gateway: | ||
In API Gateway there are **two ways**(!) to setup a lambda: | ||
- Default Lambda Integration | ||
- API Gateway gets the response from the AWS Lambda API which **contains** your response | ||
- So AWS Lambda returns a 200 and by default API Gateway sees this and also returns 200 | ||
- To configure API Gateway to return different responses: | ||
- Add a Method Response for the return code you want to use | ||
- Add an Integration Response | ||
- Set the 'Method response status code' you just created | ||
- Set the 'Lambda error regex' to match the error message you want to catch | ||
- You can use '(\n|.)+' to catch all errors or use custom regex | ||
- This matches against the **'errorMessage'** key in the **JSON Response** | ||
- Not the whole response, or any other part of the response! | ||
- https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-method-settings-execution-console.html | ||
- Lambda **Proxy** Integration | ||
- Gets the full request in the context parameter | ||
- This format: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-input-format | ||
- Expects the following format for response: | ||
- https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-output-format | ||
While testing the code I used was: | ||
```clojure | ||
(ns lambda | ||
(:gen-class | ||
:implements [com.amazonaws.services.lambda.runtime.RequestStreamHandler])) | ||
(defn -handleRequest [_ is os context] | ||
(let [input (slurp is)] | ||
(println "input:" input) | ||
(println "context:" (bean context)) | ||
(if (re-find #"error" input) | ||
(throw (ex-info "test" {:my "error"})) | ||
;; Note this is valid JSON | ||
(spit os "\"test\"")))) | ||
``` | ||
This allowed me to see the input and context (the context was basically always the same). | ||
For most configurations I tested three different scenarios: | ||
Valid json in the body: | ||
`curl -v --data '{}' -XPOST <url under test>` | ||
Invalid json in the body: | ||
`curl -v --data '{' -XPOST <url under test>` | ||
Throwing an uncaught exception: | ||
`curl -v --data '{}' -H 'Test: error' -XPOST <url under test>` | ||
An alternative layout for this article might be: | ||
1. How to setup a fast clojure lambda | ||
- SnapStart + alias + bump memory | ||
2. Compare the different ways to get HTTP to the lambda | ||
For each: | ||
- Small introduction to why you'd use it | ||
- Links to how to set it up (like the current paragraphs) | ||
- A table like below comparing the results of the above queries | ||
| Query | Status code | Response | `input` in lambda | | ||
| --- | --- | --- | --- | | ||
| Valid json in body | 200 | `"test"` | `{}` | | ||
| Invalid json in body | 502 | `{"message": "Internal server error"}` | - | | ||
| Throwing an uncaught exception | 502 | `{"message": "Internal server error"}` | - | | ||
Maybe add `Valid json in lambda response` and `Invalid json in lambda response`? | ||
*/} |