diff --git a/.vscode/settings.json b/.vscode/settings.json index ee4e98e3..89ddf7e2 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -75,5 +75,8 @@ ], "editor.defaultFormatter": "esbenp.prettier-vscode", "editor.formatOnSave": true, - "search.useIgnoreFiles": false + "search.useIgnoreFiles": false, + "editor.rulers": [ + 80 + ] } diff --git a/docs/reference/ois/2.0/processing.md b/docs/reference/ois/2.0/processing.md index 9fb62b27..b8dcfcc3 100644 --- a/docs/reference/ois/2.0/processing.md +++ b/docs/reference/ois/2.0/processing.md @@ -45,7 +45,7 @@ Snippets for both specifications follow this schema: with the output value as an argument. Airnode will use the resolved value as the input to subsequent snippets (if defined). - `value` - The processing code written as a string. -- `timeoutMs` - The maximum timeout that this snippet can run. In case the +- `timeoutMs` - The maximum duration that this snippet can run. In case the timeout is exceeded an error is thrown. Try the [Post processing](/guides/airnode/post-processing/) guide to further diff --git a/docs/reference/ois/2.1/processing.md b/docs/reference/ois/2.1/processing.md index 177d190f..7a7c729b 100644 --- a/docs/reference/ois/2.1/processing.md +++ b/docs/reference/ois/2.1/processing.md @@ -45,7 +45,7 @@ Snippets for both specifications follow this schema: with the output value as an argument. Airnode will use the resolved value as the input to subsequent snippets (if defined). - `value` - The processing code written as a string. -- `timeoutMs` - The maximum timeout that this snippet can run. In case the +- `timeoutMs` - The maximum duration that this snippet can run. In case the timeout is exceeded an error is thrown. Try the [Post processing](/guides/airnode/post-processing/) guide to further diff --git a/docs/reference/ois/latest/processing.md b/docs/reference/ois/latest/processing.md index a920793a..81e074ba 100644 --- a/docs/reference/ois/latest/processing.md +++ b/docs/reference/ois/latest/processing.md @@ -45,7 +45,7 @@ Snippets for both specifications follow this schema: with the output value as an argument. Airnode will use the resolved value as the input to subsequent snippets (if defined). - `value` - The processing code written as a string. -- `timeoutMs` - The maximum timeout that this snippet can run. In case the +- `timeoutMs` - The maximum duration that this snippet can run. In case the timeout is exceeded an error is thrown. Try the [Post processing](/guides/airnode/post-processing/) guide to further diff --git a/docs/reference/ois/next/processing.md b/docs/reference/ois/next/processing.md index eef56484..3f273ca4 100644 --- a/docs/reference/ois/next/processing.md +++ b/docs/reference/ois/next/processing.md @@ -19,6 +19,23 @@ tags: # {{$frontmatter.title}} +Processing allows Airnode operators to define custom logic that executes before +or after an API call. This feature is useful for multiple use cases, including: + +- Authentication +- Data transformation +- Data aggregation +- Data validation +- Skipping an API call + +## Processing versions + +OIS specification has two versions of pre/post processing. Both versions serve +the same use cases, but the second version is more flexible and convenient. +Users are encouraged to use the second version. + +### v1 + The processing schema accepts an array of processing snippets (user defined code) which are chained. The first snippet receives parameters submitted as part of a template or on-chain request. The output of this snippet is passed to the @@ -29,9 +46,9 @@ Airnode executes snippets for `preProcessingSpecifications` and work flow Airnode uses: 1. Run `preProcessingSpecifications` -2. Airnode calls requested OIS endpoint +2. Airnode calls requested OIS endpoint (unless the API call is skipped) 3. Run `postProcessingSpecifications` -4. Airnode encodes the response values defined by reservedParameters +4. Airnode encodes the response values defined by `reservedParameters` The processing schema is the same for both [`preProcessingSpecifications`](/reference/ois/next/specification.md#_5-9-preprocessingspecifications) @@ -45,13 +62,13 @@ Snippets for both specifications follow this schema: with the output value as an argument. Airnode will use the resolved value as the input to subsequent snippets (if defined). - `value` - The processing code written as a string. -- `timeoutMs` - The maximum timeout that this snippet can run. In case the +- `timeoutMs` - The maximum duration that this snippet can run. In case the timeout is exceeded an error is thrown. Try the [Post processing](/guides/airnode/post-processing/) guide to further understand pre/post processing. -## Input and Output +#### Input and Output The processing snippet receives an `input` value which is either the initial value or the output value from the previous processing snippet. The snippet must @@ -61,7 +78,7 @@ source code of Airnode to understand how processing works and what modules are made available to the snippet code. Modules cannot be imported directly in cloud environments. -## Accessing endpoint parameters +#### Accessing endpoint parameters Endpoint parameters, with the exception of reserved parameters, are accessible within pre-processing and post-processing via the immutable `endpointParameters` @@ -77,6 +94,80 @@ post-processing. ::: +### v2 + +The processing snippet receives parameters submitted as part of a template or +on-chain request. + +Airnode executes snippets for `preProcessingSpecificationV2` and +`postProcessingSpecificationV2` during its run cycle. The following describes +the work flow Airnode uses: + +1. Run `preProcessingSpecificationV2` +2. Airnode calls requested OIS endpoint (unless the API call is skipped) +3. Run `postProcessingSpecificationV2` +4. Airnode encodes the response values defined by `reservedParameters` + +The processing schema is the same for both +[`preProcessingSpecificationV2`](/reference/ois/next/specification.md#_5-11-preprocessingspecificationv2) +and +[`postProcessingSpecificationV2`](/reference/ois/next/specification.md#_5-12-postprocessingspecificationv2). +Snippets for both specifications follow this schema: + +- `environment` - Currently only possible value is `Node`. This options + interprets the code as JavaScript and executes it in Node.js. The function can + be also asynchronous (async/await is supported as well). The processing + implementation will wait for the function to resolve. +- `value` - The processing code written as a string. +- `timeoutMs` - The maximum duration that this snippet can run. In case the + timeout is exceeded an error is thrown. + +Try the [Post processing](/guides/airnode/post-processing/) guide to further +understand pre/post processing. + +#### Input and Output + +The processing snippet is a function which receives a payload as an argument. +The return value of the function is treated as a processing result. Apart from +the payload argument, you can use most Node.js built-in modules. + +The payload argument for pre-processing is an object with the following +property: + +- `endpointParameters` - The endpoint parameters with the exception of reserved + parameters. For example, if there was a parameter named `myParameter` defined + in the `endpoints[n].parameters` array, its value could be accessed using + `endpointParameters.myParameter` within pre-processing snippet. + +The output of the pre-processing snippet is an object with the following +property: + +- `endpointParameters` - The pre-processed endpoint parameters parameters. These + are used to make the API call. + +The payload argument for post-processing is an object with the following +properties: + +- `response` - The response of the underlying data provider API call. In case of + Airnode skipping the API call, the `response` contains the output of + pre-processing snippet. +- `endpointParameters` - The raw (not pre-processed) endpoint parameters with + the exception of reserved parameters. For example, if there was a parameter + named `myParameter` defined in the `endpoints[n].parameters` array, its value + could be accessed using `endpointParameters.myParameter` within pre-processing + snippet. + +The output of the post-processing snippet is an object with the following +properties: + +- `response` - The post-processed API call response (or post-processed result of + pre-processing snippet in case of skipping an API call). This is used to + encode the response values defined by reserved parameters. +- `timestamp` - (Optional) The timestamp of the API call response. Use this if + you want Airnode to use a specific timestamp (instead of a current time at + request processing) when using the [signed data + gateway](/reference/airnode/latest/understand/http-gateways.md#http-signed-data-gateway). + ## Interpolation Note, that config.json supports interpolation of secrets via the JavaScript @@ -84,7 +175,8 @@ string interpolation pattern (e.g `${SECRET_NAME}`). This syntax conflicts with the string interpolation inside the processing snippets. In order to use the interpolation in snippets, you need to escape the interpolation. -For example, the following code: +For example, the following code (using the v1 processing snippet, but the +concept is the same for v2): ```js console.log(`Received input ${input}`); @@ -125,8 +217,10 @@ Airnode to place on-chain. Instead of calling an API, Airnode uses the output of `preProcessingSpecifications`, `postProcessingSpecifications`, or both. The field `operation` must be undefined, `fixedOperationParameters` must be an empty -array and one of `preProcessingSpecifications` or `postProcessingSpecifications` -must be defined and not be an empty array. +array and some processing specification needs to be defined. This means that one +of `preProcessingSpecifications` or `postProcessingSpecifications` must be +defined and not be an empty array or `preProcessingSpecificationV2` or +`postProcessingSpecificationV2` must be defined. ### Use case: random number @@ -245,4 +339,24 @@ endpoints: [ ] ``` +## Example #3 + +One of the possible use cases for post-processing would be to use override the +timestamp used by [signed data +gateway](/reference/airnode/latest/understand/http-gateways.md#http-signed-data-gateway). +By default the signed data gateway uses the timestamp of the request processing. +However, sometimes the API itself returns the timestamp. Modifying timestamp is +only supported with the v2 of the processing. + +```json +{ + "postProcessingSpecificationV2": { + "environment": "Node", + // Reuses the timestamp from the API call response. + "value": "async ({ response }) => { return { response, timestamp: response.timestamp }; }", + "timeoutMs": 5000, + } +} +``` + diff --git a/docs/reference/ois/next/specification.md b/docs/reference/ois/next/specification.md index a1000ba4..56cc7d68 100644 --- a/docs/reference/ois/next/specification.md +++ b/docs/reference/ois/next/specification.md @@ -327,6 +327,10 @@ node. [preProcessingSpecifications](/reference/ois/next/specification.md#_5-9-preprocessingspecifications) - 5.10. [postProcessingSpecifications](/reference/ois/next/specification.md#_5-10-postprocessingspecifications) +- 5.11. + [preProcessingSpecificationV2](/reference/ois/next/specification.md#_5-11-preprocessingspecificationv2) +- 5.12. + [postProcessingSpecificationV2](/reference/ois/next/specification.md#_5-12-postprocessingspecificationv2) ```json // endpoints @@ -375,25 +379,16 @@ node. } } ], - "preProcessingSpecifications": [ - { - "environment": "Node", - "value": "const output = {...input, from: \"eth\"};", - "timeoutMs": "5000" - }, - { - "environment": "Node", - "value": "const output = {...input, from: input.from.toUpperCase()};", - "timeoutMs": "5000" - } - ], - "postProcessingSpecifications": [ - { - "environment": "Node", - "value": "const output = Math.round(input.price * 1000);", - "timeoutMs": "5000" - } - ] + "preProcessingSpecificationV2": { + "environment": "Node", + "value": "({ endpointParameters }) => { return { endpointParameters: {...endpointParameters, from: 'ETH'} }; }", + "timeoutMs": 5000 + }, + "postProcessingSpecificationV2": { + "environment": "Node", + "value": "({ response }) => { return { response: parseInt(response.price) * 1000 }; }", + "timeoutMs": 5000 + } } ] ``` @@ -594,8 +589,16 @@ corresponding operation parameter.--> ### 5.9. `preProcessingSpecifications` \* -(Optional) Defines the preprocessing code that can be used to modify the -endpoint parameter before making the API request defined by an Airnode endpoint. +::: warning Deprecation + +The `preProcessingSpecifications` field is deprecated. Use +`preProcessingSpecificationV2` instead. + +::: + +(Optional) Defines the pre-processing code that can be used to modify the +endpoint parameters before making the API request defined by an Airnode +endpoint. See the [Pre/Post Processing](/reference/ois/next/processing.md) doc for additional details. @@ -610,7 +613,7 @@ additional details. // Define a new "from" parameter with value "eth" "value": "const output = {...input, from: \"eth\"};", // Run for 5 seconds maximum - "timeoutMs": "5000" + "timeoutMs": 5000 }, { // Execute synchronously in Node.js @@ -618,13 +621,20 @@ additional details. // Uppercase the "from" parameter defined by the previous snippet "value": "const output = {...input, from: input.from.toUpperCase()};", // Run for 5 seconds maximum - "timeoutMs": "5000" + "timeoutMs": 5000 } ] ``` ### 5.10. `postProcessingSpecifications` \* +::: warning Deprecation + +The `postProcessingSpecifications` field is deprecated. Use +`postProcessingSpecificationV2` instead. + +::: + (Optional) Defines the post-processing code that can be used to modify the API response from the request defined by an Airnode endpoint. @@ -641,7 +651,52 @@ additional details. // Multiply the API return value by 1000 and round it to an integer "value": "const output = Math.round(input.price * 1000);", // Run for 5 seconds maximum - "timeoutMs": "5000" + "timeoutMs": 5000 + } +] +``` + +### 5.11. `preProcessingSpecificationV2` \* + +(Optional) Defines the pre-processing code that can be used to modify the +endpoint parameters before making the API request defined by an Airnode +endpoint. + +See the [Pre/Post Processing](/reference/ois/next/processing.md) doc for +additional details. + +#### Example + +```json +"preProcessingSpecificationV2": { + // Execute in Node.js. The v2 specification supports both synchronous and asynchronous code + "environment": "Node", + // Define a new "from" parameter with value "ETH" + "value": "({ endpointParameters }) => { return { endpointParameters: {...endpointParameters, from: 'ETH'} }; }", + // Run for 5 seconds maximum + "timeoutMs": 5000 +} +``` + +### 5.12. `postProcessingSpecificationV2` \* + +(Optional) Defines the post-processing code that can be used to modify the API +response from the request defined by an Airnode endpoint. + +See the [Pre/Post Processing](/reference/ois/next/processing.md) doc for +additional details. + +#### Example + +```json +"postProcessingSpecificationV2": [ + { + // Execute in Node.js. The v2 specification supports both synchronous and asynchronous code + "environment": "Node", + // Multiply the API return value by 1000 and round it to an integer + "value": "({ response }) => { return { response: parseInt(response.price * 1000) }; }", + // Run for 5 seconds maximum + "timeoutMs": 5000 } ] ```