Document processing v2

.vscode/settings.json

@@ -75,5 +75,8 @@
   ],
   "editor.defaultFormatter": "esbenp.prettier-vscode",
   "editor.formatOnSave": true,
-  "search.useIgnoreFiles": false
+  "search.useIgnoreFiles": false,
+  "editor.rulers": [
+    80
+  ]
 }

docs/reference/ois/next/processing.md

@@ -19,6 +19,22 @@ tags:
 
 # {{$frontmatter.title}}
 
+Processing allows Airnode operators to define custom logic that executes before
+or after an API call. This feature is useful for multiple use cases, including:
+
+- Authentication
+- Data transformation
+- Data aggregation
+- Data validation
+
+## Processing versions
+
+OIS specification has two versions of pre/post processing. Both versions serve
+the same use cases, but the second version is more flexible and convenient.
+Users are encouraged to use the second version.
+
+### v1
+
 The processing schema accepts an array of processing snippets (user defined
 code) which are chained. The first snippet receives parameters submitted as part
 of a template or on-chain request. The output of this snippet is passed to the
@@ -51,7 +67,7 @@ Snippets for both specifications follow this schema:
 Try the [Post processing](/guides/airnode/post-processing/) guide to further
 understand pre/post processing.
 
-## Input and Output
+#### Input and Output
 
 The processing snippet receives an `input` value which is either the initial
 value or the output value from the previous processing snippet. The snippet must
@@ -61,7 +77,7 @@ source code of Airnode to understand how processing works and what modules are
 made available to the snippet code. Modules cannot be imported directly in cloud
 environments.
 
-## Accessing endpoint parameters
+#### Accessing endpoint parameters
 
 Endpoint parameters, with the exception of reserved parameters, are accessible
 within pre-processing and post-processing via the immutable `endpointParameters`
@@ -77,14 +93,84 @@ post-processing.
 
 :::
 
+### v2
+
+The processing snippet receives parameters submitted as part of a template or
+on-chain request.
+
+Airnode executes snippets for `preProcessingSpecificationV2` and
+`postProcessingSpecificationV2` during its run cycle. The following describes
+the work flow Airnode uses:
+
+1. Run `preProcessingSpecificationV2`
+2. Airnode calls requested OIS endpoint
+3. Run `postProcessingSpecificationV2`
+4. Airnode encodes the response values defined by reservedParameters
+
+The processing schema is the same for both
+[`preProcessingSpecificationV2`](/reference/ois/next/specification.md#_5-11-preprocessingspecificationv2)
+and
+[`postProcessingSpecificationV2`](/reference/ois/next/specification.md#_5-12-postprocessingspecificationv2).
+Snippets for both specifications follow this schema:
+
+- `environment` - Currently only possible value is `Node`. This options
+  interprets the code as JavaScript and execute in Node.js. The function can be
+  also asynchronous (async/await is supported as well). The processing
+  implementation will wait for the function to resolve.
+- `value` - The processing code written as a string.
+- `timeoutMs` - The maximum timeout that this snippet can run. In case the
+  timeout is exceeded an error is thrown.
+
+Try the [Post processing](/guides/airnode/post-processing/) guide to further
+understand pre/post processing.
+
+#### Input and Output
+
+The processing snippet is a function which receives payload as an argument. The
+return value of the function is treated as a processing result. Apart from the
+payload argument, you can use most Node.js built-in modules.
+
+The payload argument for pre-processing is an object with the following
+properties:
+
+- `apiCallParameters` - The API call parameters with the exception of reserved
+  parameters. For example, if there was a parameter named `myParameter` defined
+  in the `endpoints[n].parameters` array, its value could be accessed using
+  `endpointParameters.myParameter` within pre-processing snippet.
+
+The output of the pre-processing snippet is an object with the following fields:
+
+- `apiCallParameters` - The pre-processed API call parameters. These are used to
+  make the API call.
+
+The payload argument for post-processing is an object with the following
+properties:
+
+- `apiCallResponse` - The API call response.
+- `endpointParameters` - The API call parameters with the exception of reserved
+  parameters. For example, if there was a parameter named `myParameter` defined
+  in the `endpoints[n].parameters` array, its value could be accessed using
+  `endpointParameters.myParameter` within pre-processing snippet.
+
+The output of the post-processing snippet is an object with the following
+fields:
+
+- `apiCallResponse` - The post-processed API call response. This is used to
+  encode the response values defined by reservedParameters.
+- `timestamp` - (Optional) The timestamp of the API call response. Use this if
+  you want Airnode to use a specific timestamp (instead of a current time at
+  request processing) when using the [signed data
+  gateway](https://docs.api3.org/reference/airnode/latest/understand/http-gateways.html#http-signed-data-gateway).
+
 ## Interpolation
 
 Note, that config.json supports interpolation of secrets via the JavaScript
 string interpolation pattern (e.g `${SECRET_NAME}`). This syntax conflicts with
 the string interpolation inside the processing snippets. In order to use the
 interpolation in snippets, you need to escape the interpolation.
 
-For example, the following code:
+For example, the following code (using the v1 processing snippet, but the
+concept is the same for v2):
 
 ```js
 console.log(`Received input ${input}`);
@@ -125,8 +211,10 @@ Airnode to place on-chain.
 Instead of calling an API, Airnode uses the output of
 `preProcessingSpecifications`, `postProcessingSpecifications`, or both. The
 field `operation` must be undefined, `fixedOperationParameters` must be an empty
-array and one of `preProcessingSpecifications` or `postProcessingSpecifications`
-must be defined and not be an empty array.
+array and some processing specification needs to be defined. This means that one
+of `preProcessingSpecifications` or `postProcessingSpecifications` must be
+defined and not be an empty array or `preProcessingSpecificationV2` or
+`postProcessingSpecificationV2` must be defined.
 
 ### Use case: random number
 
@@ -245,4 +333,24 @@ endpoints: [
 ]
 ```
 
+## Example #3
+
+One of the possible use cases for post-processing would be to use override the
+timestamp used by [signed data
+gateway](https://docs.api3.org/reference/airnode/latest/understand/http-gateways.html#http-signed-data-gateway).
+By default the signed data gateway uses the timestamp of the request processing.
+However, sometimes the API itself returns the timestamp. Modifying timestamp is
+only supported with the v2 of the processing.
+
+```json
+{
+  "postProcessingSpecificationV2": {
+    "environment": "Node",
+    // Reuses the timestamp from the API call response.
+    "value": "async ({ apiCallResponse }) => { return { apiCallResponse, timestamp: apiCallResponse.timestamp }; }",
+    "timeoutMs": 5000,
+  }
+}
+```
+
 <FlexEndTag/>

docs/reference/ois/next/specification.md

@@ -327,6 +327,10 @@ node.
   [preProcessingSpecifications](/reference/ois/next/specification.md#_5-9-preprocessingspecifications)
 - 5.10.
   [postProcessingSpecifications](/reference/ois/next/specification.md#_5-10-postprocessingspecifications)
+- 5.11.
+  [preProcessingSpecificationV2](/reference/ois/next/specification.md#_5-11-preprocessingspecificationv2)
+- 5.12.
+  [postProcessingSpecificationV2](/reference/ois/next/specification.md#_5-12-postprocessingspecificationv2)
 
 ```json
 // endpoints
@@ -375,25 +379,16 @@ node.
         }
       }
     ],
-    "preProcessingSpecifications": [
-      {
-        "environment": "Node",
-        "value": "const output = {...input, from: \"eth\"};",
-        "timeoutMs": "5000"
-      },
-      {
-        "environment": "Node",
-        "value": "const output = {...input, from: input.from.toUpperCase()};",
-        "timeoutMs": "5000"
-      }
-    ],
-    "postProcessingSpecifications": [
-      {
-        "environment": "Node",
-        "value": "const output = Math.round(input.price * 1000);",
-        "timeoutMs": "5000"
-      }
-    ]
+    "preProcessingSpecificationV2": {
+      "environment": "Node",
+      "value": "({ apiCallParameters }) => { return { apiCallParameters: {...apiCallParameters, from: 'ETH'} }; }",
+      "timeoutMs": 5000
+    },
+    "postProcessingSpecificationV2": {
+      "environment": "Node",
+      "value": "({ apiCallResponse }) => { return { apiCallResponse: parseInt(apiCallResponse.price) * 1000 }; }",
+      "timeoutMs": 5000
+    }
   }
 ]
 ```
@@ -594,8 +589,16 @@ corresponding operation parameter.-->
 
 ### 5.9. `preProcessingSpecifications` \*
 
-(Optional) Defines the preprocessing code that can be used to modify the
-endpoint parameter before making the API request defined by an Airnode endpoint.
+::: deprecation warning
+
+The `preProcessingSpecifications` field is deprecated. Use
+`preProcessingSpecificationV2` instead.
+
+:::
+
+(Optional) Defines the pre-processing code that can be used to modify the
+endpoint parameters before making the API request defined by an Airnode
+endpoint.
 
 See the [Pre/Post Processing](/reference/ois/next/processing.md) doc for
 additional details.
@@ -610,21 +613,28 @@ additional details.
     // Define a new "from" parameter with value "eth"
     "value": "const output = {...input, from: \"eth\"};",
     // Run for 5 seconds maximum
-    "timeoutMs": "5000"
+    "timeoutMs": 5000
   },
   {
     // Execute synchronously in Node.js
     "environment": "Node",
     // Uppercase the "from" parameter defined by the previous snippet
     "value": "const output = {...input, from: input.from.toUpperCase()};",
     // Run for 5 seconds maximum
-    "timeoutMs": "5000"
+    "timeoutMs": 5000
   }
 ]
 ```
 
 ### 5.10. `postProcessingSpecifications` \*
 
+::: deprecation warning
+
+The `postProcessingSpecifications` field is deprecated. Use
+`postProcessingSpecificationV2` instead.
+
+:::
+
 (Optional) Defines the post-processing code that can be used to modify the API
 response from the request defined by an Airnode endpoint.
 
@@ -641,7 +651,52 @@ additional details.
     // Multiply the API return value by 1000 and round it to an integer
     "value": "const output = Math.round(input.price * 1000);",
     // Run for 5 seconds maximum
-    "timeoutMs": "5000"
+    "timeoutMs": 5000
+  }
+]
+```
+
+### 5.11. `preProcessingSpecificationV2` \*
+
+(Optional) Defines the pre-processing code that can be used to modify the
+endpoint parameters before making the API request defined by an Airnode
+endpoint.
+
+See the [Pre/Post Processing](/reference/ois/next/processing.md) doc for
+additional details.
+
+#### Example
+
+```json
+"preProcessingSpecificationV2": {
+  // Execute in Node.js. The v2 specification supports both synchronous and asynchronous code
+  "environment": "Node",
+  // Define a new "from" parameter with value "ETH"
+  "value": "({ apiCallParameters }) => { return { apiCallParameters: {...apiCallParameters, from: 'ETH'} }; }",
+  // Run for 5 seconds maximum
+  "timeoutMs": 5000
+}
+```
+
+### 5.12. `postProcessingSpecificationV2` \*
+
+(Optional) Defines the post-processing code that can be used to modify the API
+response from the request defined by an Airnode endpoint.
+
+See the [Pre/Post Processing](/reference/ois/next/processing.md) doc for
+additional details.
+
+#### Example
+
+```json
+"postProcessingSpecificationV2": [
+  {
+    // Execute in Node.js. The v2 specification supports both synchronous and asynchronous code
+    "environment": "Node",
+    // Multiply the API return value by 1000 and round it to an integer
+    "value": "({ apiCallResponse }) => { return { apiCallResponse: parseInt(apiCallResponse.price * 1000) }; }",
+    // Run for 5 seconds maximum
+    "timeoutMs": 5000
   }
 ]
 ```