From 618d516ea387ad2ad614cfc55b966233a3cc213f Mon Sep 17 00:00:00 2001 From: t-burch <119930761+t-burch@users.noreply.github.com> Date: Wed, 19 Jul 2023 15:18:55 +0200 Subject: [PATCH 1/6] Validation example rewrite (#670) * Added missing readme for the validation/form sample * Minor changes to the validation/form sample reamde * Updated the JSON schema validation readme and proxies xml * Updated the JSON schema validation readme and proxies xml * Updated the JSON schema validation readme and proxies xml * Updated the JSON schema validation readme * Updated the soap validation readme * Minute change in formatting in the json schema validation readme * Minute change in formatting in the soap validation readme * Updated the XML schema validation readme and proxies xml * Rephrasing in the soap validation readme --- .../examples/validation/form/README.md | 43 ++++++++++++++++ .../examples/validation/json-schema/README.md | 51 ++++++++++++++----- .../validation/json-schema/proxies.xml | 12 ++--- .../examples/validation/soap-Proxy/README.md | 43 ++++++++-------- .../examples/validation/xml/README.md | 41 +++++++++++++-- .../examples/validation/xml/proxies.xml | 8 +-- 6 files changed, 148 insertions(+), 50 deletions(-) create mode 100644 distribution/examples/validation/form/README.md diff --git a/distribution/examples/validation/form/README.md b/distribution/examples/validation/form/README.md new file mode 100644 index 000000000..c0ebc851f --- /dev/null +++ b/distribution/examples/validation/form/README.md @@ -0,0 +1,43 @@ +# Validation - Forms + +This sample explains how to set up and use the `formValidation` plugin. + + +## Running the Example + +1. Navigate to the `/examples/validation/form` directory. + + +2. Execute the `service-proxy.sh` script, or its batch file equivalent. + + +3. Use the following `curl` command in a terminal to send valid form data to Membrane: + `curl -o /dev/null -s -w "%{http_code}\n" -X POST -d "name=JohnSmith" http://localhost:2000` + The command returns the status code 200 "Ok", indicating that the request was successful. + + +4. Now send some invalid data to see how the system responds. Run the following curl command, which includes digits in the form data (making it invalid as per the rules defined in the `proxies.xml` file): + `curl -o /dev/null -s -w "%{http_code}\n" -X POST -d "name=JohnSmith1234" http://localhost:2000` + This time, we receive the status code 400, denoting a "Bad Request". + +## How it is done + +Let's examine the `proxies.xml` file. + +```xml + + + + + + + + +``` + +We define an `` component on port 2000 that uses the `formValidation` plugin. +By adding child elements to the plugin, we establish the necessary validation rules for the form. Each field specified is associated with a regex pattern that acts as the validation rule. + +--- +See: +- []() reference \ No newline at end of file diff --git a/distribution/examples/validation/json-schema/README.md b/distribution/examples/validation/json-schema/README.md index bead2e950..5d9015f52 100644 --- a/distribution/examples/validation/json-schema/README.md +++ b/distribution/examples/validation/json-schema/README.md @@ -1,11 +1,11 @@ -### JSON Schema Validation +# Validation - JSON Schema -To run this example you should install Curl from http://curl.haxx.se/download.html , if -you have not done so already. +This sample explains how to set up and use the `validator` plugin, utilizing JSON schemas for validation. -To run this example execute the following steps: -1. Go to the directory `examples/validation/json-schema`. +## Running the Example + +1. Go to the directory `/examples/validation/json-schema`. 2. Start `service-proxy.bat` or `service-proxy.sh`. @@ -15,8 +15,6 @@ To run this example execute the following steps: 5. Run `curl -d @bad2000.json http://localhost:2000/`. Observe that you get a validation error response. - - Keeping the router running, you can try a more complex schema. 1. Have a look at `schema2001.json`, `good2001.json` and `bad2001.json`. @@ -25,14 +23,39 @@ Keeping the router running, you can try a more complex schema. 3. Run `curl -d @bad2001.json http://localhost:2001/`. Observe that you get a validation error response. - - -Resources: - http://tools.ietf.org/html/draft-zyp-json-schema-03 - -(The file schema2001.json is loosely based on chapter 3 of -http://tools.ietf.org/html/draft-zyp-json-schema-03 .) +## How it is done + +Let's examine the `proxies.xml` file. + +```xml + + + + + + + + + + + + + + + + + + Response.ok("<response>good request</response>").build() + + + +``` + +We define three `` components, running on ports 2000-2002. +The first two validate all requests using the JSON schema defined in the `` component's `jsonSchema` attribute. +The successfully validated requests then get sent to the third `` component, where we simply return a 200 "Ok" response. --- See: +- [JSON Schema](https://json-schema.org/) documentation - [validator](https://membrane-soa.org/api-gateway-doc/current/configuration/reference/validator.htm) reference \ No newline at end of file diff --git a/distribution/examples/validation/json-schema/proxies.xml b/distribution/examples/validation/json-schema/proxies.xml index 2f3a1c070..e762e0d76 100644 --- a/distribution/examples/validation/json-schema/proxies.xml +++ b/distribution/examples/validation/json-schema/proxies.xml @@ -6,25 +6,25 @@ - + - + - + - + - + Response.ok("<response>good request</response>").build() - + diff --git a/distribution/examples/validation/soap-Proxy/README.md b/distribution/examples/validation/soap-Proxy/README.md index 53b0bf9ec..56d0a22f0 100644 --- a/distribution/examples/validation/soap-Proxy/README.md +++ b/distribution/examples/validation/soap-Proxy/README.md @@ -1,36 +1,37 @@ -### SOAP Message Validation +# Validation - SOAP -To run this example execute the following steps +This sample explains how to set up and use the `validator` plugin within a `soapProxy` component. -#### PREREQUISITES: -- Install Curl from http://curl.haxx.se/download.html , if you have not done so already. Let us assume it is in your PATH. +## Running the Example -As the URL of a WSDL is specified in proxies.xml, the Service Proxy retrieves all corresponding schemas and tries to validate the message body using them. +1. Go to `/examples/validation/soap-Proxy` -As a SOAP-Proxy is a specialized version of `the ServiceProxy`, a SOAP-Proxy has a certain advantage if it is used for a SOAP Service. A SOAP-Proxy needs the WSDL's URL just once. Adding a Validator then needs less code, as you can see in the example below: -``` - - - -``` +2. Start `service-proxy.bat` or `service-proxy.sh` -Execute the following steps: -1. Go to `examples/validation/soap-Proxy` +3. Run the following command, observe a successful response: + `curl --header "Content-Type: application/soap+xml" -d @blz-soap.xml http://localhost:2000/axis2/services/BLZService/getBankResponse` -2. Start `service-proxy.bat` or `service-proxy.sh` -3. Navigate into the `soap-Proxy` folder and run the following command on the console. Observe a successful response. +4. Run this next command and observe that verification fails: + `curl --header "Content-Type: application/soap+xml" -d @invalid-blz-soap.xml http://localhost:2000/axis2/services/BLZService/getBankResponse` +## How it is done + +Let's examine the `proxies.xml` file. + +```xml + + + + + ``` -curl --header "Content-Type: application/soap+xml" -d @blz-soap.xml http://localhost:2000/axis2/services/BLZService/getBankResponse -``` -4. Run the following command in the same directory and observe that verification fails. -``` -curl --header "Content-Type: application/soap+xml" -d @invalid-blz-soap.xml http://localhost:2000/axis2/services/BLZService/getBankResponse -``` + +We define a `` component running on port 2000, which is an expanded version of the basic `` and `` components, capable of storing a WSDL manifest. +Now we simply put a `` component withing the ``, without any attributes, as it inherits the WSDL from the proxy component. --- See: diff --git a/distribution/examples/validation/xml/README.md b/distribution/examples/validation/xml/README.md index 0aaf6ff16..4cfc9dfd4 100644 --- a/distribution/examples/validation/xml/README.md +++ b/distribution/examples/validation/xml/README.md @@ -1,18 +1,49 @@ -### XML Schema Validation +### Validation - XML -To run this example you should install Curl from http://curl.haxx.se/download.html , if -you have not done so already. Let us assume it is in your PATH. +This sample explains how to set up and use the `validator` plugin, utilizing XML schemas for validation. -Execute the following steps: -1. Go to the directory `examples/validation/xml`. +## Running the Example + +1. Go to the directory `/examples/validation/xml`. + 2. Start `service-proxy.bat` or `service-proxy.sh`. + 3. Run `curl -d @year.xml http://localhost:2000/`. Observe that you get a successful response. + 4. Run `curl -d @invalid-year.xml http://localhost:2000/`. Observe that you get a validation error response. +## How it is done + +Let's examine the `proxies.xml` file. + +```xml + + + + + + + + + + + + + + Response.ok("<amount>100</amount>").build() + + + +``` + +We have two `` components in action, operating on ports `2000` and `2001`. +The initial one employs the XML schema in the `` component's schema attribute for validating requests. Upon successful validation, these requests are forwarded to the second `` component. +Here, an XML document is generated and redirected back to the first `` component for a secondary round of validation, this time using a different schema for response validation. + --- See: - [validator](https://membrane-soa.org/api-gateway-doc/current/configuration/reference/validator.htm) reference \ No newline at end of file diff --git a/distribution/examples/validation/xml/proxies.xml b/distribution/examples/validation/xml/proxies.xml index e854904b0..1b1880502 100644 --- a/distribution/examples/validation/xml/proxies.xml +++ b/distribution/examples/validation/xml/proxies.xml @@ -6,7 +6,7 @@ - + @@ -14,13 +14,13 @@ - + - + Response.ok("<amount>100</amount>").build() - + From ed7e7a43fd14f8693d48e481026c1426b53c4ec0 Mon Sep 17 00:00:00 2001 From: Tobias Polley Date: Wed, 19 Jul 2023 15:19:42 +0200 Subject: [PATCH 2/6] ACME: implemented azure table storage (#673) * wip azure table storage engine for acme * wip azure table storage client * wip remove list wrapping * implemented updates when needed * wip implemented debug logging * cleanup refactored common occurences wip implemented debug logging implemented updates when needed wip remove list wrapping wip azure table storage client wip azure table storage engine for acme * wip azure dns provisioning * updated payload * update dns provisioning * remove table storage * cleanup old tablestorage azure * added sleep for dns provisioning * refactoring * fix npe * wip test of azure api client * switch to DNS-based validation for --------- Co-authored-by: koin612 Co-authored-by: Thomas Bayer --- core/pom.xml | 3 +- .../predic8/membrane/core/azure/AzureDns.java | 73 +++++ .../membrane/core/azure/AzureIdentity.java | 59 ++++ .../core/azure/AzureTableStorage.java | 72 +++++ .../core/azure/api/AzureApiClient.java | 56 ++++ .../azure/api/HttpClientConfigurable.java | 8 + .../azure/api/auth/AuthenticationApi.java | 54 ++++ .../core/azure/api/dns/DnsProvisionable.java | 5 + .../core/azure/api/dns/DnsRecordApi.java | 37 +++ .../api/dns/DnsRecordCommandExecutor.java | 74 +++++ .../core/azure/api/dns/DnsRecordType.java | 21 ++ .../azure/api/dns/SupportedDnsRecordType.java | 8 + .../core/azure/api/dns/TxtRecordBuilder.java | 30 +++ .../TableEntityCommandExecutor.java | 75 ++++++ .../api/tablestorage/TableStorageApi.java | 96 +++++++ .../TableStorageCommandExecutor.java | 41 +++ .../config/security/acme/AcmeValidation.java | 7 +- .../membrane/core/rules/AbstractProxy.java | 5 +- .../core/transport/ssl/AcmeSSLContext.java | 4 +- .../acme/AcmeAzureTableApiStorageEngine.java | 252 ++++++++++++++++++ .../core/transport/ssl/acme/AcmeClient.java | 17 +- .../ssl/acme/AcmeKubernetesStorageEngine.java | 4 +- .../core/transport/ssl/acme/AcmeRenewal.java | 2 +- .../AcmeAzureTableApiStorageEngineTest.java | 39 +++ .../core/azure/AzureApiClientTest.java | 121 +++++++++ .../core/azure/AzureDnsApiSimulator.java | 215 +++++++++++++++ .../core/transport/ssl/acme/AcmeStepTest.java | 1 - 27 files changed, 1364 insertions(+), 15 deletions(-) create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/AzureDns.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/AzureIdentity.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/AzureTableStorage.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/AzureApiClient.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/HttpClientConfigurable.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/auth/AuthenticationApi.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsProvisionable.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordApi.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordCommandExecutor.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordType.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/dns/SupportedDnsRecordType.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/dns/TxtRecordBuilder.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableEntityCommandExecutor.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableStorageApi.java create mode 100644 core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableStorageCommandExecutor.java create mode 100644 core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeAzureTableApiStorageEngine.java create mode 100644 core/src/test/java/com/predic8/membrane/core/azure/AcmeAzureTableApiStorageEngineTest.java create mode 100644 core/src/test/java/com/predic8/membrane/core/azure/AzureApiClientTest.java create mode 100644 core/src/test/java/com/predic8/membrane/core/azure/AzureDnsApiSimulator.java diff --git a/core/pom.xml b/core/pom.xml index 63dcce4c7..51db5664e 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -299,7 +299,6 @@ org.apache.httpcomponents httpclient - @@ -432,4 +431,4 @@ - \ No newline at end of file + diff --git a/core/src/main/java/com/predic8/membrane/core/azure/AzureDns.java b/core/src/main/java/com/predic8/membrane/core/azure/AzureDns.java new file mode 100644 index 000000000..f26310f81 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/AzureDns.java @@ -0,0 +1,73 @@ +package com.predic8.membrane.core.azure; + +import com.predic8.membrane.annot.MCAttribute; +import com.predic8.membrane.annot.MCElement; +import com.predic8.membrane.core.config.security.acme.AcmeValidation; + +@MCElement(topLevel = false, name = "azureDns") +public class AzureDns extends AcmeValidation { + + private String dnsZoneName; + private String subscriptionId; + private String tenantId; + private String resourceGroup; + private String resource = "https://management.azure.com"; + private AzureIdentity identity; + + public String getDnsZoneName() { + return dnsZoneName; + } + + @MCAttribute + public void setDnsZoneName(String dnsZoneName) { + this.dnsZoneName = dnsZoneName; + } + + public String getSubscriptionId() { + return subscriptionId; + } + + @MCAttribute + public void setSubscriptionId(String subscriptionId) { + this.subscriptionId = subscriptionId; + } + + public String getTenantId() { + if (identity != null) { + return identity.getTenantId(); + } + return tenantId; + } + + @MCAttribute + public void setTenantId(String tenantId) { + this.tenantId = tenantId; + } + + public String getResourceGroup() { + return resourceGroup; + } + + @MCAttribute + public void setResourceGroup(String resourceGroup) { + this.resourceGroup = resourceGroup; + } + + public String getResource() { + return resource; + } + + @MCAttribute + public void setResource(String resource) { + this.resource = resource; + } + + public AzureIdentity getIdentity() { + return identity; + } + + @MCAttribute + public void setIdentity(AzureIdentity identity) { + this.identity = identity; + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/AzureIdentity.java b/core/src/main/java/com/predic8/membrane/core/azure/AzureIdentity.java new file mode 100644 index 000000000..8a1fe5ad5 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/AzureIdentity.java @@ -0,0 +1,59 @@ +package com.predic8.membrane.core.azure; + +import com.predic8.membrane.annot.MCAttribute; +import com.predic8.membrane.annot.MCElement; + +@MCElement(name = "azureIdentity") +public class AzureIdentity { + + private String grantType = "client_credentials"; + private String clientId; + private String clientSecret; + private String resource = "https://management.azure.com"; + private String tenantId; + + public String getGrantType() { + return grantType; + } + + @MCAttribute + public void setGrantType(String grantType) { + this.grantType = grantType; + } + + public String getClientId() { + return clientId; + } + + @MCAttribute + public void setClientId(String clientId) { + this.clientId = clientId; + } + + public String getClientSecret() { + return clientSecret; + } + + @MCAttribute + public void setClientSecret(String clientSecret) { + this.clientSecret = clientSecret; + } + + public String getResource() { + return resource; + } + + @MCAttribute + public void setResource(String resource) { + this.resource = resource; + } + + public String getTenantId() { + return tenantId; + } + + @MCAttribute + public void setTenantId(String tenantId) { + this.tenantId = tenantId; + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/AzureTableStorage.java b/core/src/main/java/com/predic8/membrane/core/azure/AzureTableStorage.java new file mode 100644 index 000000000..31dff3173 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/AzureTableStorage.java @@ -0,0 +1,72 @@ +package com.predic8.membrane.core.azure; + +import com.predic8.membrane.annot.MCAttribute; +import com.predic8.membrane.annot.MCChildElement; +import com.predic8.membrane.annot.MCElement; +import com.predic8.membrane.core.config.security.acme.AcmeSynchronizedStorage; +import com.predic8.membrane.core.transport.http.client.HttpClientConfiguration; + +@MCElement(name = "azureTableStorage", topLevel = false) +public class AzureTableStorage implements AcmeSynchronizedStorage { + + private String storageAccountName; + private String storageAccountKey; + private String tableName = "membrane"; + private String partitionKey = "acme"; + private HttpClientConfiguration httpClientConfiguration; + + private String customHost; + + public String getCustomHost() { + return customHost; + } + + public void setCustomHost(String customHost) { + this.customHost = customHost; + } + + public String getStorageAccountName() { + return storageAccountName; + } + + @MCAttribute + public void setStorageAccountName(String storageAccountName) { + this.storageAccountName = storageAccountName; + } + + public String getStorageAccountKey() { + return storageAccountKey; + } + + @MCAttribute + public void setStorageAccountKey(String storageAccountKey) { + this.storageAccountKey = storageAccountKey; + } + + public String getTableName() { + return tableName; + } + + @MCAttribute + public void setTableName(String tableName) { + this.tableName = tableName; + } + + public String getPartitionKey() { + return partitionKey; + } + + @MCAttribute + public void setPartitionKey(String partitionKey) { + this.partitionKey = partitionKey; + } + + public HttpClientConfiguration getHttpClientConfiguration() { + return httpClientConfiguration; + } + + @MCChildElement + public void setHttpClientConfiguration(HttpClientConfiguration httpClientConfiguration) { + this.httpClientConfiguration = httpClientConfiguration; + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/AzureApiClient.java b/core/src/main/java/com/predic8/membrane/core/azure/api/AzureApiClient.java new file mode 100644 index 000000000..1f1ac85c6 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/AzureApiClient.java @@ -0,0 +1,56 @@ +package com.predic8.membrane.core.azure.api; + +import com.predic8.membrane.core.azure.AzureDns; +import com.predic8.membrane.core.azure.AzureIdentity; +import com.predic8.membrane.core.azure.AzureTableStorage; +import com.predic8.membrane.core.azure.api.auth.AuthenticationApi; +import com.predic8.membrane.core.azure.api.dns.DnsRecordApi; +import com.predic8.membrane.core.azure.api.tablestorage.TableStorageApi; +import com.predic8.membrane.core.transport.http.HttpClient; +import com.predic8.membrane.core.transport.http.HttpClientFactory; +import com.predic8.membrane.core.util.TimerManager; + +import javax.annotation.Nullable; + +public class AzureApiClient implements AutoCloseable { + + private final HttpClient httpClient; + private final AuthenticationApi authApi; + private final TableStorageApi tableStorageApi; + + + public AzureApiClient( + @Nullable AzureIdentity identityConfig, + AzureTableStorage tableStorage, + HttpClientFactory httpClientFactory + ) { + if (httpClientFactory == null) { + httpClientFactory = new HttpClientFactory(new TimerManager()); + } + this.httpClient = httpClientFactory.createClient(tableStorage.getHttpClientConfiguration()); + + authApi = new AuthenticationApi(httpClient, identityConfig); + tableStorageApi = new TableStorageApi(this, tableStorage); + } + + public DnsRecordApi dnsRecords(AzureDns dnsOperator) { + return new DnsRecordApi(this, dnsOperator); + } + + public TableStorageApi tableStorage() { + return tableStorageApi; + } + + public AuthenticationApi auth() { + return authApi; + } + + public HttpClient httpClient() { + return httpClient; + } + + @Override + public void close() throws Exception { + this.httpClient.close(); + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/HttpClientConfigurable.java b/core/src/main/java/com/predic8/membrane/core/azure/api/HttpClientConfigurable.java new file mode 100644 index 000000000..91ccd7c1e --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/HttpClientConfigurable.java @@ -0,0 +1,8 @@ +package com.predic8.membrane.core.azure.api; + +import com.predic8.membrane.core.transport.http.HttpClient; + +public interface HttpClientConfigurable { + HttpClient http(); + T config(); +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/auth/AuthenticationApi.java b/core/src/main/java/com/predic8/membrane/core/azure/api/auth/AuthenticationApi.java new file mode 100644 index 000000000..9998f9b7c --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/auth/AuthenticationApi.java @@ -0,0 +1,54 @@ +package com.predic8.membrane.core.azure.api.auth; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.predic8.membrane.core.azure.AzureIdentity; +import com.predic8.membrane.core.exchange.Exchange; +import com.predic8.membrane.core.http.Request; +import com.predic8.membrane.core.transport.http.HttpClient; + +import javax.annotation.Nullable; +import java.net.URISyntaxException; +import java.util.Map; +import java.util.stream.Collectors; + +public class AuthenticationApi { + + private final HttpClient http; + private final AzureIdentity config; + private final Map tokenPayload; + + public AuthenticationApi(HttpClient http, @Nullable AzureIdentity config) { + this.http = http; + this.config = config; + + if (config == null) { + tokenPayload = Map.of(); + return; + } + + tokenPayload = Map.of( + "grant_type", config.getGrantType(), + "client_id", config.getClientId(), + "client_secret", config.getClientSecret(), + "resource", config.getResource() + ); + } + + public String accessToken() throws Exception { + var response = http.call(tokenExchange()).getResponse(); + return new ObjectMapper() + .readTree(response.getBodyAsStringDecoded()) + .get("access_token") + .asText(); + } + + private Exchange tokenExchange() throws URISyntaxException { + var tenantId = config.getTenantId(); + return new Request.Builder() + .post("https://login.microsoftonline.com/" + tenantId + "/oauth2/token") + .body(tokenPayload.entrySet().stream() + .map(entry -> entry.getKey() + "=" + entry.getValue()) + .collect(Collectors.joining("&"))) + .buildExchange(); + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsProvisionable.java b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsProvisionable.java new file mode 100644 index 000000000..f561d7344 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsProvisionable.java @@ -0,0 +1,5 @@ +package com.predic8.membrane.core.azure.api.dns; + +public interface DnsProvisionable { + void provisionDns(String domain, String record); +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordApi.java b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordApi.java new file mode 100644 index 000000000..dc4c00ebd --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordApi.java @@ -0,0 +1,37 @@ +package com.predic8.membrane.core.azure.api.dns; + +import com.predic8.membrane.core.azure.AzureDns; +import com.predic8.membrane.core.azure.api.AzureApiClient; +import com.predic8.membrane.core.azure.api.HttpClientConfigurable; +import com.predic8.membrane.core.http.Request; +import com.predic8.membrane.core.transport.http.HttpClient; + +public class DnsRecordApi implements HttpClientConfigurable { + + private final AzureApiClient apiClient; + private final AzureDns config; + + public DnsRecordApi(AzureApiClient apiClient, AzureDns config) { + this.apiClient = apiClient; + this.config = config; + } + + public DnsRecordCommandExecutor txt(String name) { + return new DnsRecordCommandExecutor(this, name, DnsRecordType.TXT); + } + + protected Request.Builder requestBuilder() throws Exception { + return new Request.Builder() + .header("Authorization", "Bearer " + apiClient.auth().accessToken()); + } + + @Override + public HttpClient http() { + return apiClient.httpClient(); + } + + @Override + public AzureDns config() { + return config; + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordCommandExecutor.java b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordCommandExecutor.java new file mode 100644 index 000000000..96c4848e8 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordCommandExecutor.java @@ -0,0 +1,74 @@ +package com.predic8.membrane.core.azure.api.dns; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +public class DnsRecordCommandExecutor { + + private final DnsRecordApi api; + private final String basePath; + private int ttl = 3600; + + private final List records = new ArrayList<>(); + + public DnsRecordCommandExecutor(DnsRecordApi api, String recordSetName, DnsRecordType type) { + this.api = api; + + var resourceBasePath = String.format("%s/subscriptions/%s/resourceGroups/%s/providers", + api.config().getResource(), + api.config().getSubscriptionId(), + api.config().getResourceGroup() + ); + + basePath = String.format("%s/Microsoft.Network/dnsZones/%s/%s/%s?api-version=2018-05-01", + resourceBasePath, + api.config().getDnsZoneName(), + type.toString(), + recordSetName + ); + } + + public DnsRecordCommandExecutor ttl(int ttl) { + this.ttl = ttl; + return this; + } + + public JsonNode create() throws Exception { + Map properties = new HashMap<>(Map.of("TTL", ttl)); + + records.forEach(record -> properties.putAll(record.payload())); + + var payload = Map.of("properties", properties); + + var response = api.http().call( + api.requestBuilder() + .put(basePath) + .contentType("application/json") + .body(new ObjectMapper().writeValueAsString(payload)) + .buildExchange() + ) + .getResponse(); + + return new ObjectMapper().readTree(response.getBodyAsStringDecoded()); + } + + public TxtRecordBuilder addRecord() { + var builder = new TxtRecordBuilder(this); + records.add(builder); + return builder; + } + + public void delete() throws Exception { + api.http().call( + api.requestBuilder() + .delete(basePath) + .header("Accept", "application/json") + .buildExchange() + ); + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordType.java b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordType.java new file mode 100644 index 000000000..eab82df1d --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/DnsRecordType.java @@ -0,0 +1,21 @@ +package com.predic8.membrane.core.azure.api.dns; + +public enum DnsRecordType { + TXT("txt") + ; + + private final String value; + + DnsRecordType(String value) { + this.value = value; + } + + public String getValue() { + return value; + } + + @Override + public String toString() { + return getValue(); + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/dns/SupportedDnsRecordType.java b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/SupportedDnsRecordType.java new file mode 100644 index 000000000..b8332fc36 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/SupportedDnsRecordType.java @@ -0,0 +1,8 @@ +package com.predic8.membrane.core.azure.api.dns; + +import java.util.List; +import java.util.Map; + +public interface SupportedDnsRecordType { + Map>>> payload(); +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/dns/TxtRecordBuilder.java b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/TxtRecordBuilder.java new file mode 100644 index 000000000..906caf44a --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/dns/TxtRecordBuilder.java @@ -0,0 +1,30 @@ +package com.predic8.membrane.core.azure.api.dns; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Map; + +public class TxtRecordBuilder implements SupportedDnsRecordType { + + private final DnsRecordCommandExecutor parent; + private final List values = new ArrayList<>(); + + public TxtRecordBuilder(DnsRecordCommandExecutor parent) { + this.parent = parent; + } + + public DnsRecordCommandExecutor withValue(String... values) { + Collections.addAll(this.values, values); + return parent; + } + + @Override + public Map>>> payload() { + return Map.of( + "TXTRecords", List.of( + Collections.singletonMap("value", List.of(String.join("", values))) + ) + ); + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableEntityCommandExecutor.java b/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableEntityCommandExecutor.java new file mode 100644 index 000000000..177f5bdce --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableEntityCommandExecutor.java @@ -0,0 +1,75 @@ +package com.predic8.membrane.core.azure.api.tablestorage; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; + +import java.net.URLEncoder; +import java.nio.charset.StandardCharsets; +import java.util.Map; +import java.util.NoSuchElementException; + +public class TableEntityCommandExecutor { + + private final TableStorageApi api; + private final String path; + + public TableEntityCommandExecutor(TableStorageApi api, String rowKey) { + this.api = api; + + path = api.config().getCustomHost() == null + ? String.format("https://%s.table.core.windows.net/%s%s", + api.config().getStorageAccountName(), + api.config().getTableName(), + URLEncoder.encode("(PartitionKey='" + api.config().getPartitionKey() + "',RowKey='" + rowKey + "')", StandardCharsets.UTF_8)) + : String.format("%s/%s%s", + api.config().getCustomHost(), + api.config().getTableName(), + URLEncoder.encode("(PartitionKey='" + api.config().getPartitionKey() + "',RowKey='" + rowKey + "')", StandardCharsets.UTF_8)); + } + + public JsonNode get() throws Exception { + var res = api.http().call( + api.requestBuilder(path) + .get(path) + .buildExchange() + ).getResponse(); + + var response = new ObjectMapper().readTree(res.getBodyAsStringDecoded()); + + if (response.has("odata.error")) { + throw new NoSuchElementException(res.getBodyAsStringDecoded()); + } + + return response; + } + + public void insertOrReplace(String data) throws Exception { + var payload = Map.of("data", data); + + var exc = api.http().call( + api.requestBuilder(path) + .put(path) + .body(new ObjectMapper().writeValueAsString(payload)) + .buildExchange() + ); + + if (exc.getResponse().getStatusCode() != 204) { + throw new RuntimeException(exc.getResponse().toString()); + } + } + + public void delete() throws Exception { + var exc = api.http().call( + api.requestBuilder(path) + .delete(path) + .header("If-Match", "*") + .buildExchange() + ); + + var response = exc.getResponse(); + + if (response.getStatusCode() != 204) { + throw new RuntimeException(response.toString()); + } + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableStorageApi.java b/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableStorageApi.java new file mode 100644 index 000000000..fff7093b8 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableStorageApi.java @@ -0,0 +1,96 @@ +package com.predic8.membrane.core.azure.api.tablestorage; + +import com.predic8.membrane.core.azure.AzureTableStorage; +import com.predic8.membrane.core.azure.api.AzureApiClient; +import com.predic8.membrane.core.azure.api.HttpClientConfigurable; +import com.predic8.membrane.core.http.Request; +import com.predic8.membrane.core.transport.http.HttpClient; + +import javax.annotation.Nullable; +import javax.crypto.Mac; +import javax.crypto.spec.SecretKeySpec; +import java.nio.charset.StandardCharsets; +import java.time.Instant; +import java.time.ZoneId; +import java.time.format.DateTimeFormatter; +import java.util.Base64; +import java.util.Locale; + +public class TableStorageApi implements HttpClientConfigurable { + + private final AzureApiClient apiClient; + private final AzureTableStorage config; + + public TableStorageApi(AzureApiClient apiClient, AzureTableStorage config) { + this.apiClient = apiClient; + this.config = config; + } + + public TableStorageCommandExecutor table() { + return new TableStorageCommandExecutor(this); + } + + public TableEntityCommandExecutor entity(String rowKey) { + return new TableEntityCommandExecutor(this, rowKey); + } + + @Override + public HttpClient http() { + return apiClient.httpClient(); + } + + @Override + public AzureTableStorage config() { + return config; + } + + protected Request.Builder requestBuilder(@Nullable String announceUrl) throws Exception { + var date = now(); + + var stringToSign = buildStringToSign(date, announceUrl); + var storageAccountKey = config.getStorageAccountKey(); + + var signature = sign(storageAccountKey, stringToSign); + + var storageAccountName = config.getStorageAccountName(); + + return new Request.Builder() + .contentType("application/json") + .header("Date", date) + .header("x-ms-version", "2020-12-06") + .header("Accept", "application/json") + .header("DataServiceVersion", "3.0;NetFx") + .header("MaxDataServiceVersion", "3.0;NetFx") + .header("Authorization", + String.format("SharedKeyLite %s:%s", storageAccountName, signature)); + } + + private String now() { + return DateTimeFormatter + .ofPattern("EEE, dd MMM yyyy HH:mm:ss z", Locale.ENGLISH) + .withZone(ZoneId.of("GMT")) + .format(Instant.now()); + } + + private String buildStringToSign(String date, @Nullable String url) { + var storageAccount = config.getStorageAccountName(); + var base = String.format("%s\n/%s", date, storageAccount); + + if (url == null) { + return base + "/Tables"; + } + + return base + url.substring(url.lastIndexOf("/" + config.getTableName())); + } + + private String sign(String base64Key, String stringToSign) throws Exception { + var algo = "HmacSHA256"; + + byte[] key = java.util.Base64.getDecoder().decode(base64Key); + Mac hmacSHA256 = Mac.getInstance(algo); + hmacSHA256.init(new SecretKeySpec(key, algo)); + byte[] utf8Bytes = stringToSign.getBytes(StandardCharsets.UTF_8); + + return Base64.getEncoder().encodeToString(hmacSHA256.doFinal(utf8Bytes)); + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableStorageCommandExecutor.java b/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableStorageCommandExecutor.java new file mode 100644 index 000000000..fcbcdfb18 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/azure/api/tablestorage/TableStorageCommandExecutor.java @@ -0,0 +1,41 @@ +package com.predic8.membrane.core.azure.api.tablestorage; + +import com.fasterxml.jackson.databind.ObjectMapper; + +import java.util.Map; + +public class TableStorageCommandExecutor { + + private final TableStorageApi api; + private final String path; + + public TableStorageCommandExecutor(TableStorageApi api) { + this.api = api; + + path = api.config().getCustomHost() == null + ? String.format("https://%s.table.core.windows.net/Tables", api.config().getStorageAccountName()) + : api.config().getCustomHost() + "/Tables"; + } + + public void create() throws Exception { + var payload = Map.of( + "TableName", api.config().getTableName() + ); + + var exc = api.http().call( + api.requestBuilder(null) + .post(path) + .body(new ObjectMapper().writeValueAsString(payload)) + .buildExchange() + ); + + var response = exc.getResponse(); + + var isCreated = response.getStatusCode() == 201; + var isExisting = response.getStatusCode() == 409; + + if (!(isCreated || isExisting)) { + throw new RuntimeException(exc.getResponse().toString()); + } + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/config/security/acme/AcmeValidation.java b/core/src/main/java/com/predic8/membrane/core/config/security/acme/AcmeValidation.java index 30e54a5a7..b94e19c88 100644 --- a/core/src/main/java/com/predic8/membrane/core/config/security/acme/AcmeValidation.java +++ b/core/src/main/java/com/predic8/membrane/core/config/security/acme/AcmeValidation.java @@ -13,5 +13,10 @@ limitations under the License. */ package com.predic8.membrane.core.config.security.acme; -public class AcmeValidation { +public abstract class AcmeValidation { + + public boolean useDnsValidation() { + return true; + } + } diff --git a/core/src/main/java/com/predic8/membrane/core/rules/AbstractProxy.java b/core/src/main/java/com/predic8/membrane/core/rules/AbstractProxy.java index 73e0b73ff..99ecfa516 100644 --- a/core/src/main/java/com/predic8/membrane/core/rules/AbstractProxy.java +++ b/core/src/main/java/com/predic8/membrane/core/rules/AbstractProxy.java @@ -15,8 +15,6 @@ import java.util.ArrayList; import java.util.List; -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; import com.predic8.membrane.core.stats.RuleStatisticCollector; import com.predic8.membrane.core.transport.ssl.*; @@ -27,7 +25,6 @@ import com.predic8.membrane.annot.MCAttribute; import com.predic8.membrane.annot.MCChildElement; import com.predic8.membrane.core.Router; -import com.predic8.membrane.core.exchange.Exchange; import com.predic8.membrane.core.interceptor.Interceptor; public abstract class AbstractProxy implements Rule { @@ -189,7 +186,7 @@ public void init() throws Exception { if (acmeCtx == null) acmeCtx = new AcmeSSLContext(sslInboundParser, host, router.getHttpClientFactory(), router.getTimerManager()); setSslInboundContext(acmeCtx); - acmeCtx.init(router.getKubernetesClientFactory()); + acmeCtx.init(router.getKubernetesClientFactory(), router.getHttpClientFactory()); return; } diff --git a/core/src/main/java/com/predic8/membrane/core/transport/ssl/AcmeSSLContext.java b/core/src/main/java/com/predic8/membrane/core/transport/ssl/AcmeSSLContext.java index c667dec14..b57204ba0 100644 --- a/core/src/main/java/com/predic8/membrane/core/transport/ssl/AcmeSSLContext.java +++ b/core/src/main/java/com/predic8/membrane/core/transport/ssl/AcmeSSLContext.java @@ -61,8 +61,8 @@ public AcmeSSLContext(SSLParser parser, this.timerManager = timerManager != null ? timerManager : new TimerManager(); } - public void init(@Nullable KubernetesClientFactory kubernetesClientFactory) { - client.init(kubernetesClientFactory); + public void init(@Nullable KubernetesClientFactory kubernetesClientFactory, @Nullable HttpClientFactory httpClientFactory) { + client.init(kubernetesClientFactory, httpClientFactory); initAndSchedule(); } diff --git a/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeAzureTableApiStorageEngine.java b/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeAzureTableApiStorageEngine.java new file mode 100644 index 000000000..e39dcfa03 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeAzureTableApiStorageEngine.java @@ -0,0 +1,252 @@ +package com.predic8.membrane.core.transport.ssl.acme; + +import com.fasterxml.jackson.databind.JsonNode; +import com.predic8.membrane.core.azure.AzureDns; +import com.predic8.membrane.core.azure.AzureTableStorage; +import com.predic8.membrane.core.azure.api.dns.DnsProvisionable; +import com.predic8.membrane.core.azure.api.AzureApiClient; +import com.predic8.membrane.core.transport.http.HttpClientFactory; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.annotation.Nullable; +import java.util.Arrays; + +public class AcmeAzureTableApiStorageEngine implements AcmeSynchronizedStorageEngine, DnsProvisionable { + + private static final Logger log = LoggerFactory.getLogger(AcmeAzureTableApiStorageEngine.class); + + private static final String CURRENT = "current"; + private static final String CURRENT_ERROR = "current-error"; + private static final String CURRENT_KEY = "current-key"; + + private final AzureApiClient apiClient; + private final AzureDns azureDns; + + public AcmeAzureTableApiStorageEngine( + AzureTableStorage tableStorage, + @Nullable AzureDns azureDns, + @Nullable HttpClientFactory httpClientFactory + ) { + this.azureDns = azureDns; + apiClient = new AzureApiClient(azureDns == null ? null : azureDns.getIdentity(), tableStorage, httpClientFactory); + + try { + apiClient.tableStorage().table().create(); + } catch (Exception e) { + // ignore if table exists already + log.debug("Ignore table already exists exception"); + } + + log.debug("Loaded {}", this.getClass().getSimpleName()); + } + + private JsonNode getEntity(String rowKey) { + try { + log.debug("Get entity for {}", rowKey); + return apiClient.tableStorage().entity(rowKey).get(); + } catch (Exception e) { + log.debug("Entity {} does not exist, returning null", rowKey); + return null; + } + } + + private String getDataPropertyOfEntity(String rowKey) { + var entity = getEntity(rowKey); + + return entity != null + ? entity.get("data").asText() + : null; + } + + private void upsertDataEntity(String rowKey, String data) { + try { + log.debug("Upserting key {}", rowKey); + apiClient.tableStorage().entity(rowKey).insertOrReplace(data); + } catch (Exception e) { + throw new RuntimeException(e); + } + } + + private String id(String[] hosts) { + int i = Arrays.hashCode(hosts); + if (i < 0) i = Integer.MAX_VALUE + i + 1; + return hosts[0] + "-" + i; + } + + private String getPublicKeyRowKey(String[] hosts) { + return String.format("key-%s-pub.pem", id(hosts)); + } + + private String getPrivateKeyRowKey(String[] hosts) { + return String.format("key-%s.pem", id(hosts)); + } + + private String getCertChainRowKey(String[] hosts) { + return String.format("cert-%s.pem", id(hosts)); + } + + private String getTokenRowKey(String host) { + return String.format("token-%s", host); + } + + private String getOALRowKey(String[] hosts, String postfix) { + return String.format("oal-%s-%s.json", id(hosts), postfix); + } + + @Override + public String getAccountKey() { + return getDataPropertyOfEntity("account"); + } + + @Override + public void setAccountKey(String key) { + upsertDataEntity("account", key); + } + + @Override + public void setKeyPair(String[] hosts, AcmeKeyPair key) { + upsertDataEntity(getPublicKeyRowKey(hosts), key.getPublicKey()); + upsertDataEntity(getPrivateKeyRowKey(hosts), key.getPrivateKey()); + } + + @Override + public String getPublicKey(String[] hosts) { + return getDataPropertyOfEntity(getPublicKeyRowKey(hosts)); + } + + @Override + public String getPrivateKey(String[] hosts) { + return getDataPropertyOfEntity(getPrivateKeyRowKey(hosts)); + } + + @Override + public void setCertChain(String[] hosts, String caChain) { + upsertDataEntity(getCertChainRowKey(hosts), caChain); + } + + @Override + public String getCertChain(String[] hosts) { + return getDataPropertyOfEntity(getCertChainRowKey(hosts)); + } + + @Override + public void setToken(String host, String token) { + upsertDataEntity(getTokenRowKey(host), token); + } + + @Override + public String getToken(String host) { + return getDataPropertyOfEntity(getTokenRowKey(host)); + } + + @Override + public String getOAL(String[] hosts) { + return getDataPropertyOfEntity(getOALRowKey(hosts, CURRENT)); + } + + @Override + public void setOAL(String[] hosts, String oal) { + upsertDataEntity(getOALRowKey(hosts, CURRENT), oal); + } + + @Override + public String getAccountURL() { + return getDataPropertyOfEntity("account-url"); + } + + @Override + public void setAccountURL(String url) { + upsertDataEntity("account-url", url); + } + + @Override + public String getAccountContacts() { + return getDataPropertyOfEntity("account-contacts"); + } + + @Override + public void setAccountContacts(String contacts) { + upsertDataEntity("account-contacts", contacts); + } + + @Override + public String getOALError(String[] hosts) { + return getDataPropertyOfEntity(getOALRowKey(hosts, CURRENT_ERROR)); + } + + @Override + public void setOALError(String[] hosts, String oalError) { + upsertDataEntity(getOALRowKey(hosts, CURRENT_ERROR), oalError); + } + + @Override + public String getOALKey(String[] hosts) { + return getDataPropertyOfEntity(getOALRowKey(hosts, CURRENT_KEY)); + } + + @Override + public void setOALKey(String[] hosts, String oalKey) { + upsertDataEntity(getOALRowKey(hosts, CURRENT_KEY), oalKey); + } + + @Override + public void archiveOAL(String[] hosts) { + long now = System.currentTimeMillis(); + attemptRename(getOALRowKey(hosts, CURRENT), getOALRowKey(hosts, String.valueOf(now))); + attemptRename(getOALRowKey(hosts, CURRENT_ERROR), getOALRowKey(hosts, now + "-error")); + attemptRename(getOALRowKey(hosts, CURRENT_KEY), getOALRowKey(hosts, now + "-key")); + } + + private void attemptRename(String f1, String f2) { + log.debug("Attempt rename {} to {}", f1, f2); + var first = getDataPropertyOfEntity(f1); + + if (first != null) { + try { + log.debug("creating {}", f2); + apiClient.tableStorage().entity(f2).insertOrReplace(first); + log.debug("removing {}", f1); + apiClient.tableStorage().entity(f1).delete(); + } catch (Exception e) { + throw new RuntimeException(e); + } + return; + } + + log.debug("Attempt rename, but there was nothing to rename"); + } + + @Override + public void provisionDns(String domain, String record) { + try { + apiClient.dnsRecords(azureDns).txt("_acme-challenge") + .ttl(300) + .addRecord() + .withValue(record) + .create(); + + // https://learn.microsoft.com/en-us/azure/dns/dns-faq#how-long-does-it-take-for-dns-changes-to-take-effect- + Thread.sleep(60_000); + } catch (Exception e) { + throw new RuntimeException(e); + } + } + + @Override + public boolean acquireLease(long durationMillis) { + // Single instance + return true; + } + + @Override + public boolean prolongLease(long durationMillis) { + // Single instance + return true; + } + + @Override + public void releaseLease() { + // Single instance + } +} diff --git a/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeClient.java b/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeClient.java index 98cd546eb..e1ea86b26 100644 --- a/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeClient.java +++ b/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeClient.java @@ -18,6 +18,9 @@ import com.fasterxml.jackson.datatype.joda.*; import com.google.common.collect.*; import com.predic8.membrane.core.*; +import com.predic8.membrane.core.azure.AzureDns; +import com.predic8.membrane.core.azure.AzureTableStorage; +import com.predic8.membrane.core.azure.api.dns.DnsProvisionable; import com.predic8.membrane.core.config.security.acme.*; import com.predic8.membrane.core.exchange.*; import com.predic8.membrane.core.http.*; @@ -91,6 +94,7 @@ public class AcmeClient { private final String algorithm = AlgorithmIdentifiers.ECDSA_USING_P256_CURVE_AND_SHA256; private final Duration validity; private AcmeSynchronizedStorageEngine asse; + private AcmeValidation acmeValidation; public AcmeClient(Acme acme, @Nullable HttpClientFactory httpClientFactory) { directoryUrl = acme.getDirectoryUrl(); @@ -101,7 +105,8 @@ public AcmeClient(Acme acme, @Nullable HttpClientFactory httpClientFactory) { httpClientFactory = new HttpClientFactory(null); hc = httpClientFactory.createClient(acme.getHttpClientConfiguration()); validity = acme.getValidityDuration(); - challengeType = acme.getValidationMethod() != null && acme.getValidationMethod() instanceof DnsOperatorAcmeValidation ? TYPE_DNS_01 : TYPE_HTTP_01; + this.acmeValidation = acme.getValidationMethod(); + challengeType = acme.getValidationMethod() != null && acme.getValidationMethod().useDnsValidation() ? TYPE_DNS_01 : TYPE_HTTP_01; om.registerModule(new JodaModule()); @@ -110,7 +115,7 @@ public AcmeClient(Acme acme, @Nullable HttpClientFactory httpClientFactory) { throw new RuntimeException("The ACME client is still experimental, please set to acknowledge."); } - public void init(@Nullable KubernetesClientFactory kubernetesClientFactory) { + public void init(@Nullable KubernetesClientFactory kubernetesClientFactory, @Nullable HttpClientFactory httpClientFactory) { if (ass == null) { throw new RuntimeException(" is used, but to storage is configured."); } else if (ass instanceof FileStorage) { @@ -119,9 +124,15 @@ public void init(@Nullable KubernetesClientFactory kubernetesClientFactory) { asse = new AcmeKubernetesStorageEngine((KubernetesStorage) ass, kubernetesClientFactory); } else if (ass instanceof MemoryStorage) { asse = new AcmeMemoryStorageEngine(); + } else if (ass instanceof AzureTableStorage) { + asse = new AcmeAzureTableApiStorageEngine((AzureTableStorage) ass, (AzureDns) acmeValidation, httpClientFactory); } else { throw new RuntimeException("Unsupported: Storage type " + ass.getClass().getName()); } + + if (challengeType.equals(TYPE_DNS_01) && !(asse instanceof DnsProvisionable)) { + throw new RuntimeException("A"); + } } public void loadDirectory() throws Exception { @@ -284,7 +295,7 @@ private void provisionDns(Authorization auth, Challenge challenge) throws JoseEx MessageDigest digest = MessageDigest.getInstance("SHA-256"); String record = java.util.Base64.getUrlEncoder().withoutPadding().encodeToString(digest.digest(keyAuth.getBytes(UTF_8))); - ((AcmeKubernetesStorageEngine)asse).provisionDns(auth.getIdentifier().getValue(), record); + ((DnsProvisionable)asse).provisionDns(auth.getIdentifier().getValue(), record); } private void provisionHttp(Authorization auth, Challenge challenge) { diff --git a/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeKubernetesStorageEngine.java b/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeKubernetesStorageEngine.java index 27e57da2c..c226fc471 100644 --- a/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeKubernetesStorageEngine.java +++ b/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeKubernetesStorageEngine.java @@ -13,6 +13,7 @@ limitations under the License. */ package com.predic8.membrane.core.transport.ssl.acme; +import com.predic8.membrane.core.azure.api.dns.DnsProvisionable; import com.predic8.membrane.core.config.security.acme.KubernetesStorage; import com.predic8.membrane.core.kubernetes.client.KubernetesApiException; import com.predic8.membrane.core.kubernetes.client.KubernetesClient; @@ -30,7 +31,7 @@ import static com.google.common.collect.Lists.newArrayList; import static java.nio.charset.StandardCharsets.UTF_8; -public class AcmeKubernetesStorageEngine implements AcmeSynchronizedStorageEngine { +public class AcmeKubernetesStorageEngine implements AcmeSynchronizedStorageEngine, DnsProvisionable { private static final Logger LOG = LoggerFactory.getLogger(AcmeKubernetesStorageEngine.class); @@ -370,6 +371,7 @@ public LeaseException(Throwable cause) { } } + @Override public void provisionDns(String domain, String record) { Map wantedRecord = of("type", "TXT", "timeout", 300, diff --git a/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeRenewal.java b/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeRenewal.java index 3b1fe39ac..774efd0cd 100644 --- a/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeRenewal.java +++ b/core/src/main/java/com/predic8/membrane/core/transport/ssl/acme/AcmeRenewal.java @@ -201,7 +201,7 @@ private void waitFor(String what, Supplier condition, Runnable job) thr private Challenge getChallenge(Authorization auth) throws JsonProcessingException, FatalAcmeException { Optional challenge = auth.getChallenges().stream().filter(c -> client.getChallengeType().equals(c.getType())).findAny(); if (challenge.isEmpty()) - throw new FatalAcmeException("Could not find challenge of type http01: " + om.writeValueAsString(auth)); + throw new FatalAcmeException("Could not find challenge of type "+client.getChallengeType()+": " + om.writeValueAsString(auth)); return challenge.get(); } diff --git a/core/src/test/java/com/predic8/membrane/core/azure/AcmeAzureTableApiStorageEngineTest.java b/core/src/test/java/com/predic8/membrane/core/azure/AcmeAzureTableApiStorageEngineTest.java new file mode 100644 index 000000000..36aec117d --- /dev/null +++ b/core/src/test/java/com/predic8/membrane/core/azure/AcmeAzureTableApiStorageEngineTest.java @@ -0,0 +1,39 @@ +package com.predic8.membrane.core.azure; + +import com.predic8.membrane.core.transport.ssl.acme.AcmeAzureTableApiStorageEngine; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +public class AcmeAzureTableApiStorageEngineTest { + + private AcmeAzureTableApiStorageEngine engine; + + @BeforeAll + void setup() { + var tableStorage = new AzureTableStorage(); + tableStorage.setStorageAccountName(""); + tableStorage.setStorageAccountKey(""); + + + engine = new AcmeAzureTableApiStorageEngine( + tableStorage, + null, + null + ); + } + + + + @Test + void createTable() { + + } + + @Test + void ignoreExistingTable() { + + } + + +} + diff --git a/core/src/test/java/com/predic8/membrane/core/azure/AzureApiClientTest.java b/core/src/test/java/com/predic8/membrane/core/azure/AzureApiClientTest.java new file mode 100644 index 000000000..9b406fe1d --- /dev/null +++ b/core/src/test/java/com/predic8/membrane/core/azure/AzureApiClientTest.java @@ -0,0 +1,121 @@ +package com.predic8.membrane.core.azure; + +import com.predic8.membrane.core.azure.api.AzureApiClient; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import java.io.IOException; +import java.util.NoSuchElementException; + +import static org.junit.jupiter.api.Assertions.*; + +class AzureApiClientTest { + + AzureDnsApiSimulator simulator; + AzureApiClient apiClient; + + @BeforeEach + void setup() throws IOException { + var tableStorage = new AzureTableStorage(); + tableStorage.setStorageAccountName("hello"); + tableStorage.setStorageAccountKey("thisisasecretandshouldhaveenoughbits"); + + int port = 3050; + + tableStorage.setCustomHost("http://localhost:" + port); + + simulator = new AzureDnsApiSimulator(port); + simulator.start(); + + apiClient = new AzureApiClient(null, tableStorage, null); + } + + @AfterEach + void tearDown() { + simulator.stop(); + } + + @Test + void createNonExistingTable() { + assertDoesNotThrow(() -> apiClient.tableStorage().table().create()); + } + + @Test + void ignoreExistingTable() { + assertDoesNotThrow(() -> { + apiClient.tableStorage().table().create(); + apiClient.tableStorage().table().create(); + }); + } + + @Test + void insertWithoutBaseTable() { + assertThrows(RuntimeException.class, () -> apiClient + .tableStorage() + .entity("account") + .insertOrReplace("initial")); + } + + @Test + void createTableEntry() throws Exception { + assertDoesNotThrow(() -> { + apiClient.tableStorage().table().create(); + + apiClient + .tableStorage() + .entity("account") + .insertOrReplace("initial"); + }); + + var data = apiClient.tableStorage().entity("account").get() + .get("data").asText(); + + assertEquals("initial", data); + } + + @Test + void getNonExistingEntity() { + assertDoesNotThrow(() -> apiClient.tableStorage().table().create()); + assertThrows(NoSuchElementException.class, () -> apiClient.tableStorage().entity("foo").get()); + } + + @Test + void updateTableEntry() throws Exception { + assertDoesNotThrow(() -> { + apiClient.tableStorage().table().create(); + + apiClient + .tableStorage() + .entity("account") + .insertOrReplace("initial"); + + apiClient + .tableStorage() + .entity("account") + .insertOrReplace("updated"); + }); + + var data = apiClient.tableStorage().entity("account").get() + .get("data").asText(); + + assertEquals("updated", data); + } + + @Test + void deleteExistingEntity() { + assertDoesNotThrow(() -> { + apiClient.tableStorage().table().create(); + apiClient.tableStorage().entity("account") + .insertOrReplace("initial"); + }); + assertDoesNotThrow(() -> apiClient.tableStorage().entity("account").get()); + assertDoesNotThrow(() -> apiClient.tableStorage().entity("account").delete()); + } + + @Test + void deleteNonExisting() { + assertDoesNotThrow(() -> apiClient.tableStorage().table().create()); + assertDoesNotThrow(() -> apiClient.tableStorage().entity("account").delete()); + } +} diff --git a/core/src/test/java/com/predic8/membrane/core/azure/AzureDnsApiSimulator.java b/core/src/test/java/com/predic8/membrane/core/azure/AzureDnsApiSimulator.java new file mode 100644 index 000000000..30962a9ce --- /dev/null +++ b/core/src/test/java/com/predic8/membrane/core/azure/AzureDnsApiSimulator.java @@ -0,0 +1,215 @@ +package com.predic8.membrane.core.azure; + +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.predic8.membrane.core.HttpRouter; +import com.predic8.membrane.core.exchange.Exchange; +import com.predic8.membrane.core.http.Response; +import com.predic8.membrane.core.interceptor.AbstractInterceptor; +import com.predic8.membrane.core.interceptor.Outcome; +import com.predic8.membrane.core.rules.ServiceProxy; +import com.predic8.membrane.core.rules.ServiceProxyKey; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; +import java.util.*; +import java.util.regex.Pattern; + +public class AzureDnsApiSimulator { + + private static final Logger log = LoggerFactory.getLogger(AzureDnsApiSimulator.class); + + private final int port; + private HttpRouter router; + + private Map>> tableStorage = new HashMap<>(); + + public AzureDnsApiSimulator(int port) { + this.port = port; + } + + public void start() throws IOException { + router = new HttpRouter(); + router.setHotDeploy(false); + + var sp = new ServiceProxy(new ServiceProxyKey(port), "localhost", port); + + sp.getInterceptors().add(new AbstractInterceptor() { + @Override + public Outcome handleRequest(Exchange exc) throws Exception { + log.info("got request {}" + exc.getRequestURI()); + + if (missingHeaders(exc)) { + exc.setResponse(Response.badRequest().build()); + return Outcome.RETURN; + } + + if (exc.getRequestURI().equals("/Tables")) { + return createTableStorageTable(exc); + } + + if (exc.getRequestURI().startsWith("/membrane")) { + return switch (exc.getRequest().getMethod()) { + case "PUT" -> insertOrReplaceTableStorageEntity(exc); + case "GET" -> getEntityFromTableStorage(exc); + case "DELETE" -> deleteEntityFromTableStorage(exc); + default -> Outcome.RETURN; + }; + } + + exc.setResponse(Response.notFound().build()); + return Outcome.RETURN; + } + }); + + router.add(sp); + router.start(); + } + + private boolean missingHeaders(Exchange exc) { + var hasNeededHeaders = Arrays.stream(exc.getRequest().getHeader().getAllHeaderFields()).allMatch(headerField -> + List.of("Date", "x-ms-version", "DataServiceVersion", "MaxDataServiceVersion", "Authorization") + .contains(headerField.getHeaderName()) + ); + + if (!hasNeededHeaders) { + return false; + } + + return true; + } + + private Outcome deleteEntityFromTableStorage(Exchange exc) { + var uriPayload = extractValuesFromUri(exc.getRequestURI()); + + if (uriPayload == null) { + exc.setResponse(Response.badRequest().build()); + return Outcome.RETURN; + } + + var tableName = uriPayload.get("tableName"); + var rowKey = uriPayload.get("rowKey"); + + tableStorage.put(tableName, + tableStorage.get(tableName).stream() + .filter(map -> !map.get("RowKey").equals(rowKey)) + .toList() + ); + + exc.setResponse(Response.statusCode(204).build()); + return Outcome.RETURN; + } + + private Outcome insertOrReplaceTableStorageEntity(Exchange exc) throws JsonProcessingException { + var data = new ObjectMapper() + .readTree(exc.getRequest().getBodyAsStringDecoded()) + .get("data") + .asText(); + + if (data == null) { + exc.setResponse(Response.badRequest().build()); + return Outcome.RETURN; + } + + var uriPayload = extractValuesFromUri(exc.getRequestURI()); + + if (uriPayload == null) { + exc.setResponse(Response.badRequest().build()); + return Outcome.RETURN; + } + + var tableName = uriPayload.get("tableName"); + var partitionKey = uriPayload.get("partitionKey"); + var rowKey = uriPayload.get("rowKey"); + + if (!tableStorage.containsKey(tableName)) { + exc.setResponse(Response.notFound().build()); + return Outcome.RETURN; + } + + var table = tableStorage.get(tableName); + var row = table.stream().filter(t -> t.get("RowKey").equals(rowKey)).findFirst(); + + if (row.isPresent()) { + row.get().put("data", data); + } else { + table.add(new HashMap<>(Map.of( + "PartitionKey", partitionKey, + "RowKey", rowKey, + "data", data + ))); + } + + exc.setResponse(Response.statusCode(204).build()); + return Outcome.RETURN; + } + + private Outcome getEntityFromTableStorage(Exchange exc) throws JsonProcessingException { + var uriPayload = extractValuesFromUri(exc.getRequestURI()); + + if (uriPayload == null) { + exc.setResponse(Response.badRequest().build()); + return Outcome.RETURN; + } + + var tableName = uriPayload.get("tableName"); + var rowKey = uriPayload.get("rowKey"); + + var row = tableStorage.get(tableName).stream() + .filter(e -> e.get("RowKey").equals(rowKey)) + .findFirst(); + + if (row.isEmpty()) { + exc.setResponse(Response.statusCode(400) + .body(new ObjectMapper().writeValueAsString(Map.of("odata.error", "foo"))) + .build()); + return Outcome.RETURN; + } + + exc.setResponse(Response.statusCode(200) + .body(new ObjectMapper().writeValueAsString(row.get())) + .build()); + return Outcome.RETURN; + } + + private Map extractValuesFromUri(String uri) { + var pattern = Pattern.compile("/(\\w+)%28PartitionKey%3D%27(\\w+)%27%2CRowKey%3D%27(\\w+)%27%29"); + var matcher = pattern.matcher(uri); + + if (matcher.matches()) { + return Map.of( + "tableName", matcher.group(1), + "partitionKey", matcher.group(2), + "rowKey", matcher.group(3) + ); + } + + return null; + } + + private Outcome createTableStorageTable(Exchange exc) throws JsonProcessingException { + var tableName = new ObjectMapper() + .readTree(exc.getRequest().getBodyAsStringDecoded()) + .get("TableName") + .asText(); + + if (tableName == null || tableName.isBlank() || tableName.isEmpty()) { + exc.setResponse(Response.badRequest().build()); + return Outcome.RETURN; + } + + if (tableStorage.containsKey(tableName)) { + exc.setResponse(Response.statusCode(409).build()); + } else { + tableStorage.put(tableName, new ArrayList<>()); + exc.setResponse(Response.statusCode(201).build()); + } + + return Outcome.RETURN; + } + + public void stop() { + router.stop(); + } +} diff --git a/core/src/test/java/com/predic8/membrane/core/transport/ssl/acme/AcmeStepTest.java b/core/src/test/java/com/predic8/membrane/core/transport/ssl/acme/AcmeStepTest.java index de3000d0c..8853e9e33 100644 --- a/core/src/test/java/com/predic8/membrane/core/transport/ssl/acme/AcmeStepTest.java +++ b/core/src/test/java/com/predic8/membrane/core/transport/ssl/acme/AcmeStepTest.java @@ -45,7 +45,6 @@ import static com.predic8.membrane.core.transport.ssl.acme.Order.ORDER_STATUS_PROCESSING; import static com.predic8.membrane.core.transport.ssl.acme.Order.ORDER_STATUS_READY; import static org.junit.jupiter.api.Assertions.*; -import static org.junit.jupiter.api.Assertions.*; public class AcmeStepTest { From 2e4fa29821ef7344e5b005a303a7354c947350ed Mon Sep 17 00:00:00 2001 From: Thomas Bayer Date: Wed, 19 Jul 2023 15:22:21 +0200 Subject: [PATCH 3/6] #601 Rewritting for OpenAPI - Refactoring of OpenAPI code (#665) --- .../membrane/core/exchange/Exchange.java | 11 +- .../core/openapi/serviceproxy/APIProxy.java | 98 +--------- .../serviceproxy/OpenAPIInterceptor.java | 18 +- .../OpenAPIPublisherInterceptor.java | 81 +------- .../openapi/serviceproxy/OpenAPIRecord.java | 21 ++- .../serviceproxy/OpenAPIRecordFactory.java | 29 +-- .../openapi/serviceproxy/OpenAPISpec.java | 122 ++++++++++++ .../core/openapi/serviceproxy/Rewrite.java | 175 ++++++++++++++++++ .../openapi/serviceproxy/APIProxyTest.java | 23 +-- .../serviceproxy/OpenAPIInterceptorTest.java | 24 ++- .../OpenAPIPublisherInterceptorTest.java | 44 +---- .../serviceproxy/OpenAPIRecordTest.java | 22 ++- .../openapi/serviceproxy/RewriteTest.java | 172 +++++++++++++++++ .../serviceproxy/XMembraneExtentionTest.java | 4 +- .../membrane/core/openapi/util/TestUtils.java | 2 +- .../openapi/specs/info-no-servers.yml | 11 ++ 16 files changed, 593 insertions(+), 264 deletions(-) create mode 100644 core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPISpec.java create mode 100644 core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/Rewrite.java create mode 100644 core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/RewriteTest.java create mode 100644 core/src/test/resources/openapi/specs/info-no-servers.yml diff --git a/core/src/main/java/com/predic8/membrane/core/exchange/Exchange.java b/core/src/main/java/com/predic8/membrane/core/exchange/Exchange.java index 99b42857f..561281386 100644 --- a/core/src/main/java/com/predic8/membrane/core/exchange/Exchange.java +++ b/core/src/main/java/com/predic8/membrane/core/exchange/Exchange.java @@ -77,7 +77,7 @@ public Exchange(AbstractHttpHandler handler) { /** * For HttpResendRunnable * - * @param original + * @param original Exchange */ public Exchange(Exchange original, AbstractHttpHandler handler) { super(original); @@ -234,7 +234,7 @@ public long getId() { } @Override - public AbstractExchange createSnapshot(Runnable bodyUpdatedCallback, BodyCollectingMessageObserver.Strategy strategy, long limit) throws Exception { + public AbstractExchange createSnapshot(Runnable bodyUpdatedCallback, BodyCollectingMessageObserver.Strategy strategy, long limit) { Exchange exc = updateCopy(this, new Exchange(null), bodyUpdatedCallback, strategy, limit); exc.setId(this.getId()); return exc; @@ -259,4 +259,11 @@ public Exception[] getNodeExceptions() { public void setNodeExceptions(Exception[] nodeExceptions) { this.nodeExceptions = nodeExceptions; } + + public String getInboundProtocol() { + if (getRule().getSslInboundContext() == null) + return "http"; + else + return "https"; + } } diff --git a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/APIProxy.java b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/APIProxy.java index 6a839fe61..9982e864b 100644 --- a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/APIProxy.java +++ b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/APIProxy.java @@ -28,8 +28,6 @@ import java.net.*; import java.util.*; -import static com.predic8.membrane.core.openapi.serviceproxy.APIProxy.Spec.YesNoOpenAPIOption.*; - /** * @description The APIProxy extends the serviceProxy with API related functions like OpenAPI support. * @@ -59,100 +57,10 @@ protected AbstractProxy getNewInstance() { return new APIProxy(); } - /** - * @description Reads an OpenAPI description and deploys an API with the information of it. - */ - @MCElement(name = "openapi", topLevel = false) - public static class Spec { - - String location; - String dir; - YesNoOpenAPIOption validateRequests = ASINOPENAPI; - YesNoOpenAPIOption validateResponses = ASINOPENAPI; - YesNoOpenAPIOption validationDetails = ASINOPENAPI; - - public Spec() { - } - - public String getLocation() { - return location; - } - - /** - * @description Filename or URL pointing to an OpenAPI document. Relative filenames use the %MEMBRANE_HOME%/conf folder as base directory. - * @example openapi/fruitstore-v1.yaml, https://api.predic8.de/shop/swagger - */ - @MCAttribute() - public void setLocation(String location) { - this.location = location; - } - - public String getDir() { - return dir; - } - - /** - * @description Directory containing OpenAPI definitions to deploy. - * @example openapi - */ - @MCAttribute() - public void setDir(String dir) { - this.dir = dir; - } - - public YesNoOpenAPIOption getValidateRequests() { - return validateRequests; - } - - /** - * @description Turn validation of requests on or off. - * @example yes - * @default no - */ - @MCAttribute - public void setValidateRequests(YesNoOpenAPIOption validateRequests) { - this.validateRequests = validateRequests; - } - - @SuppressWarnings("unused") - public YesNoOpenAPIOption getValidateResponses() { - return validateResponses; - } - - /** - * @description Turn validation of responses on or off. - * @example yes - * @default no - */ - @MCAttribute() - public void setValidateResponses(YesNoOpenAPIOption validateResponses) { - this.validateResponses = validateResponses; - } - - /** - * @description Show details of the validation to the caller. - * @example yes - * @default no - */ - @MCAttribute() - public void setValidationDetails(YesNoOpenAPIOption validationDetails) { - this.validationDetails = validationDetails; - } - - public YesNoOpenAPIOption getValidationDetails() { - return validationDetails; - } - - public enum YesNoOpenAPIOption { - YES, - NO, - ASINOPENAPI - } - } - protected List specs = new ArrayList<>(); + protected List specs = new ArrayList<>(); - public List getSpecs() { + public List getSpecs() { return specs; } @@ -160,7 +68,7 @@ public List getSpecs() { * @description Deploys an API from an OpenAPI document. */ @MCChildElement(order = 25) - public void setSpecs(List specs) { + public void setSpecs(List specs) { this.specs = specs; } diff --git a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIInterceptor.java b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIInterceptor.java index ea85de929..d70c76448 100644 --- a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIInterceptor.java +++ b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIInterceptor.java @@ -43,10 +43,10 @@ public class OpenAPIInterceptor extends AbstractInterceptor { - protected final APIProxy proxy; + protected final APIProxy apiProxy; - public OpenAPIInterceptor(APIProxy proxy) { - this.proxy = proxy; + public OpenAPIInterceptor(APIProxy apiProxy) { + this.apiProxy = apiProxy; } @Override @@ -61,7 +61,7 @@ public Outcome handleRequest(Exchange exc) throws Exception { return RETURN; } - OpenAPIRecord rec = proxy.getBasePaths().get(basePath); + OpenAPIRecord rec = apiProxy.getBasePaths().get(basePath); // If OpenAPIProxy has a Element use this for routing otherwise // take the urls from the info.servers field in the OpenAPI document. @@ -71,7 +71,7 @@ public Outcome handleRequest(Exchange exc) throws Exception { ValidationErrors errors = validateRequest(rec.api, exc); if (!errors.isEmpty()) { - proxy.statisticCollector.collect(errors); + apiProxy.statisticCollector.collect(errors); return returnErrors(exc, errors, REQUEST, validationDetails(rec.api)); } @@ -81,7 +81,7 @@ public Outcome handleRequest(Exchange exc) throws Exception { } private boolean hasProxyATargetElement() { - return proxy.getTarget() != null && (proxy.getTarget().getHost() != null || proxy.getTarget().getUrl() != null); + return apiProxy.getTarget() != null && (apiProxy.getTarget().getHost() != null || apiProxy.getTarget().getUrl() != null); } @Override @@ -92,14 +92,14 @@ public Outcome handleResponse(Exchange exc) throws Exception { if (errors != null && errors.hasErrors()) { exc.getResponse().setStatusCode(500); // A validation error in the response is a server error! - proxy.statisticCollector.collect(errors); + apiProxy.statisticCollector.collect(errors); return returnErrors(exc, errors, RESPONSE, validationDetails(rec.api)); } return CONTINUE; } protected String getMatchingBasePath(Exchange exc) { - for (String basePath : proxy.getBasePaths().keySet()) { + for (String basePath : apiProxy.getBasePaths().keySet()) { if (exc.getRequest().getUri().startsWith(basePath)) { return basePath; } @@ -196,7 +196,7 @@ public String getLongDescription() { sb.append(""); sb.append(""); - for (Map.Entry entry : proxy.getBasePaths().entrySet()) { + for (Map.Entry entry : apiProxy.getBasePaths().entrySet()) { sb.append(""); sb.append("
APIBase PathValidation
"); sb.append(entry.getValue().api.getInfo().getTitle()); diff --git a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIPublisherInterceptor.java b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIPublisherInterceptor.java index 0f1a6d41b..802129a8b 100644 --- a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIPublisherInterceptor.java +++ b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIPublisherInterceptor.java @@ -36,7 +36,6 @@ import static com.predic8.membrane.core.http.MimeType.*; import static com.predic8.membrane.core.interceptor.Outcome.*; import static com.predic8.membrane.core.openapi.util.OpenAPIUtil.*; -import static com.predic8.membrane.core.openapi.util.UriUtil.*; import static com.predic8.membrane.core.openapi.util.Utils.*; public class OpenAPIPublisherInterceptor extends AbstractInterceptor { @@ -83,7 +82,7 @@ public Outcome handleRequest(Exchange exc) throws Exception { return handleOverviewOpenAPIDoc(exc); } - private Outcome handleOverviewOpenAPIDoc(Exchange exc) throws JsonProcessingException, URISyntaxException { + private Outcome handleOverviewOpenAPIDoc(Exchange exc) throws IOException, URISyntaxException { Matcher m = patternMeta.matcher(exc.getRequest().getUri()); if (!m.matches()) { // No id specified if (acceptsHtmlExplicit(exc)) { @@ -119,95 +118,31 @@ private Outcome returnNoFound(Exchange exc, String id) { return RETURN; } - private Outcome returnOpenApiAsYaml(Exchange exc, OpenAPIRecord rec) throws JsonProcessingException, URISyntaxException { - rewriteOpenAPIaccordingToRequest(exc, rec); + private Outcome returnOpenApiAsYaml(Exchange exc, OpenAPIRecord rec) throws IOException, URISyntaxException { + rec.rewriteOpenAPI(exc, getRouter().getUriFactory()); exc.setResponse(Response.ok().contentType(APPLICATION_X_YAML).body(omYaml.writeValueAsBytes(rec.node)).build()); return RETURN; } - protected void rewriteOpenAPIaccordingToRequest(Exchange exc, OpenAPIRecord rec) throws URISyntaxException { - rewriteOpenAPIVersion3(exc, rec); - rewriteSwaggerVersion2(exc, rec); - } - - private void rewriteSwaggerVersion2(Exchange exc, OpenAPIRecord rec) { - // Rewrite OpenAPI 2.X - JsonNode host = rec.node.get("host"); - if (host == null) - return; - - String rewrittenHost = rewriteHost(exc); - ((ObjectNode) rec.node).put("host", rewrittenHost); - - // Add protocol http or https - ArrayNode schemes = ((ObjectNode) rec.node).putArray("schemes"); - schemes.add(getProtocol(exc)); - - log.debug("Rewriting {} to {}", host, rewrittenHost); - } - - private void rewriteOpenAPIVersion3(Exchange exc, OpenAPIRecord rec) throws URISyntaxException { - JsonNode servers = rec.node.get("servers"); - if (servers == null) - return; - - for (JsonNode server : servers) { - String serverUrl = server.get("url").asText(); - String rewrittenUrl = rewriteUrl(exc, serverUrl); - ((ObjectNode) server).put("url", rewrittenUrl); - log.debug("Rewriting {} to {}", serverUrl, rewrittenUrl); - } - } - - /** - * Rewrites URL from OpenAPI 3.X - * @param exc Exchange - * @param url URL to rewrite - * @return Rewritten URL - * @throws URISyntaxException syntax error ín URL - */ - protected String rewriteUrl(Exchange exc, String url) throws URISyntaxException { - return rewrite(router.getUriFactory(), url, - getProtocol(exc), - exc.getOriginalHostHeaderHost(), - exc.getOriginalHostHeaderPort()); - } - - /** - * Rewrites Host from Swagger 2.X - * @param exc Exchange - * @return Rewritten host with port - */ - protected String rewriteHost(Exchange exc) { - return exc.getOriginalHostHeaderHost() + ":" + exc.getOriginalHostHeaderPort(); - } - - private String getProtocol(Exchange exc) { - if (exc.getRule().getSslInboundContext() == null) - return "http"; - else - return "https"; - } - private Outcome handleSwaggerUi(Exchange exc) { Matcher m = patternUI.matcher(exc.getRequest().getUri()); // No id specified if (!m.matches()) { - Map details = new HashMap<>(); - details.put("message","Please specify an id of an OpenAPI document. Path should match this pattern: /api-doc/ui/<>"); - exc.setResponse(createProblemDetails(404,"/openapi/wrong-id","No OpenAPI document id",details)); + Map details = new HashMap<>(); + details.put("message", "Please specify an id of an OpenAPI document. Path should match this pattern: /api-doc/ui/<>"); + exc.setResponse(createProblemDetails(404, "/openapi/wrong-id", "No OpenAPI document id", details)); return RETURN; } // /api-doc/ui/(.*) String id = m.group(1); - log.info("OpenAPI with id {} requested",id); + log.info("OpenAPI with id {} requested", id); OpenAPIRecord record = apis.get(id); if (record == null) { - return returnNoFound(exc,id); + return returnNoFound(exc, id); } exc.setResponse(Response.ok().contentType(HTML_UTF_8).body(renderSwaggerUITemplate(id, record.api)).build()); diff --git a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecord.java b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecord.java index ec1d28ac0..b0a6452e2 100644 --- a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecord.java +++ b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecord.java @@ -17,7 +17,13 @@ package com.predic8.membrane.core.openapi.serviceproxy; import com.fasterxml.jackson.databind.*; +import com.predic8.membrane.core.exchange.*; +import com.predic8.membrane.core.util.*; import io.swagger.v3.oas.models.*; +import org.slf4j.*; + +import java.io.*; +import java.net.*; import static com.predic8.membrane.core.openapi.util.OpenAPIUtil.*; @@ -33,15 +39,23 @@ public class OpenAPIRecord { */ JsonNode node; + OpenAPISpec spec; + /** * Version of the OpenAPI standard e.g. 2.0, 3.0.1 */ String version; - public OpenAPIRecord(OpenAPI api, JsonNode node) { + /** + * Used for tests + */ + public OpenAPIRecord() {} + + public OpenAPIRecord(OpenAPI api, JsonNode node, OpenAPISpec spec) { this.api = api; this.node = node; this.version = getOpenAPIVersion(node); + this.spec = spec; } public boolean isVersion2() { @@ -51,4 +65,9 @@ public boolean isVersion2() { public boolean isVersion3() { return version.startsWith("3"); } + + public JsonNode rewriteOpenAPI(Exchange exc, URIFactory uriFactory) throws URISyntaxException, IOException { + return spec.rewrite.rewrite(this,exc,uriFactory); + } + } diff --git a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecordFactory.java b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecordFactory.java index e57e9d51a..66e4f415e 100644 --- a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecordFactory.java +++ b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecordFactory.java @@ -31,7 +31,8 @@ import java.util.*; import static com.predic8.membrane.core.openapi.serviceproxy.APIProxy.*; -import static com.predic8.membrane.core.openapi.serviceproxy.APIProxy.Spec.YesNoOpenAPIOption.*; +import static com.predic8.membrane.core.openapi.serviceproxy.OpenAPISpec.YesNoOpenAPIOption.ASINOPENAPI; +import static com.predic8.membrane.core.openapi.serviceproxy.OpenAPISpec.YesNoOpenAPIOption.YES; import static com.predic8.membrane.core.openapi.util.OpenAPIUtil.*; import static com.predic8.membrane.core.util.FileUtil.*; import static java.lang.String.*; @@ -48,9 +49,9 @@ public OpenAPIRecordFactory(Router router) { this.router = router; } - public Map create(Collection specs) throws IOException { + public Map create(Collection specs) throws IOException { Map apiRecords = new LinkedHashMap<>(); - for (Spec spec : specs) { + for (OpenAPISpec spec : specs) { // Maybe a spec has both location and dir. addOpenApisFromLocation(apiRecords, spec); addOpenApisFromDirectory(apiRecords, spec); @@ -58,7 +59,7 @@ public Map create(Collection specs) throws IOExcept return apiRecords; } - private void addOpenApisFromDirectory(Map apiRecords, Spec spec) throws IOException { + private void addOpenApisFromDirectory(Map apiRecords, OpenAPISpec spec) throws IOException { if (spec.dir == null) return; @@ -75,7 +76,7 @@ private void addOpenApisFromDirectory(Map apiRecords, Spe } } - private void addOpenApisFromLocation(Map apiRecords, Spec spec) { + private void addOpenApisFromLocation(Map apiRecords, OpenAPISpec spec) { if (spec.location == null) return; @@ -112,19 +113,19 @@ private String getUniqueId(Map apiRecords, OpenAPIRecord return id; } - private OpenAPIRecord create(Spec spec) throws IOException { - OpenAPIRecord record = new OpenAPIRecord(getOpenAPI(router, spec), getSpec(router, spec)); + private OpenAPIRecord create(OpenAPISpec spec) throws IOException { + OpenAPIRecord record = new OpenAPIRecord(getOpenAPI(router, spec), getSpec(router, spec),spec); setExtentsionOnAPI(spec, record.api); return record; } - private OpenAPIRecord create(Spec spec, File file) throws IOException { - OpenAPIRecord record = new OpenAPIRecord(parseFileAsOpenAPI(file), getSpec(file)); + private OpenAPIRecord create(OpenAPISpec spec, File file) throws IOException { + OpenAPIRecord record = new OpenAPIRecord(parseFileAsOpenAPI(file), getSpec(file),spec); setExtentsionOnAPI(spec, record.api); return record; } - private OpenAPI getOpenAPI(Router router, Spec spec) throws ResourceRetrievalException { + private OpenAPI getOpenAPI(Router router, OpenAPISpec spec) throws ResourceRetrievalException { OpenAPI openAPI = new OpenAPIParser().readContents(readInputStream(getInputStreamForLocation(router, spec.location)), null, null).getOpenAPI(); @@ -143,7 +144,7 @@ private InputStream getInputStreamForLocation(Router router, String location) th return router.getResolverMap().resolve(ResolverMap.combine(router.getBaseLocation(), location)); } - private JsonNode getSpec(Router router, Spec spec) throws IOException { + private JsonNode getSpec(Router router, OpenAPISpec spec) throws IOException { return omYaml.readTree(getInputStreamForLocation(router, spec.location)); } @@ -151,7 +152,7 @@ private JsonNode getSpec(File file) throws IOException { return omYaml.readTree(file); } - private void setExtentsionOnAPI(Spec spec, OpenAPI api) { + private void setExtentsionOnAPI(OpenAPISpec spec, OpenAPI api) { if (api.getExtensions() == null) { api.setExtensions(new HashMap<>()); } @@ -170,7 +171,7 @@ private Map getXValidationExtension(OpenAPI api) { return extension; } - private Map updateExtension(Map extension, Spec spec) { + private Map updateExtension(Map extension, OpenAPISpec spec) { if (spec.validationDetails != ASINOPENAPI) extension.put(VALIDATION_DETAILS, toYesNo(spec.validationDetails)); @@ -187,7 +188,7 @@ private Map updateExtension(Map extension, Spec return extension; } - private boolean toYesNo(Spec.YesNoOpenAPIOption option) { + private boolean toYesNo(OpenAPISpec.YesNoOpenAPIOption option) { return option == YES; } diff --git a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPISpec.java b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPISpec.java new file mode 100644 index 000000000..970f20541 --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPISpec.java @@ -0,0 +1,122 @@ +/* + * Copyright 2022 predic8 GmbH, www.predic8.com + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.predic8.membrane.core.openapi.serviceproxy; + +import com.predic8.membrane.annot.*; + +import static com.predic8.membrane.core.openapi.serviceproxy.OpenAPISpec.YesNoOpenAPIOption.ASINOPENAPI; + +/** + * @description Reads an OpenAPI description and deploys an API with the information of it. + */ +@MCElement(name = "openapi", topLevel = false) +public class OpenAPISpec { + + public String location; + public String dir; + public YesNoOpenAPIOption validateRequests = ASINOPENAPI; + YesNoOpenAPIOption validateResponses = ASINOPENAPI; + YesNoOpenAPIOption validationDetails = ASINOPENAPI; + Rewrite rewrite = new Rewrite(); + + public OpenAPISpec() { + } + + @MCChildElement(order = 50) + public void setRewrite(Rewrite rewrite) { + this.rewrite = rewrite; + } + + public Rewrite getRewrite() { + return rewrite; + } + + public String getLocation() { + return location; + } + + /** + * @description Filename or URL pointing to an OpenAPI document. Relative filenames use the %MEMBRANE_HOME%/conf folder as base directory. + * @example openapi/fruitstore-v1.yaml, https://api.predic8.de/shop/swagger + */ + @MCAttribute() + public void setLocation(String location) { + this.location = location; + } + + public String getDir() { + return dir; + } + + /** + * @description Directory containing OpenAPI definitions to deploy. + * @example openapi + */ + @MCAttribute() + public void setDir(String dir) { + this.dir = dir; + } + + public YesNoOpenAPIOption getValidateRequests() { + return validateRequests; + } + + /** + * @description Turn validation of requests on or off. + * @example yes + * @default no + */ + @MCAttribute + public void setValidateRequests(YesNoOpenAPIOption validateRequests) { + this.validateRequests = validateRequests; + } + + @SuppressWarnings("unused") + public YesNoOpenAPIOption getValidateResponses() { + return validateResponses; + } + + /** + * @description Turn validation of responses on or off. + * @example yes + * @default no + */ + @MCAttribute() + public void setValidateResponses(YesNoOpenAPIOption validateResponses) { + this.validateResponses = validateResponses; + } + + /** + * @description Show details of the validation to the caller. + * @example yes + * @default no + */ + @MCAttribute() + public void setValidationDetails(YesNoOpenAPIOption validationDetails) { + this.validationDetails = validationDetails; + } + + public YesNoOpenAPIOption getValidationDetails() { + return validationDetails; + } + + public enum YesNoOpenAPIOption { + YES, + NO, + ASINOPENAPI + } +} \ No newline at end of file diff --git a/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/Rewrite.java b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/Rewrite.java new file mode 100644 index 000000000..c9b33741a --- /dev/null +++ b/core/src/main/java/com/predic8/membrane/core/openapi/serviceproxy/Rewrite.java @@ -0,0 +1,175 @@ +/* + * Copyright 2023 predic8 GmbH, www.predic8.com + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.predic8.membrane.core.openapi.serviceproxy; + +import com.fasterxml.jackson.databind.*; +import com.fasterxml.jackson.databind.node.*; +import com.predic8.membrane.annot.*; +import com.predic8.membrane.core.exchange.*; +import com.predic8.membrane.core.openapi.util.*; +import com.predic8.membrane.core.util.*; +import org.slf4j.*; + +import java.io.*; +import java.net.*; + +import static java.util.Objects.*; + +/** + * @description + */ +@MCElement(name = "rewrite", topLevel = false) +public class Rewrite { + + private static final Logger log = LoggerFactory.getLogger(Rewrite.class.getName()); + + Integer port; + String protocol; + String host; + + public JsonNode rewrite(OpenAPIRecord rec, Exchange exc, URIFactory uriFactory) throws URISyntaxException, IOException { + if (rec.isVersion3()) + return rewriteOpenAPI3(exc, uriFactory, rec.node); + + return rewriteSwagger2(exc, rec.node); + } + + private JsonNode rewriteOpenAPI3(Exchange exc, URIFactory uriFactory, JsonNode node) throws URISyntaxException { + // Expensive cloning before null check is fine cause most of OpenAPIs will have servers. + // Caching should not be needed cause the OpenAPI is not so often retrieved. Maybe practice + // will prove that wrong. + JsonNode rewritten = node.deepCopy(); + for (JsonNode server : rewritten.get("servers")) { + rewriteServerEntry(exc, uriFactory, server); + } + return rewritten; + } + + private void rewriteServerEntry(Exchange exc, URIFactory uriFactory, JsonNode server) throws URISyntaxException { + String url = server.get("url").asText(); + String rewritten = rewriteUrl(exc, url, uriFactory); + ((ObjectNode) server).put("url", rewritten); + log.debug("Rewriting {} to {}", url, rewritten); + } + + /** + * Rewrites URL from OpenAPI 3.X + * + * @param exc Exchange + * @param url URL to rewrite + * @param uriFactory URIFactory + * @return Rewritten URL + * @throws URISyntaxException syntax error ín URL + */ + protected String rewriteUrl(Exchange exc, String url, URIFactory uriFactory) throws URISyntaxException { + return UriUtil.rewrite(uriFactory, url, rewriteProtocol(exc.getInboundProtocol()), rewriteHost(exc.getOriginalHostHeaderHost()), rewritePort(exc.getOriginalHostHeaderPort())); + } + + private JsonNode rewriteSwagger2(Exchange exc, JsonNode node) { + JsonNode rewritten = node.deepCopy(); + rewriteHostAndPortSwagger2(exc, rewritten); + rewriteSchemeSwagger2(exc, rewritten); + return rewritten; + } + + private void rewriteSchemeSwagger2(Exchange exc, JsonNode node) { + // Add protocol http or https + ArrayNode schemes = ((ObjectNode) node).putArray("schemes"); + schemes.add(exc.getInboundProtocol()); + } + + private void rewriteHostAndPortSwagger2(Exchange exc, JsonNode rewrittenJson) { + JsonNode host = rewrittenJson.get("host"); + if (host == null) + return; + + String rewrittenHost = getRewrittenHostAndPortSwagger2(exc); + ((ObjectNode) rewrittenJson).put("host", rewrittenHost); + log.debug("Rewriting {} to {}", host, rewrittenHost); + } + + /** + * Rewrites Host from Swagger 2.X + * + * @param exc Exchange + * @return Rewritten host with port + */ + protected String getRewrittenHostAndPortSwagger2(Exchange exc) { + return rewriteHost(exc.getOriginalHostHeaderHost()) + ":" + rewritePort(exc.getOriginalHostHeaderPort()); + } + + /** + * Rewrites the protocol if there was a value given. + * + * @param protocol from the OpenAPI + * @return rewritten value + */ + public String rewriteProtocol(String protocol) { + return requireNonNullElse(this.protocol, protocol); + } + + /** + * Rewrites the host if there was a value given. + * + * @param host from the OpenAPI + * @return rewritten value + */ + public String rewriteHost(String host) { + return requireNonNullElse(this.host, host); + } + + /** + * Rewrites the port if there was a value given. + * + * @param port from the OpenAPI + * @return rewritten value + */ + public String rewritePort(String port) { + if (this.port != null) { + return this.port.toString(); + } + return port; + } + + + public int getPort() { + return port; + } + + @MCAttribute() + public void setPort(int port) { + this.port = port; + } + + public String getProtocol() { + return protocol; + } + + @MCAttribute() + public void setProtocol(String protocol) { + this.protocol = protocol; + } + + public String getHost() { + return host; + } + + @MCAttribute() + public void setHost(String host) { + this.host = host; + } +} diff --git a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/APIProxyTest.java b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/APIProxyTest.java index 8dad6b7cd..c876446ac 100644 --- a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/APIProxyTest.java +++ b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/APIProxyTest.java @@ -22,8 +22,9 @@ import java.util.*; -import static com.predic8.membrane.core.openapi.serviceproxy.APIProxy.Spec.YesNoOpenAPIOption.*; import static com.predic8.membrane.core.openapi.serviceproxy.APIProxy.*; +import static com.predic8.membrane.core.openapi.serviceproxy.OpenAPISpec.YesNoOpenAPIOption.NO; +import static com.predic8.membrane.core.openapi.serviceproxy.OpenAPISpec.YesNoOpenAPIOption.YES; import static com.predic8.membrane.core.openapi.util.TestUtils.*; import static org.junit.jupiter.api.Assertions.*; @@ -39,7 +40,7 @@ public void setUp() { @Test public void noOptionsNoExtensions() throws Exception { - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.location = "src/test/resources/openapi/openapi-proxy/no-extensions.yml"; APIProxy proxy = createProxy(router,spec); @@ -61,7 +62,7 @@ public void noOptionsNoExtensions() throws Exception { @Test public void validationRequestNoExtensions() throws Exception { - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.location = "src/test/resources/openapi/openapi-proxy/no-extensions.yml"; spec.validateRequests = YES; @@ -83,7 +84,7 @@ public void validationRequestNoExtensions() throws Exception { @Test public void validationResponsesNoExtensions() throws Exception { - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.location = "src/test/resources/openapi/openapi-proxy/no-extensions.yml"; spec.validateResponses = YES; @@ -106,7 +107,7 @@ public void validationResponsesNoExtensions() throws Exception { @Test public void validationAllNoExtensions() throws Exception { - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.location = "src/test/resources/openapi/openapi-proxy/no-extensions.yml"; spec.validateRequests = YES; spec.validateResponses = YES; @@ -129,7 +130,7 @@ public void validationAllNoExtensions() throws Exception { @Test public void requestsExtensions() throws Exception { - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.location = "src/test/resources/openapi/openapi-proxy/validate-requests-extensions.yml"; APIProxy proxy = createProxy(router,spec); @@ -150,7 +151,7 @@ public void requestsExtensions() throws Exception { @Test public void responsesExtensions() throws Exception { - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.location = "src/test/resources/openapi/openapi-proxy/validate-responses-extensions.yml"; APIProxy proxy = createProxy(router,spec); @@ -172,7 +173,7 @@ public void responsesExtensions() throws Exception { @Test public void validationRequestNoDetailsNoExtensions() throws Exception { - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.location = "src/test/resources/openapi/openapi-proxy/no-extensions.yml"; spec.validateRequests = YES; spec.validationDetails = NO; @@ -196,7 +197,7 @@ public void validationRequestNoDetailsNoExtensions() throws Exception { @Test public void validationDetailsFalseExtensions() throws Exception { - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.location = "src/test/resources/openapi/openapi-proxy/validation-details-false-extensions.yml"; spec.validateRequests = YES; @@ -238,8 +239,8 @@ public void oneOpenAPIWithMultipleServerUrlsSharingTheSamePath() throws Exceptio api.setSpecs(List.of(extracted("api-c-multiple-server-urls.yml"))); } - private APIProxy.Spec extracted(String location) { - APIProxy.Spec spec = new APIProxy.Spec(); + private OpenAPISpec extracted(String location) { + OpenAPISpec spec = new OpenAPISpec(); spec.location = location; return spec; } diff --git a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIInterceptorTest.java b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIInterceptorTest.java index 38461faea..ed667e189 100644 --- a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIInterceptorTest.java +++ b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIInterceptorTest.java @@ -29,8 +29,6 @@ import static com.predic8.membrane.core.http.MimeType.APPLICATION_JSON; import static com.predic8.membrane.core.http.MimeType.APPLICATION_PROBLEM_JSON; import static com.predic8.membrane.core.interceptor.Outcome.*; -import static com.predic8.membrane.core.openapi.serviceproxy.APIProxy.Spec.YesNoOpenAPIOption.NO; -import static com.predic8.membrane.core.openapi.serviceproxy.APIProxy.Spec.YesNoOpenAPIOption.YES; import static com.predic8.membrane.core.openapi.util.JsonUtil.*; import static com.predic8.membrane.core.openapi.util.TestUtils.*; import static org.junit.jupiter.api.Assertions.*; @@ -39,9 +37,9 @@ public class OpenAPIInterceptorTest { Router router; - APIProxy.Spec specInfoServers; - APIProxy.Spec specInfo3Servers; - APIProxy.Spec specCustomers; + OpenAPISpec specInfoServers; + OpenAPISpec specInfo3Servers; + OpenAPISpec specCustomers; Exchange exc = new Exchange(null); OpenAPIInterceptor interceptor1Server; @@ -52,13 +50,13 @@ public void setUp() throws Exception { router = new Router(); router.setUriFactory(new URIFactory()); - specInfoServers = new APIProxy.Spec(); + specInfoServers = new OpenAPISpec(); specInfoServers.location = "src/test/resources/openapi/specs/info-servers.yml"; - specInfo3Servers = new APIProxy.Spec(); + specInfo3Servers = new OpenAPISpec(); specInfo3Servers.location = "src/test/resources/openapi/specs/info-3-servers.yml"; - specCustomers = new APIProxy.Spec(); + specCustomers = new OpenAPISpec(); specCustomers.location = "src/test/resources/openapi/specs/customers.yml"; exc.setRequest(new Request.Builder().method("GET").build()); @@ -128,7 +126,7 @@ public void destinationsTargetSet() throws Exception { @Test public void validateRequest() throws Exception { - specCustomers.validateRequests = YES; + specCustomers.validateRequests = OpenAPISpec.YesNoOpenAPIOption.YES; Map customer = new HashMap<>(); customer.put("id","CUST-7"); @@ -151,20 +149,20 @@ public void validateRequest() throws Exception { @SuppressWarnings({"unchecked"}) @Test public void validateResponse() throws Exception { - specCustomers.validateResponses = YES; + specCustomers.validateResponses = OpenAPISpec.YesNoOpenAPIOption.YES; assertEquals("PUT", getMapFromResponse(callPut(specCustomers)).get("method")); testValidationResults(getMapFromResponse(callPut(specCustomers)), "RESPONSE"); } @Test public void validateResponseLessDetails() throws Exception { - specCustomers.validateResponses = YES; - specCustomers.validationDetails = NO; + specCustomers.validateResponses = OpenAPISpec.YesNoOpenAPIOption.YES; + specCustomers.validationDetails = OpenAPISpec.YesNoOpenAPIOption.NO; assertEquals("Message validation failed!", getMapFromResponse(callPut(specCustomers)).get("error")); } @NotNull - private Exchange callPut(APIProxy.Spec spec) throws Exception { + private Exchange callPut(OpenAPISpec spec) throws Exception { Exchange exc = new Exchange(null); exc.setOriginalRequestUri("/customers"); exc.setRequest(new Request.Builder().method("PUT").url(new URIFactory(), "/customers").contentType(APPLICATION_JSON).build()); diff --git a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIPublisherInterceptorTest.java b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIPublisherInterceptorTest.java index 38b3e5d04..7df3fc40e 100644 --- a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIPublisherInterceptorTest.java +++ b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIPublisherInterceptorTest.java @@ -27,7 +27,6 @@ import org.junit.jupiter.api.*; import java.io.*; -import java.net.*; import java.util.*; import static com.predic8.membrane.core.http.MimeType.*; @@ -50,11 +49,11 @@ void setUp() throws Exception { router.setUriFactory(new URIFactory()); router.setBaseLocation(""); openAPIRecordFactory = new OpenAPIRecordFactory(router); - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.setDir("src/test/resources/openapi/specs"); records = openAPIRecordFactory.create(Collections.singletonList(spec)); - interceptor = new OpenAPIPublisherInterceptor(records); + interceptor = new OpenAPIPublisherInterceptor( records); interceptor.init(router); get.setRequest(new Request.Builder().method("GET").build()); @@ -64,7 +63,7 @@ void setUp() throws Exception { @Test public void constuctor() { - assertEquals(27, interceptor.apis.size()); + assertTrue(interceptor.apis.size() >= 27); assertNotNull(interceptor.apis.get("references-test-v1-0")); assertNotNull(interceptor.apis.get("strings-test-api-v1-0")); assertNotNull(interceptor.apis.get("extension-sample-v1-4")); @@ -77,7 +76,7 @@ public void constuctor() { public void getApiDirectory() throws Exception { get.getRequest().setUri(OpenAPIPublisherInterceptor.PATH); assertEquals( RETURN, interceptor.handleRequest(get)); - assertEquals(27, TestUtils.getMapFromResponse(get).size()); + assertTrue(TestUtils.getMapFromResponse(get).size() >= 27); } @Test @@ -134,39 +133,4 @@ public void getApiById() throws Exception { private JsonNode getJsonFromYamlResponse(Exchange exc) throws IOException { return omYaml.readTree(exc.getResponse().getBody().getContent()); } - - @Test - public void rewriteOpenAPIAccordingToRequestTest() throws URISyntaxException { - OpenAPIRecord rec = records.get("servers-1-api-v1-0"); - interceptor.rewriteOpenAPIaccordingToRequest(get, rec); - assertEquals("http://api.predic8.de/base/v2",rec.node.get("servers").get(0).get("url").asText()); - assertEquals("Test System",rec.node.get("servers").get(0).get("description").asText()); - } - - @Test - public void rewriteOpenAPIAccordingToRequest3Servers() throws URISyntaxException { - OpenAPIRecord rec = records.get("servers-3-api-v1-0"); - interceptor.rewriteOpenAPIaccordingToRequest(get, rec); - assertEquals(3,rec.node.get("servers").size()); - assertEquals("http://api.predic8.de/foo",rec.node.get("servers").get(0).get("url").asText()); - assertEquals("http://api.predic8.de/foo",rec.node.get("servers").get(1).get("url").asText()); - assertEquals("http://api.predic8.de/foo",rec.node.get("servers").get(2).get("url").asText()); - } - - @Test - void rewriteRequestHostHeaderWithoutPort() throws URISyntaxException { - OpenAPIRecord rec = records.get("servers-3-api-v1-0"); - get.setOriginalHostHeader("api.predic8.de"); - interceptor.rewriteOpenAPIaccordingToRequest(get, rec); - JsonNode servers = rec.node.get("servers"); - assertEquals("http://api.predic8.de/foo", servers.get(0).get("url").textValue()); - assertEquals("http://api.predic8.de/foo", servers.get(1).get("url").textValue()); - assertEquals("http://api.predic8.de/foo", servers.get(2).get("url").textValue()); - } - - @Test - void rewriteUrl() throws URISyntaxException { - assertEquals("http://api.predic8.de/foo",interceptor.rewriteUrl(get,"http://localhost:3000/foo")); - assertEquals("http://api.predic8.de/foo",interceptor.rewriteUrl(get,"http://localhost/foo")); - } } \ No newline at end of file diff --git a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecordTest.java b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecordTest.java index e6371760d..579540d08 100644 --- a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecordTest.java +++ b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/OpenAPIRecordTest.java @@ -16,6 +16,11 @@ package com.predic8.membrane.core.openapi.serviceproxy; +import com.predic8.membrane.core.*; +import com.predic8.membrane.core.exchange.*; +import com.predic8.membrane.core.http.*; +import com.predic8.membrane.core.rules.*; +import com.predic8.membrane.core.util.*; import org.junit.jupiter.api.*; import java.io.*; @@ -25,16 +30,29 @@ class OpenAPIRecordTest { + Exchange get = new Exchange(null); + + @BeforeEach + void setUp() { + Router router = new Router(); + router.setUriFactory(new URIFactory()); + router.setBaseLocation(""); + + get.setRequest(new Request.Builder().method("GET").build()); + get.setRule(new NullRule()); + get.setOriginalHostHeader("api.predic8.de:80"); + } + @Test void isVersion3() throws IOException { - OpenAPIRecord rec = new OpenAPIRecord(null, getYAMLResource(this,"/openapi/specs/array.yml")); + OpenAPIRecord rec = new OpenAPIRecord(null, getYAMLResource(this,"/openapi/specs/array.yml"), null); assertEquals("3.0.2", rec.version); assertTrue(rec.isVersion3()); } @Test void isVersion2() throws IOException { - OpenAPIRecord rec = new OpenAPIRecord(null, getYAMLResource(this,"/openapi/specs/fruitshop-swagger-2.0.json")); + OpenAPIRecord rec = new OpenAPIRecord(null, getYAMLResource(this,"/openapi/specs/fruitshop-swagger-2.0.json"), null); assertEquals("2.0", rec.version); assertTrue(rec.isVersion2()); } diff --git a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/RewriteTest.java b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/RewriteTest.java new file mode 100644 index 000000000..198be1010 --- /dev/null +++ b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/RewriteTest.java @@ -0,0 +1,172 @@ +/* + * Copyright 2023 predic8 GmbH, www.predic8.com + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.predic8.membrane.core.openapi.serviceproxy; + +import com.fasterxml.jackson.databind.*; +import com.predic8.membrane.core.*; +import com.predic8.membrane.core.exchange.*; +import com.predic8.membrane.core.http.*; +import com.predic8.membrane.core.rules.*; +import com.predic8.membrane.core.util.*; +import org.jetbrains.annotations.*; +import org.junit.jupiter.api.*; + +import java.io.*; +import java.net.*; +import java.util.*; + +import static java.util.Collections.singletonList; +import static org.junit.jupiter.api.Assertions.*; + +class RewriteTest { + + Rewrite rewriteAll = new Rewrite(); + Rewrite rewriteNothing = new Rewrite(); + + OpenAPISpec rewritesAllSpec; + + Map records; + + Exchange get = new Exchange(null); + + @BeforeEach + void setUp() throws IOException { + rewriteAll.host = "predic8.de"; + rewriteAll.port = 8080; + rewriteAll.protocol = "https"; + + Router router = new Router(); + router.setUriFactory(new URIFactory()); + router.setBaseLocation(""); + + OpenAPIRecordFactory openAPIRecordFactory = new OpenAPIRecordFactory(router); + + rewritesAllSpec = getSpecRewriteAll(); + + OpenAPISpec spec = new OpenAPISpec(); + spec.setDir("src/test/resources/openapi/specs"); + records = openAPIRecordFactory.create(singletonList(spec)); + + get.setRequest(new Request.Builder().method("GET").build()); + get.setRule(new NullRule()); + get.setOriginalHostHeader("api.predic8.de:80"); + } + + @NotNull + private static OpenAPISpec getSpecRewriteAll() { + OpenAPISpec spec = new OpenAPISpec(); + spec.setLocation("src/test/resources/openapi/specs/info-servers.yml"); + Rewrite rewrite = new Rewrite(); + rewrite.setProtocol("https"); + rewrite.setPort(443); + rewrite.setHost("membrane-api.io"); + spec.setRewrite(rewrite); + return spec; + } + + @Test + void rewriteProtocol() { + assertEquals("https", rewriteAll.rewriteProtocol("http")); + assertEquals("http", rewriteNothing.rewriteProtocol("http")); + } + + @Test + void rewriteHost() { + assertEquals("predic8.de", rewriteAll.rewriteHost("membrane-api.io")); + assertEquals("membrane-api.io", rewriteNothing.rewriteHost("membrane-api.io")); + } + + @Test + void rewritePort() { + assertEquals("8080", rewriteAll.rewritePort("80")); + assertEquals("80", rewriteNothing.rewritePort("80")); + } + + /** + * Test if rewritting changes the OpenAPI document. + * Secures currency. + */ + @Test + void rewritingDoesNotChangeTheSpec() throws Exception { + OpenAPIRecord rec = records.get("servers-1-api-v1-0"); + JsonNode servers = rec.node.get("servers"); + + String urlBefore = servers.get(0).get("url").asText(); + + rec.rewriteOpenAPI(get, new URIFactory()); + + JsonNode servers1 = rec.node.get("servers"); + assertEquals(urlBefore, servers1.get(0).get("url").asText()); + } + + @Test + void rewriteSwagger2AccordingToRequestTest() throws Exception { + assertEquals("api.predic8.de:80", records.get("fruit-shop-api-swagger-2-v1-0-0").rewriteOpenAPI(get, new URIFactory()).get("host").asText()); + } + + @Test + void rewriteOpenAPIAccordingToRequestTest() throws Exception { + JsonNode servers = records.get("servers-1-api-v1-0").rewriteOpenAPI(get, new URIFactory()).get("servers"); + assertEquals(1,servers.size()); + assertEquals("http://api.predic8.de/base/v2", servers.get(0).get("url").asText()); + assertEquals("Test System", records.get("servers-1-api-v1-0").node.get("servers").get(0).get("description").asText()); + } + + @Test + void rewriteOpenAPIAccordingToRequest3Servers() throws Exception { + OpenAPIRecord rec = records.get("servers-3-api-v1-0"); + JsonNode servers = rec.rewriteOpenAPI(get, new URIFactory()).get("servers"); + assertEquals(3,servers.size()); + assertEquals("http://api.predic8.de/foo",servers.get(0).get("url").asText()); + assertEquals("http://api.predic8.de/foo",servers.get(1).get("url").asText()); + assertEquals("http://api.predic8.de/foo",servers.get(2).get("url").asText()); + } + + @Test + void rewriteOpenAPIAccordingToRewrite3Servers() throws Exception { + OpenAPIRecord rec = records.get("servers-3-api-v1-0"); + rec.spec.rewrite.setHost("membrane-api.do"); + rec.spec.rewrite.setPort(8443); + rec.spec.rewrite.setProtocol("https"); + JsonNode servers = rec.rewriteOpenAPI(get, new URIFactory()).get("servers"); + assertEquals(3,servers.size()); + assertEquals("https://membrane-api.do:8443/foo",servers.get(0).get("url").asText()); + assertEquals("https://membrane-api.do:8443/foo",servers.get(1).get("url").asText()); + assertEquals("https://membrane-api.do:8443/foo",servers.get(2).get("url").asText()); + } + + @Test + void rewriteRequestHostHeaderWithoutPort() throws Exception { + OpenAPIRecord rec = records.get("servers-3-api-v1-0"); + get.setOriginalHostHeader("api.predic8.de"); + JsonNode servers = rec.rewriteOpenAPI(get, new URIFactory()).get("servers"); + assertEquals("http://api.predic8.de/foo", servers.get(0).get("url").textValue()); + assertEquals("http://api.predic8.de/foo", servers.get(1).get("url").textValue()); + assertEquals("http://api.predic8.de/foo", servers.get(2).get("url").textValue()); + } + + @Test + void rewriteUrl() throws URISyntaxException { + assertEquals("http://api.predic8.de/foo", new Rewrite().rewriteUrl(get,"http://localhost:3000/foo", new URIFactory())); + assertEquals("http://api.predic8.de/foo", new Rewrite().rewriteUrl(get,"http://localhost/foo", new URIFactory())); + } + + @Test + void rewriteOpenAPI3WithNoServers() throws Exception { + assertTrue(records.get("no-servers-v1-0").rewriteOpenAPI(get, new URIFactory()).get("servers").isEmpty()); + } +} \ No newline at end of file diff --git a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/XMembraneExtentionTest.java b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/XMembraneExtentionTest.java index 43b930cb0..70e9ecd03 100644 --- a/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/XMembraneExtentionTest.java +++ b/core/src/test/java/com/predic8/membrane/core/openapi/serviceproxy/XMembraneExtentionTest.java @@ -16,11 +16,9 @@ package com.predic8.membrane.core.openapi.serviceproxy; -import com.fasterxml.jackson.databind.*; import com.predic8.membrane.core.*; import com.predic8.membrane.core.exchange.*; import com.predic8.membrane.core.http.*; -import io.swagger.v3.parser.*; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; @@ -38,7 +36,7 @@ public void setUp() throws IOException, ClassNotFoundException { Router router = new Router(); router.setBaseLocation(""); OpenAPIRecordFactory factory = new OpenAPIRecordFactory(router); - APIProxy.Spec spec = new APIProxy.Spec(); + OpenAPISpec spec = new OpenAPISpec(); spec.setDir("src/test/resources/openapi/specs"); Map records = factory.create(Collections.singletonList(spec)); diff --git a/core/src/test/java/com/predic8/membrane/core/openapi/util/TestUtils.java b/core/src/test/java/com/predic8/membrane/core/openapi/util/TestUtils.java index 57c8b98ee..e8c69718e 100644 --- a/core/src/test/java/com/predic8/membrane/core/openapi/util/TestUtils.java +++ b/core/src/test/java/com/predic8/membrane/core/openapi/util/TestUtils.java @@ -51,7 +51,7 @@ public static InputStream getResourceAsStream(Object obj, String fileName) { return obj.getClass().getResourceAsStream(fileName); } - public static APIProxy createProxy(Router router, APIProxy.Spec spec) throws Exception { + public static APIProxy createProxy(Router router, OpenAPISpec spec) throws Exception { APIProxy proxy = new APIProxy(); proxy.init(router); proxy.setSpecs(singletonList(spec)); diff --git a/core/src/test/resources/openapi/specs/info-no-servers.yml b/core/src/test/resources/openapi/specs/info-no-servers.yml new file mode 100644 index 000000000..8404c77ca --- /dev/null +++ b/core/src/test/resources/openapi/specs/info-no-servers.yml @@ -0,0 +1,11 @@ +openapi: '3.0.2' +info: + title: No Servers + version: '1.0' +servers: [] +paths: + /foo: + get: + responses: + '200': + description: OK From 62407fff629b3c7f0e50093170bca6f8d00c89de Mon Sep 17 00:00:00 2001 From: t-burch <119930761+t-burch@users.noreply.github.com> Date: Wed, 19 Jul 2023 15:23:02 +0200 Subject: [PATCH 4/6] Throttle example rewrite (#669) * Rewrote throttle sample * Rewording and typo fixes --------- Co-authored-by: Thomas Bayer --- distribution/examples/throttle/README.md | 51 ++++++++++++++++++---- distribution/examples/throttle/proxies.xml | 22 ++++++---- distribution/examples/throttle/timing.bat | 14 ++++++ distribution/examples/throttle/timing.sh | 14 ++++++ 4 files changed, 83 insertions(+), 18 deletions(-) create mode 100644 distribution/examples/throttle/timing.bat create mode 100644 distribution/examples/throttle/timing.sh diff --git a/distribution/examples/throttle/README.md b/distribution/examples/throttle/README.md index 1e7f27a69..9499d6628 100644 --- a/distribution/examples/throttle/README.md +++ b/distribution/examples/throttle/README.md @@ -1,21 +1,54 @@ -### THROTTLE INTERCEPTOR +# Throttle Plugin -With the `ThrottleInterceptor` you can delay and limit parallel requests. +This sample demonstrates how to throttle requests to an API or web app using the `throttle` plugin. -#### RUNNING THE EXAMPLE +## Running the Example -In this example we will throttle down access to a web page. +Execute the following steps: -To run the example execute the following steps: +1. Set sample directory (`/examples/throttle/`) as working directory. -Execute the following steps: +2. Execute the `service-proxy.sh` script or its Windows batch file equivalent. + +3. Run the `timing.sh` script or its Windows batch file equivalent. + +4. Observe as Curl carries out five requests each, initially without any throttling and subsequently with it. + +5. Check the recorded times, with throttling enabled, every request will take an additional second. + + +## How it is done + +Let's check the elements inside the `router` component in the `proxies.xml` file: + +```xml + + + + + + + + +``` + +Observe the two api components, which respectively proxy requests from the localhost ports 2000 and 3000 to the predic8 homepage. +The former api component additionally configures the throttle plugin for a delay of 1 second. -1. Execute `examples/throttle/service-proxy.bat` +Let's examine the commands executed by the timing script: +```sh +#!/bin/bash -2. Open the URL http://localhost:2000 in your browser. +echo "No throttling applied:" +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +# ... (x5) +echo "With throttling enabled:" +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +# ... (x5) +``` -3. Notice the delay when you browse. +It simply executes a curl request to the two Membrane addresses, five times each, though discarding the response body and displaying the total response time instead. --- See: diff --git a/distribution/examples/throttle/proxies.xml b/distribution/examples/throttle/proxies.xml index 0074f1249..c51d8ed49 100644 --- a/distribution/examples/throttle/proxies.xml +++ b/distribution/examples/throttle/proxies.xml @@ -1,16 +1,20 @@ - + - - - + + + + + + + - - \ No newline at end of file + + diff --git a/distribution/examples/throttle/timing.bat b/distribution/examples/throttle/timing.bat new file mode 100644 index 000000000..f62b596ca --- /dev/null +++ b/distribution/examples/throttle/timing.bat @@ -0,0 +1,14 @@ +@echo off + +echo "No throttling applied:" +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +echo "With throttling enabled:" +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 diff --git a/distribution/examples/throttle/timing.sh b/distribution/examples/throttle/timing.sh new file mode 100644 index 000000000..5c053d19f --- /dev/null +++ b/distribution/examples/throttle/timing.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +echo "No throttling applied:" +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:3000 +echo "With throttling enabled:" +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 +curl -o /dev/null -s -w "Total time: %{time_total}\n" http://localhost:2000 From b38e477fbee40b82aa6b8de8be247244e3c3b02a Mon Sep 17 00:00:00 2001 From: t-burch <119930761+t-burch@users.noreply.github.com> Date: Wed, 19 Jul 2023 15:24:20 +0200 Subject: [PATCH 5/6] Versioning example rewrite (#672) * Updated the versioned SOAP routing readme and proxies xml * Updated the SOAP message version transformation readme and proxies xml * Minor improvements * Fixed incorrect soap xml content type * Minor fixes * Fixed incorrect port in readme * Fixed incorrect error message description in readme * Minor fixes --------- Co-authored-by: Thomas Bayer --- .../examples/versioning/routing/README.md | 98 +++++++------- .../examples/versioning/routing/proxies.xml | 4 +- .../examples/versioning/xslt/README.md | 120 ++++++++++-------- 3 files changed, 120 insertions(+), 102 deletions(-) diff --git a/distribution/examples/versioning/routing/README.md b/distribution/examples/versioning/routing/README.md index aaeeffb9a..14a2e58a7 100644 --- a/distribution/examples/versioning/routing/README.md +++ b/distribution/examples/versioning/routing/README.md @@ -1,66 +1,72 @@ -### VERSIONING WEBSERVICES BY ROUTING TO DIFFERENT ENDPOINTS +# Versioning - Versioned SOAP Endpoint Routing -In this example we investigate how web services can be versioned by running -each version on its own separate end point. +This example walks you through configuring versioned endpoints in a SOAP service, with automatic routing by Membrane. -Membrane acts as a common proxy and routes SOAP requests to the service endpoint, -depending on the XML namespace used in the SOAP message body. -The version `1.1` of our "ContactService" uses the namespace -"http://predic8.com/contactService/v11", while version `2.0` uses -"http://predic8.com/contactService/v20". +## Running the Example +1. In a terminal, set `/examples/versioning/routing` as your working directory. -(Note that the WSDL files reference the actual endpoint (depending on the -service version), and using them in any client (e.g. SoapUI) therefore -bypasses the proxy.) +2. Compile and run the demo SOAP service: + ```sh + mvn clean compile assembly:single + java -jar ./target/routing-maven-1.0-SNAPSHOT.jar + ``` -#### RUNNING THE EXAMPLE -To run the example execute the following steps: +3. Now test if the two endpoints are reachable, in a separate terminal instance run: + ```sh + # Should respond with "Hello ... version 1.1" + curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:8080/ContactService/v11 + # Should Respond with "Hello ... version 2.0" + curl --header "Content-Type: text/xml" -d @request_v20.xml http://localhost:8080/ContactService/v20 + ``` + -Execute the following steps: +4. If you now send a `1.1` request to a `2.0` endpoint using the command below, you get a response containing "Cannot find dispatch method". + ```sh + curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:8080/ContactService/v20 + ``` + -1. To test the router we will use the command line tool curl that can transfer - data with URL syntax. You can download it from the following location: - - http://curl.haxx.se/download.html +6. Start Membrane by running `service-proxy.sh` or the `.bat` equivalent. -2. Open a new console window and execute: -``` -cd [MEMBRANE_HOME]\examples\versioning\routing-maven -mvn clean compile assembly:single -java -jar ./target/routing-maven-1.0-SNAPSHOT.jar -``` -3. Open yet another console window and run -``` -cd versioning\routing -curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:8080/ContactService/v11 -``` -You should see a response containing "Hello ... version 1.1". -4. Now run below command. You should see a response containing "Hello ... version 2.0". -``` - curl --header "Content-Type: text/xml" -d @request_v20.xml http://localhost:8080/ContactService/v20 -``` - +7. Return to another terminal, now send two requests to the single Membrane endpoint, + once using the `v1.1` namespace URL and another using the `2.0` version: + ```sh + curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:2000/ContactService + curl --header "Content-Type: text/xml" -d @request_v20.xml http://localhost:2000/ContactService + ``` + Observe that both requests, `1.1` and `2.0`, get a response from their respective service, although the endpoint used is the same. -5. If you now send a `1.1` response to a `2.0` endpoint using command below, you get a response containing "Cannot find dispatch method". -``` - curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:8080/ContactService/v20 +## How it is done + +Let's examine the `proxies.xml` file. + +```xml + + + + + + + + ``` +We define an `` component with the endpoint context path being `ContactService`. +Within we simply set up a `` component with two cases, each utilizing xPath to identify the namespace URL, +and proxying the requests to their respective endpoint in our demo SOAP service. -6. We now start Membrane: Execute `examples/versioning/routing/service-proxy.bat` . +The version `1.1` of our "ContactService" uses the namespace +"http://predic8.com/contactService/v11", while version `2.0` uses +"http://predic8.com/contactService/v20". -7. Return to the console. Run both: -``` - curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:2000/ContactService - curl --header "Content-Type: text/xml" -d @request_v20.xml http://localhost:2000/ContactService -``` - Observe that both requests, `1.1` and `2.0`, get a response from their respective service, although - the endpoint used is the same. +(Note that the WSDL files reference the actual endpoint (depending on the +service version), and using them in any client (e.g. SoapUI) therefore +bypasses the proxy.) --- See: diff --git a/distribution/examples/versioning/routing/proxies.xml b/distribution/examples/versioning/routing/proxies.xml index fec5c3c22..ff76e27c4 100644 --- a/distribution/examples/versioning/routing/proxies.xml +++ b/distribution/examples/versioning/routing/proxies.xml @@ -6,12 +6,12 @@ - + - + diff --git a/distribution/examples/versioning/xslt/README.md b/distribution/examples/versioning/xslt/README.md index 7aefa1728..40cc253a4 100644 --- a/distribution/examples/versioning/xslt/README.md +++ b/distribution/examples/versioning/xslt/README.md @@ -1,68 +1,80 @@ -### VERSIONING WEBSERVICES BY PERFORMING XSLT +# Versioning - SOAP Message Version Transformation -In this example we investigate how web services can be versioned by transforming -old requests using XSLT. +This sample demonstrates how to rewrite SOAP message versions using `XSLT`. -Membrane acts as a common proxy and routes SOAP requests to the new service -endpoint, depending on the XML namespace used in the SOAP message body. Old -requests are transformed into new ones using an XSLT stylesheet. -The version `1.1` of our "ContactService" used the namespace -"http://predic8.com/contactService/v11", while version `2.0` uses -"http://predic8.com/contactService/v20". +## Running the Example +1. In a terminal, set `/examples/versioning/xslt` as your working directory. -(Note that the WSDL file references the actual endpoint (depending on the -service version), and using them in any client (e.g. SoapUI) therefore -bypasses the proxy.) +2. Compile and run the demo SOAP service: + ```sh + mvn clean compile assembly:single + java -jar ./target/xslt-maven-1.0-SNAPSHOT.jar + ``` -#### RUNNING THE EXAMPLE -To run the example execute the following steps: - -Execute the following steps: - -1. To test the router we will use the command line tool curl that can transfer - data with URL syntax. You can download it form the following location: - - http://curl.haxx.se/download.html - -2. Open a new console window and execute below: -``` - cd [MEMBRANE_HOME]\examples\versioning\xslt-maven - mvn clean compile assembly:single - java -jar ./target/xslt-maven-1.0-SNAPSHOT.jar -``` -3. Open yet another console window and run -``` - cd [MEMBRANE_HOME]\examples\versioning\xslt-maven - curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:9000/ContactService/v11 -``` - You should see a response containing `

404 Not Found

`, `No context found for request`. - -4. Now run below command. You should see a response containing `Hello ... version 2.0`. -``` - curl --header "Content-Type: text/xml" -d @request_v20.xml http://localhost:9000/ContactService/v20 -``` +3. Now test if the endpoint is reachable, in a separate terminal instance run: + ```sh + # Should Respond with "Hello ... version 2.0" + curl --header "Content-Type: text/xml" -d @request_v20.xml http://localhost:8080/ContactService/v20 + ``` - -5. If you now send a `1.1` response to a `2.0` endpoint using below, you get - -``` - curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:9000/ContactService/v20 -``` - -6. We now start Membrane: Execute `examples/versioning/xslt-maven/service-proxy.bat` . - -7. Return to the console. Run both: -``` - curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:2000/ContactService - curl --header "Content-Type: text/xml" -d @request_v20.xml http://localhost:2000/ContactService + Now run the same command but sending a message using `v1.1` instead of `v2.0`: + ```sh + curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:8080/ContactService/v20 + ``` + You should see a response containing `Cannot find dispatch method`. + + Lastly try accessing a `v1.1` endpoint with a similar command: + ```sh + curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:8080/ContactService/v11 + ``` + You should see a response containing `

404 Not Found

`, `No context found for request`. + No endpoint for this version is available. + + +4. Start Membrane by running `service-proxy.sh` or the `.bat` equivalent. + + +5. Return to another terminal, send two requests, one for versions `1.1` and `2.0` respectively, + to the single Membrane endpoint: + ```sh + curl --header "Content-Type: text/xml" -d @request_v11.xml http://localhost:2000/ContactService + curl --header "Content-Type: text/xml" -d @request_v20.xml http://localhost:2000/ContactService + ``` + Observe that both requests, `1.1` and `2.0`, get a valid response from the service, although + the first one uses an old request format. + +## How it is done + +Let's examine the `proxies.xml` file. + +```xml + + + /ContactService + + + + + + + /ContactService + + + + + ``` - Observe that both requests, `1.1` and `2.0`, get a valid response from the service, although - the first one uses an old request format. +We define two `` components and specify our single `v2.0` service/WSDL address. +When sending a SOAP message to the endpoint context path `ContactService`, +a `` component uses xPath to determine if the message is using an old version of the format. +If this is the case we proxy the request to the second ``, where a `` plugin is applied to every request. +Old requests are transformed into new ones using an XSLT stylesheet, specified within the `xslt` attribute of the `transform`. --- See: From 65751f9c12ca96c85aac5fb47a92ede4b82a46d6 Mon Sep 17 00:00:00 2001 From: t-burch <119930761+t-burch@users.noreply.github.com> Date: Wed, 19 Jul 2023 15:32:43 +0200 Subject: [PATCH 6/6] Added missing reference link in "see" section for formValidation plugin (#674) --- distribution/examples/validation/form/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distribution/examples/validation/form/README.md b/distribution/examples/validation/form/README.md index c0ebc851f..45dae82fa 100644 --- a/distribution/examples/validation/form/README.md +++ b/distribution/examples/validation/form/README.md @@ -40,4 +40,4 @@ By adding child elements to the plugin, we establish the necessary val --- See: -- []() reference \ No newline at end of file +- [formValidation](https://www.membrane-soa.org/api-gateway-doc/current/configuration/reference/formValidation.htm) reference \ No newline at end of file