- F5 BIG-IP Runtime Init
F5 BIG-IP Runtime Init is a tool that aims to simplify startup scripts for BIG-IP Virtual Edition. It does this by providing a single convenient YAML (1.2 spec) or JSON-based configuration file, which
- leverages F5 Automation Tool Chain declarations that are easier to author, validate, and maintain as code (vs. bigip.conf files);
- renders secrets from public cloud vaults; and
- renders runtime variables from metadata services.
The result is a complete overlay deployment tool for configuring a BIG-IP instance. This allows us to extend our cloud solutions from native templates to other instance provisioning tools, such as Terraform and Ansible. For more information regarding sending startup scripts to BIG-IP VE, see VE documentation.
From a high-level, using this tool involves three steps:
-
Step 1: Download and Install BIG-IP Runtime Init using the self-extracting installer:
curl -o /tmp/f5-bigip-runtime-init-2.0.3-1.gz.run https://cdn.f5.com/product/cloudsolutions/f5-bigip-runtime-init/v2.0.3/dist/f5-bigip-runtime-init-2.0.3-1.gz.run && bash /tmp/f5-bigip-runtime-init-2.0.3-1.gz.run -- '--cloud [aws|azure|gcp]'
-
Step 2: Download OR Render inline a Runtime Init configuration file (runtime-init-conf.yaml).
curl -o /config/cloud/runtime-init-conf.yaml https://my-source-host/my-repo/bigip-configs/0.0.1/runtime-init-conf.yaml
- See configuration details below.
-
Step 3: Load the configuration file:
f5-bigip-runtime-init --config-file /config/cloud/runtime-init-conf.yaml
- See usage examples below.
This repository includes both the BIG-IP Runtime Init source code and a self-extracting installer script for installing the main package.
The installer script will do the following:
- Determine the cloud environment where the script is running
- Extract and verify the appropriate cloud-specific package archive
- Install the package archive and create a command alias for f5-bigip-runtime-init
Based on the content of the provided YAML or JSON configuration file, BIG-IP Runtime Init will do the following:
- Download, verify, and install F5 Automation Toolchain packages (DO, AS3, FAST, TS, and CFE) from default package metadata, URLs, or local files.
- Download, verify, and install custom iApp LX packages from URLs or local files.
- Accept Automation Toolchain declarations from URLs or local files (must be valid JSON or YAML declarations).
- Get secrets from cloud provider secret management APIs (Azure KeyVault, AWS Secret Manager, GCP Secrets Manager, HashiCorp Vault).
- Get select attributes from cloud provider instance and network metadata.
- Render valid Automation Toolchain declarations based on rendered runtime variables (such as secrets and metadata attributes above) and provided declarations.
- POST rendered declarations to Automation Toolchain endpoints and verify success or failure.
- Run user-specified pre-onboard and post-onboard commands.
- Send a webhook with a customizable telemetry data to a user-specified endpoint.
- BIG-IP 14.1.4.6 or newer.
- A mechanism to copy the configuration file to the BIG-IP instance (cloud-init, user data, provider-specific methods).
- Access to the Internet (or other network location if files are locally hosted) for downloading the self-extracting installer package, RPM files, and SHA256 checksums for package verification.
- Access to the cloud provider metadata service if you rendering metadata runtime parameters.
- An IAM identity associated to the BIG-IP instance(s) with sufficient roles/permissions for accessing cloud provider APIs.
- If leveraging the
extension_servicesparameter to send DO declarations, the declarations cannot contain directives that will trigger a reboot. For example, a reboot would occur for any declaration that:- contains a disk_class
- provisions a module (for example, APM) that creates a disk volume
F5 BIG-IP Runtime Init has been tested and validated with the following versions of BIG-IP:
| BIG-IP Version | Build Number |
|---|---|
| 16.1.3.3 | 0.0.3 |
| 15.1.8.1 | 0.0.3 |
| 14.1.5.3 | 0.0.5 |
Newer versions are expected to work but have not been specifically tested.
The self-extracting installer accepts the following parameters:
--cloud | -c : Specifies cloud provider name. Allowed values: (aws, azure or gcp)
--key | -k : Provides location for GPG key used for verifying signature on RPM file
--skip-verify : Disables RPM signature verification
--toolchain-metadata-file-url : Provides overriding delivery URL for toolchain extension metadata file
--skip-toolchain-metadata-sync : Disables downloading automation toolchain metadata from the Internet
--telemetry-params : Specifies telemetry parameters as key:value pairs; (key01:value01,key02:value02). For sending F5 additional usage data.
NOTE: Runtime Init can be installed generically on a cloud or environment not listed above by omitting the --cloud | -c parameter. For example, it can also be used in a VMware environment to install the F5 Automation Tool Chain packages and declarations. When omitted, Runtime Init's cloud specific integrations (runtime_parameters:Â type: secret and type: metadata ) will be disabled. See runtime_parameters section for details.
The installer also allows you to configure request retries to make the installation robust and tolerant to network instability. This can be done using the following environment variables:
| Environment variable | Description | Default Value |
|---|---|---|
| HTTP_RETRY | Number of retries before script will fail. | 12 |
| HTTP_RETRY_MAX_TIME | The retry timer (in seconds) is reset before the first transfer attempt. | 60 |
| HTTP_MAX_TIME | Maximum time (in seconds) that you allow the whole operation to take. | 5 |
Examples:
Using --cloud parameter for basic Azure install:
curl https://cdn.f5.com/product/cloudsolutions/f5-bigip-runtime-init/v2.0.3/dist/f5-bigip-runtime-init-2.0.3-1.gz.run -o f5-bigip-runtime-init-2.0.3-1.gz.run && bash f5-bigip-runtime-init-2.0.3-1.gz.run -- '--cloud azure'
See Private Environments section below for more install examples.
Self-extracting installer, RPMs, and file hashes are available from the following locations:
The BIG-IP Runtime Init configuration consists of the following attributes:
| Attribute | Default Value | Required | Description |
|---|---|---|---|
| controls | none | No | List of runtime controls settings. |
| pre_onboard_enabled | none | No | List of commands to run that do not check if BIG-IP and MCPD are up and running. However, execution before BIG-IP is ready depends on cloud agent/download times/etc. |
| runtime_parameters | none | No | List of runtime parameters to gather. |
| bigip_ready_enabled | none | No | List of commands to run after BIG-IP and MCPD are up and running. Example: TMSH commands, misc optimizations, etc. |
| extension_packages | none | No | List of iControl Lx packages to download and install. |
| extension_services | none | No | List of iControl Lx declarations to configure. |
| post_onboard_enabled | none | No | List of commands to run after sending iControl LX declarations. |
| post_hook | none | No | Webhook to send upon completion. |
A basic Runtime Init configuration file (YAML).
controls:
logLevel: silly
logFilename: /var/log/cloud/bigIpRuntimeInit.log
extension_packages:
install_operations:
- extensionType: do
extensionVersion: 1.44.0
- extensionType: as3
extensionVersion: 3.51.0
- extensionType: ts
extensionVersion: 1.35.0
- extensionType: fast
extensionVersion: 1.25.0See SCHEMA.md for complete schema documentation and /examples/runtime_configs for additional examples.
Runtime Init provides a list of controls intended for tuning Runtime Init execution as well as helping with troubleshooting issues:
NOTE: These can also be set at runtime using an environment variable. For example:
export F5_BIGIP_RUNTIME_INIT_LOG_LEVEL=silly && f5-bigip-runtime-init --config-file /config/cloud/runtime-init-conf.yaml-
logLevel
- Description: Defines log level. Allowed values are
error,warn,info,debugandsilly. NOTE: F5 recommends putting Runtime Init insillyfirst. Although this results in large amounts of logs, it is generally more useful for troubleshooting initial deployments / new configurations. - Default:
info. - Environment Variable: F5_BIGIP_RUNTIME_INIT_LOG_LEVEL (string)
- Description: Defines log level. Allowed values are
-
logFilename
- Description: Defines the location of Runtime Init's log file.
- Default:
/var/log/cloud/bigIpRuntimeInit.log - Environment Variable: F5_BIGIP_RUNTIME_INIT_LOG_FILENAME (string)
-
logToJson
- Description: Defines when log is outputted into JSON format. For example,
{"message":"this is a json message","level":"info","timestamp":"2020-08-04T00:22:28.069Z"}- Default:
false - Environment Variable: F5_BIGIP_RUNTIME_INIT_LOG_TO_JSON (boolean)
-
extensionInstallDelayInMs
- Description: Defines a delay between extensions installations. NOTE: If not provided and the extension package is already installed, the default delay of 10 seconds is skipped.
- Default:
10000 - Environment Variable: F5_BIGIP_RUNTIME_EXTENSION_INSTALL_DELAY_IN_MS (number)
Examples:
controls:
logLevel: silly
logFilename: /var/log/cloud/bigIpRuntimeInit.log
logToJson: true
extensionInstallDelayInMs: 60000Description: A list of commands that run without checking if BIG-IP and MCPD are up and running. Whether these commands are executed before or after BIG-IP is ready depends on external factors like cloud agent status, network latency, etc. For instance, when the Runtime-Init installer and/or other required files are baked directly into the BIG-IP image using the BIG-IP Image Generator, pre_onboard_enabled commands can run sufficiently early. However, in some clouds or scenarios, the files may take too long to download. In that case, these commands may need to be run earlier in the startup script itself to ensure they are applied before BIG-IP is ready.
Allowed types are inline, file and url.
Examples:
-
inline
pre_onboard_enabled: - name: example_inline_command type: inline commands: - touch /tmp/pre_onboard_script.sh - chmod 777 /tmp/pre_onboard_script.sh - echo "touch /tmp/create_by_autogenerated_pre_local" > /tmp/pre_onboard_script.sh - /usr/bin/setdb provision.extramb 1000 || exit 0 - /usr/bin/setdb provision.restjavad.extramb 1384 || /usr/bin/setdb restjavad.useextramb true || exit 0 - /usr/bin/setdb iapplxrpm.timeout 300 || exit 0 - /usr/bin/setdb icrd.timeout 180 || exit 0 - /usr/bin/setdb restjavad.timeout 180 || exit 0 - /usr/bin/setdb restnoded.timeout 180 || exit 0
-
file
pre_onboard_enabled: - name: example_local_exec type: file commands: - /tmp/pre_onboard_script.sh
-
url
pre_onboard_enabled: - name: example_remote_exec type: url commands: - https://the-delivery-location.com/remote_pre_onboard.sh
NOTE: Each command is executed independently from each other. For example, a bash variable in one command cannot be referenced by the following command. See the other command based attributes (ex. bigip_ready_enabled, post_onboard_enabled for more advanced command examples.
Description: A list of parameters discovered at run (or deploy) time which are substituted (using mustache handlebars) in subsequent eligible runtime attributes:
- bigip_ready_enabled (commands)
- extension_packages
- extension_services (declarations sent to the Tool Chain endpoints)
- post_onboard_enabled: (commands)
- post_hook
Parameters can be dependent on each other, so one parameter value can be used within another parameter (see examples below for more details).
Allowed types are storage, secret, tag, metadata, url and static.
-
storage
Description: fetches a file from cloud or other storage and renders the result as a parameter value. Storage parameter files are downloaded before other runtime parameters are processed.
The example below demonstrates how to fetch the values for the downloads from Amazon S3, Azure Blob Storage, Google Cloud Storage, or another location.
NOTE: Storage provider destinations may only start with either /var/config/rest/downloads or /var/tmp. Large files such as RPMs must be saved in /var/config/rest/downloads. When a file saved in /var/config/rest/downloads is referenced by its parameter name, the parameter value will be rendered as a local file path on the BIG-IP instance. Files saved to /var/tmp will be rendered as a string or JSON, depending on the format of the response.
Examples:
runtime_parameters: - name: AWS_TO_FILE type: storage storageProvider: environment: aws source: https://mybucket.s3.amazonaws.com/mykey/f5-appsvcs-3.42.0-5.noarch.rpm destination: "/var/config/rest/downloads/f5-appsvcs-3.42.0-5.noarch.rpm" - name: AWS_TO_FILE_2 type: storage storageProvider: environment: aws source: s3://mybucket/mykey/asm-policy-v0.0.1.xml destination: /var/config/rest/downloads/asm-policy-v0.0.1.xml - name: AWS_TO_PARAMETER type: storage storageProvider: environment: aws source: s3://mybucket/mykey/myfile destination: "/var/tmp/myfile" - name: AZURE_TO_FILE type: storage storageProvider: environment: azure source: https://mystorageaccount.blob.core.windows.net/mycontainer/f5-appsvcs-3.42.0-5.noarch.rpm destination: "/var/config/rest/downloads/f5-appsvcs-3.42.0-5.noarch.rpm" - name: AZURE_TO_PARAMETER type: storage storageProvider: environment: azure source: https://mystorageaccount.blob.core.windows.net/mycontainer/myfile destination: "/var/tmp/myfile" - name: GCP_TO_FILE type: storage storageProvider: environment: gcp source: https://storage.cloud.google.com/mybucket/mykey/f5-appsvcs-3.42.0-5.noarch.rpm destination: "/var/config/rest/downloads/f5-appsvcs-3.42.0-5.noarch.rpm" - name: GCP_TO_PARAMETER type: storage storageProvider: environment: gcp source: gs://mybucket/mykey/myfile destination: "/var/tmp/myfile" - name: PRIVATE_TO_FILE type: storage storageProvider: environment: private source: https://github.com/F5Networks/f5-appsvcs-extension/releases/download/v.3.42.0/f5-appsvcs-3.42.0-5.noarch.rpm destination: "/var/config/rest/downloads/f5-appsvcs-3.42.0-5.noarch.rpm" - name: PRIVATE_TO_PARAMETER type: storage storageProvider: environment: private source: https://myserver/myfile destination: "/var/tmp/myfile"
Referencing storage provider parameter values - S3
In this example, the AWS_TO_FILE parameter is renderered as "/var/config/rest/downloads/f5-appsvcs-3.42.0-5.noarch.rpm"; to reference this as a local file URL you must prepend "file://" to the parameter name:
extension_packages: install_operations: - extensionType: as3 extensionVersion: 3.42.0 extensionUrl: 'file://{{{AWS_TO_FILE}}}'
In this example, the AWS_TO_FILE_2 parameter is renderered as "/var/config/rest/downloads/asm-policy-v0.0.1.xml":
extension_services: service_operations: - extensionType: as3 type: inline value: class: ADC schemaVersion: 3.0.0 label: Quickstart remark: Quickstart Tenant_1: class: Tenant Shared: class: Application template: shared shared_pool: class: Pool remark: Service 1 shared pool members: - serverAddresses: - 10.0.3.4 servicePort: 80 monitors: - http Custom_HTTP_Profile: class: HTTP_Profile xForwardedFor: true Custom_WAF_Policy: class: WAF_Policy file: >- '{{{AWS_TO_FILE_2}}}' enforcementMode: blocking ignoreChanges: false
In this example, the AWS_TO_PARAMETER parameter is renderered as either a string or JSON object, depending on the original file format:
post_onboard_enabled: - name: echo_downloaded_file type: inline commands: - echo "Downloaded file parameter is {{{AWS_TO_PARAMETER}}}" # renders content of s3://mybucket/mykey/myfile - echo "Downloaded JSON parameter is {{{AWS_TO_PARAMETER.key}}}" # renders value of "key" key in s3://mybucket/mykey/myfile
NOTE:
-
In AWS and GCP, both https:// and global s3:// and gs:// source URLs are supported.
-
In Azure, only blob storage URLs are supported.
-
The IAM roles/RBAC permissions required for downloading objects from cloud provider storage are listed below. Minimally, these permissions must be scoped to the storage resource.
- AWS:
"s3:ListBucket" "s3:GetObject" - Azure:
"Microsoft.Authorization/*/read" "Microsoft.Storage/storageAccounts/read" "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read" "Microsoft.Storage/storageAccounts/listKeys/action" - GCP (Labels):
"storage.buckets.get" "storage.buckets.list" "storage.objects.get" "storage.objects.list"
- AWS:
-
-
secret
Description: Fetches a secret from a provider vault. This type requires the BIG-IP instance to have IAM READ permissions for the secrets. See the respective Cloud Provider's official documentation for additional information. For basic examples, see the Terraform examples.
Allowed environments are
aws,azure,gcpandhashicorp.Examples:
AWS:
Provide secret name (assumes secret is in same region as BIG-IP instance):
runtime_parameters: - name: ADMIN_PASS type: secret secretProvider: type: SecretsManager environment: aws version: AWSCURRENT secretId: mysecret
Provide secret ARN, including 12 digit account ID and region:
runtime_parameters: - name: ADMIN_PASS type: secret secretProvider: type: SecretsManager environment: aws version: AWSCURRENT secretId: arn:aws:secretsmanager:us-east-1:123456789012:secret:mySecretId-a0BCDE
IAM Permissions:
"secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret", "secretsmanager:ListSecretVersionIds"For more information, see AWS documentation.
Azure:
runtime_parameters: - name: AZURE_SERVICE_PRINCIPAL type: secret secretProvider: type: KeyVault environment: azure vaultUrl: https://my-keyvault.vault.azure.net secretId: mysecret
IAM Permissions:
key_permissions = ["get"] secret_permissions = ["get","list"] storage_permissions = ["get"]For more information, see Azure KeyVault access control policy documentation.
KeyVault Firewall: If Azure KeyVault Firewall is enabled and the default firewall action is Deny, you must explicitly add either the Azure public management IP address of the BIG-IP instance, or the Azure virtual network and management subnet, to the KeyVault Firewall allow list.
For more information, see Azure KeyVault network security documentation.
GCP:
Provide secret name (assumes secret is in same project as BIG-IP instance):
runtime_parameters: - name: ADMIN_PASS type: secret secretProvider: type: SecretsManager environment: gcp version: latest secretId: mysecret
Provide fully-qualified secret identifier, including project number and version:
runtime_parameters: - name: ADMIN_PASS type: secret secretProvider: type: SecretsManager environment: gcp secretId: projects/123456789/secrets/mysecret/versions/latest
IAM Permissions:
"secretmanager.versions.access" "secretmanager.versions.list" "secretmanager.versions.get" "compute.instances.get"For more information, see GCP documentation.
HashiCorp (using App Role authentication):
The following example uses the special value data in the field attribute to retrieve the entire secret response, which can then be referenced inside mustache handlebars inside the configuration. When referencing multiple secret values from a single response, this limits client requests to the Vault server to a minimum (you may also create a unique runtime parameter for each secret stored in Vault, using the provided examples).
runtime_parameters: - name: ADMIN_PASS type: secret verifyTls: true trustedCertBundles: ['/config/ssl/ssl.crt/my-ca-bundle.crt'] secretProvider: type: Vault environment: hashicorp vaultServer: https://127.0.0.1:8200 appRolePath: /v1/auth/approle/login secretsEngine: kv2 secretId: secret/credential field: data version: 1 authBackend: type: approle roleId: type: url value: file:///path/to/role-id secretId: type: inline value: secret-id unwrap: true ... extension_services: service_operations: - extensionType: do type: inline value: schemaVersion: 1.0.0 class: Device async: true label: my BIG-IP declaration for declarative onboarding Common: class: Tenant hostname: '{{{ HOST_NAME }}}.local' foo: class: User userType: regular password: '{{{ ADMIN_PASS.foo_password }}}' shell: bash partitionAccess: all-partitions: role: admin bar: class: User userType: regular password: '{{{ ADMIN_PASS.bar_password }}}' shell: bash partitionAccess: all-partitions: role: admin
NOTE:
-
When the authBackend.secretId.unwrap attribute is set to
true(recommended), the secretId value must be in the form of a wrapping token. BIG-IP Runtime Init will unwrap this token to retrieve the actual secret ID. This eliminates the need to pass the secret ID in the declaration. For more information, See the HashiCorp AppRole documentation -
This example also demonstrates how to use custom PKI certs for https requests to HashiCorp Vault server when verifyTls set to
true.
For more information, see HashiCorp documentation.
By default, runtime will mask out (ex. "password":"******") the following common fields in declarations when logging:
[ "password", "localPassword", "remotePassword", "bigIqPassword", "bigIpPassword", "passphrase", "cookiePassphrase", "certificate", "privateKey", "ciphertext", "protected", "secret", "sharedSecret", "secretAccessKey", "apiAccessKey", "encodedCredentials", "encodedToken", "oldPassword", "newPassword", "bindPassword", "checkBindPassword", "md5SignaturePassphrase" ]However, it is possible to extend this list by providing an additional (field) attribute for the Secret object:
runtime_parameters: - name: MY_SECRET type: secret secretProvider: environment: azure type: KeyVault vaultUrl: https://my-keyvault.vault.azure.net secretId: mySecret01 field: newCustomSecretField
This example shows instructing Runtime Init to also mask out the value for
newCustomSecretFieldseen in any declarations. -
-
tag
Description: fetches a tag value from public cloud virtual machine resource.
The example below demonstrates how to fetch the value for the tag, with the key "CustomTag", applied to the EC2 instance:
runtime_parameters: - name: TAG_VALUE type: tag tagProvider: environment: aws key: CustomTag
NOTE:
-
In Azure, Runtime-Init will gather tags from the metadata service and doesn't require any additional permissions.
-
In AWS and GCP, Runtime-Init will gather them from the Cloud API which requires additional IAM permissions.
-
AWS:
"ec2:DescribeTags" -
GCP (Labels):
"compute.instances.get"- GCP also has several different types of tags (Labels, Network Tags, and Metadata Tags - ad hoc key/value pairs embedded in Metadata). As the Metadata Tags can be fetched using the generic
urltype if needed, this type leverages IAM roles to enable fetching "Labels".
- GCP also has several different types of tags (Labels, Network Tags, and Metadata Tags - ad hoc key/value pairs embedded in Metadata). As the Metadata Tags can be fetched using the generic
-
-
-
metadata
Description: Convenience function to grab common onboarding items from the Metadata Service.
Allowed types are
networkanduri.For example, type
networkreturns addresses reformatted in CIDR notation so you can use for Self-IPs.For more examples, see the examples/runtime_configs directory
Examples:
AWS Self-IP
runtime_parameters: - name: SELF_IP_EXTERNAL type: metadata metadataProvider: type: network environment: aws field: local-ipv4s index: 1
Azure Self-IP
runtime_parameters: - name: SELF_IP_EXTERNAL type: metadata metadataProvider: type: network environment: azure field: ipv4 index: 1 - name: SELF_IP_EXTERNAL_IPV6 type: metadata metadataProvider: type: network environment: azure field: ipv6 index: 1 service_operations: - extensionType: do value: Common: class: Tenant external-self: class: SelfIp address: '{{{SELF_IP_EXTERNAL}}}' vlan: external external-self-ipv6: class: SelfIp address: '{{{SELF_IP_EXTERNAL_IPV6}}}/64' vlan: external
GCP Self-IP
runtime_parameters: - name: SELF_IP_EXTERNAL type: metadata metadataProvider: environment: gcp type: network field: ip index: 0
IPv4: Returns the CIDR address (ex.
10.0.0.5/24) which is required by the Self-IP. IPv6: Returns the address (ex. ab:ff:ff::dfd). Must provide the prefix.The output can be further transformed using ipcalc functionality (IPv4 only):
The ipcalc functionality provides the following transformation options:
- address - The provided address without netmask prefix.
- base - The base address of the network block as a string (eg: 216.240.32.0). Base does not give an indication of the size of the network block.
- mask - The netmask as a string (eg: 255.255.255.0).
- hostmask - The host mask which is the opposite of the netmask (eg: 0.0.0.255).
- bitmask - The netmask as a number of bits in the network portion of the address for this block (eg: 24).
- size - The number of IP addresses in a block (eg: 256).
- broadcast - The blocks broadcast address (eg: 192.168.1.0/24 => 192.168.1.255).
- first - First useable address.
- last - Last useable address.
The following example uses ipcalc to get the first useable IPv4 address using the CIDR of the first AWS subnet and resolves it to a runtime parameter named
GATEWAY.runtime_parameters: - name: GATEWAY type: metadata metadataProvider: environment: aws type: network field: local-ipv4s index: 0 ipcalc: first
This example returns
10.0.0.1for the Self-IP10.0.0.5/24.The next example uses ipcalc to get the bitmask using the CIDR of the first AWS subnet and resolves it to a runtime parameter named as BITMASK.
runtime_parameters: - name: BITMASK type: metadata metadataProvider: environment: aws type: network field: local-ipv4s index: 0 ipcalc: bitmask
This example returns
24for the Self-IP10.0.0.5/24.For fetching AWS Metadata, Runtime Init allows you use a custom URI. By default, Runtime Init uses AWS IMDSv2 to get AWS metadata:
runtime_parameters: - name: ACCOUNT_ID type: metadata metadataProvider: environment: aws type: uri value: /latest/dynamic/instance-identity/document query: accountId
In the case where JSON response is returned, this type allows you to provide a JMESPath query filter. In example above, the response queries for the
accountIdkey. -
url
Description: Defines a generic URL to fetch a runtime parameter.
This type can be useful when gathering custom metadata not covered in the
metadataProviderabove, data from custom locations, session based URLs, etc.This type allows you to provide HTTP headers as well as a JMESPath query filter for JSON responses. The headers and query fields are optional.
The following examples demonstrates how to fetch hostnames or virtual machine names for AWS, GCP and Azure:AWS Hostname
runtime_parameters: - name: HOST_NAME type: url value: http://169.254.169.254/latest/meta-data/hostname
GCP Hostname
runtime_parameters: - name: HOST_NAME type: url value: http://169.254.169.254/computeMetadata/v1/instance/hostname headers: - name: Metadata-Flavor value: Google
Azure Virtual Machine Name
runtime_parameters: - name: NAME type: url value: 'http://169.254.169.254/metadata/instance/compute?api-version=2020-09-01' query: name headers: - name: Metadata value: true
The following AWS URL example has been superseded by the metadata
uritype above but demonstrates linking Runtime Parameters together (to provide a session-based request). It first fetches an AWS Session token and leverages the JMESPathqueryfield to filter for theregionkey.AWS:
runtime_parameters: - name: AWS_SESSION_TOKEN type: url value: http://169.254.169.254/latest/api/token headers: - name: Content-Type value: json - name: User-Agent value: some-user-agent - name: method value: PUT - name: X-aws-ec2-metadata-token-ttl-seconds value: 21600 - name: REGION type: url value: http://169.254.169.254/latest/dynamic/instance-identity/document query: region headers: - name: Content-Type value: json - name: User-Agent value: some-user-agent - name: X-aws-ec2-metadata-token value: "{{{AWS_SESSION_TOKEN}}}"
The
urltype also allows you to provide a local file location using the "file://" schema. The example below demonstrates how to get a parameter value from the /config/cloud/paramter-file.txt file:runtime_parameters: - name: SOME_PARAM type: url value: file:///config/cloud/paramter-file.txt
-
static
Description: defines a static value.
Examples:
This example replaces AVAILABILITY_ZONE token with "us-west-2a" string.
runtime_parameters: - name: AVAILABILITY_ZONE type: static value: us-west-2a
Description: List of commands to run after BIG-IP and MCPD are up and running. Example: TMSH commands, misc optimizations, etc.
Allowed types are inline, file and url.
Examples:
-
inline
bigip_ready_enabled: # Dependent on GUI being up - name: icontrol_settings type: inline commands: - '/usr/bin/curl -s -f -u admin: -H "Content-Type: application/json" -d ''{"maxMessageBodySize":134217728}'' -X POST http://localhost:8100/mgmt/shared/server/messaging/settings/8100 | jq .' # Dependent on MCPD being up - name: using_runtime_variables type: inline commands: - f5mku -r {{{ ADMIN_PASS }}} # Dependent on TMSH / MCPD being up - name: using_bash_variables_in_commands type: inline commands: - "EXT_GW=$(curl -sH 'Metadata-Flavor: Google' http://169.254.169.254/computeMetadata/v1/instance/network-interfaces/0/gateway); tmsh create net route ext_gw_int network $EXT_GW/32 interface external" - "INT_GW=$(curl -sH 'Metadata-Flavor: Google' http://169.254.169.254/computeMetadata/v1/instance/network-interfaces/2/gateway); tmsh create net route int_gw_int network $INT_GW/32 interface internal"
NOTE: Each command is executed in a separate shell context from each other. For example, a bash variable in one command cannot be referenced by the following command. For instance, in the example above, the command with the $INT_GW variable would not be able to render the $EXT_GW variable from command above it. For more examples, see the examples/runtime_configs directory.
Description: List of iControl Lx packages to download and install.
Allowed extensionTypes are do, as3, ts, fast and cfe.
Examples:
-
minimal
extension_packages: install_operations: - extensionType: do extensionVersion: 1.44.0 - extensionType: as3 extensionVersion: 3.51.0 - extensionType: ts extensionVersion: 1.35.0 - extensionType: fast extensionVersion: 1.25.0
-
with hash checking
extension_packages: install_operations: - extensionType: do extensionVersion: 1.44.0 extensionHash: 3b05d9bcafbcf0b5b625ff81d6bab5ad26ed90c0dd202ded51756af3598a97ec - extensionType: as3 extensionVersion: 3.51.0 extensionHash: e151a9ccd0fd60c359f31839dc3a70bfcf2b46b9fedb8e1c37e67255ee482c0f - extensionType: ts extensionVersion: 1.35.0 extensionHash: 839698d98a8651a90b3d509cde4b382338461a253878c9fd00c894699ef0e844 - extensionType: fast extensionVersion: 1.25.0 extensionHash: 434309179af405e6b663e255d4d3c0a1fd45cac9b561370e350bb8dd8b39761f
-
custom from URL
extension_packages: install_operations: - extensionType: do extensionUrl: https://github.com/F5Networks/f5-declarative-onboarding/releases/download/v1.44.0/f5-declarative-onboarding-1.44.0-5.noarch.rpm extensionVersion: 1.44.0 - extensionType: as3 extensionUrl: file:///var/config/rest/downloads/f5-appsvcs-3.51.0-5.noarch.rpm extensionVersion: 3.51.0 - extensionType: fast extensionUrl: https://github.com/F5Networks/f5-appsvcs-templates/releases/download/v1.25.0/f5-appsvcs-templates-1.25.0-1.noarch.rpm extensionVersion: 1.25.0
NOTE:
extensionVersionis not required when used with theextensionUrlfield.
For more examples, see the examples/runtime_configs directory.
Description: List of iControl Lx declarations to configure.
Allowed extensionTypes are do, as3, ts and cfe.
Allowed value types are inline and url.
Examples:
-
inline
extension_services: service_operations: - extensionType: do type: inline value: schemaVersion: 1.0.0 class: Device label: >- Quickstart 1NIC BIG-IP declaration for Declarative Onboarding with BYOL license async: true Common: class: Tenant My_DbVariables: class: DbVariables ui.advisory.enabled: true ui.advisory.color: blue ui.advisory.text: BIG-IP Quickstart My_Provisioning: class: Provision asm: nominal ltm: nominal My_Ntp: class: NTP servers: - 169.254.169.253 timezone: UTC My_Dns: class: DNS nameServers: - 169.254.169.253 My_License: class: License licenseType: regKey regKey: 'AAAAA-BBBBB-CCCCC-DDDDD-EEEEEEE' My_System: class: System autoPhonehome: true hostname: bigip.example.com quickstart: class: User partitionAccess: all-partitions: role: admin password: 'BIGIP_PASSWORD' shell: bash userType: regular - extensionType: as3 type: inline value: class: AS3 action: deploy persist: true declaration: class: ADC schemaVersion: 3.0.0 label: Sample 1 remark: Simple HTTP Service with Round-Robin Load Balancing Sample_01: class: Tenant A1: class: Application template: http serviceMain: class: Service_HTTP virtualAddresses: - 10.0.1.10 pool: web_pool web_pool: class: Pool monitors: - http members: - servicePort: 80 serverAddresses: - 192.0.1.10 - 192.0.1.11
-
url
extension_services: service_operations: - extensionType: do type: url value: https://cdn.f5.com/product/cloudsolutions/declarations/template2-0/autoscale-waf/autoscale_do_payg.json verifyTls: false - extensionType: as3 type: url value: file:///examples/automation_toolchain_declarations/as3.json
NOTE:
- If using the
urltype, the declarations can be in JSON or YAML format.
For more examples, see the examples/runtime_configs directory.
TIP:
- If creating multiple service extension services, you may need to order them in a particular sequence. For example, if creating a cluster with Declarative Onboarding (DO) and additional extension services, run the first Declarative Onboarding declaration without the device service clustering elements, then run the additional services, and finally, the Declarative Onboarding declaration again with the clustering elements.
For examples, see failover solutions in the Native Template Examples.
Description: List of commands to run after sending iControl LX declarations.
Allowed types are inline, file and url.
Examples:
-
inline
post_onboard_enabled: - name: example_inline_command type: inline commands: - touch /tmp/post_onboard_script.sh - chmod 777 /tmp/post_onboard_script.sh - echo "touch /tmp/create_by_autogenerated_post_local" > /tmp/post_onboard_script.sh
NOTE: Each command is executed independently from each other. For example, a bash variable in one command cannot be referenced by the following command. For more examples, see the examples/runtime_configs directory.
Description: Webhook to send upon completion.
Examples:
post_hook:
- name: example_webhook
type: webhook
url: https://webhook.site
properties:
optionalKey1: optional_value1
optionalKey2: optional_value2For more examples, see the examples/runtime_configs directory.
Terraform plans will generally consist of the following:
- a startup_script template (.tpl)
- passing the rendered startup script to the instance's startup script parameter
In this snippet:
resource "azurerm_linux_virtual_machine" "vm" {
name = "vm-${module.utils.env_unique_id}-bigip"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
size = var.instance_size
admin_username = var.admin_username
admin_ssh_key {
username = var.admin_username
public_key = file(var.f5_ssh_publickey)
}
os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
source_image_reference {
publisher = var.publisher
offer = var.offer
sku = var.sku
version = var.bigip_version
}
plan {
publisher = var.publisher
product = var.offer
name = var.sku
}
boot_diagnostics {
storage_account_uri = var.boot_diagnostics ? join(",", azurerm_storage_account.vm_sa.*.primary_blob_endpoint) : ""
}
network_interface_ids = [
azurerm_network_interface.nic_mgmt.id,
azurerm_network_interface.nic_external.id,
azurerm_network_interface.nic_internal.id
]
identity {
type = "UserAssigned"
identity_ids = [azurerm_user_assigned_identity.user_identity.id]
}
custom_data = base64encode(templatefile("${path.module}/startup-script.tpl", {
vault_name: "key-vault-${module.utils.env_unique_id}-bigip",
secret_id: azurerm_key_vault_secret.adminsecret.name,
package_url: var.bigip_runtime_init_package_url,
admin_username: var.admin_username,
}))
tags = merge(var.global_tags, { Name="vm-${module.utils.env_unique_id}-bigip" })
}
The startup script is templatized in startup-script.tpl and sent using the VM's custom_data parameter. On BIG-IP versions 15.1+, Cloud-Init will execute this script directly. However, for earlier versions, azurerm_virtual_machine_extension is used to run it. See BIG-IP Cloud-Init documentation for more information.
The Terraform template for the startup script contains the following contents.
#!/bin/bash -x
# Send output to log file and serial console
mkdir -p /var/log/cloud /config/cloud /var/config/rest/downloads
LOG_FILE=/var/log/cloud/startup-script.log
[[ ! -f $LOG_FILE ]] && touch $LOG_FILE || { echo "Run Only Once. Exiting"; exit; }
npipe=/tmp/$$.tmp
trap "rm -f $npipe" EXIT
mknod $npipe p
tee <$npipe -a $LOG_FILE /dev/ttyS0 &
exec 1>&-
exec 1>$npipe
exec 2>&1
# Download or Render BIG-IP Runtime Init Config
cat << 'EOF' > /config/cloud/runtime-init-conf.yaml
---
controls:
logLevel: silly
logFilename: /var/log/cloud/bigIpRuntimeInit.log
pre_onboard_enabled: []
runtime_parameters:
- name: ADMIN_PASS
type: secret
secretProvider:
environment: azure
type: KeyVault
vaultUrl: 'https://${vault_name}.vault.azure.net'
secretId: ${secret_id}
- name: HOST_NAME
type: metadata
metadataProvider:
environment: azure
type: compute
field: name
- name: SELF_IP_EXTERNAL
type: metadata
metadataProvider:
environment: azure
type: network
field: ipv4
index: 1
- name: SELF_IP_INTERNAL
type: metadata
metadataProvider:
environment: azure
type: network
field: ipv4
index: 2
- name: DEFAULT_GW
type: metadata
metadataProvider:
environment: azure
type: network
field: ipv4
index: 1
ipcalc: first
- name: MGMT_GW
type: metadata
metadataProvider:
environment: azure
type: network
field: ipv4
index: 0
ipcalc: first
bigip_ready_enabled: []
extension_packages:
install_operations:
- extensionType: do
extensionVersion: 1.44.0
extensionHash: 3b05d9bcafbcf0b5b625ff81d6bab5ad26ed90c0dd202ded51756af3598a97ec
- extensionType: as3
extensionVersion: 3.51.0
extensionHash: e151a9ccd0fd60c359f31839dc3a70bfcf2b46b9fedb8e1c37e67255ee482c0f
- extensionType: ts
extensionVersion: 1.35.0
extensionHash: 839698d98a8651a90b3d509cde4b382338461a253878c9fd00c894699ef0e844
- extensionType: fast
extensionVersion: 1.25.0
extensionHash: 434309179af405e6b663e255d4d3c0a1fd45cac9b561370e350bb8dd8b39761f
extension_services:
service_operations:
- extensionType: do
type: inline
value:
schemaVersion: 1.0.0
class: Device
async: true
label: Example 3NIC BIG-IP with Runtime-Init
Common:
class: Tenant
My_DbVariables:
class: DbVariables
provision.extramb: 1000
restjavad.useextramb: true
ui.advisory.enabled: true
ui.advisory.color: blue
ui.advisory.text: BIG-IP VE Runtime Init Example
config.allow.rfc3927: enable
dhclient.mgmt: disable
My_System:
class: System
hostname: '.local'
cliInactivityTimeout: 1200
consoleInactivityTimeout: 1200
autoPhonehome: true
My_Dns:
class: DNS
nameServers:
- 168.63.129.16
My_Ntp:
class: NTP
servers:
- 0.pool.ntp.org
timezone: UTC
My_Provisioning:
class: Provision
ltm: nominal
admin:
class: User
userType: regular
partitionAccess:
all-partitions:
role: admin
password: '{{{ ADMIN_PASS }}}'
shell: bash
${admin_username}:
class: User
userType: regular
partitionAccess:
all-partitions:
role: admin
password: '{{{ ADMIN_PASS }}}'
shell: bash
external:
class: VLAN
tag: 4094
mtu: 1500
interfaces:
- name: '1.1'
tagged: false
internal:
class: VLAN
tag: 4093
mtu: 1500
interfaces:
- name: '1.2'
tagged: false
default:
class: ManagementRoute
gw: ''
network: default
dhclient_route1:
class: ManagementRoute
gw: ''
network: 168.63.129.16/32
azureMetadata:
class: ManagementRoute
gw: ''
network: 169.254.169.254/32
external-self:
class: SelfIp
address: ''
vlan: external
allowService: default
trafficGroup: traffic-group-local-only
internal-self:
class: SelfIp
address: ''
vlan: internal
allowService: default
trafficGroup: traffic-group-local-only
defaultRoute:
class: Route
gw: ''
network: default
mtu: 1500
post_onboard_enabled: []
EOF
# Download
for i in {1..30}; do
curl -fv --retry 1 --connect-timeout 5 -L "${package_url}" -o "/var/config/rest/downloads/f5-bigip-runtime-init.gz.run" && break || sleep 10
done
# Install
bash /var/config/rest/downloads/f5-bigip-runtime-init.gz.run -- "--cloud azure"
# Run
f5-bigip-runtime-init --config-file /config/cloud/runtime-init-conf.yaml
NOTE:
--cloud azureis passed to the installer to specify the environment.- Terraform templates use the
${variable}syntax for variables. To preserve a bash variable in the rendered script, it uses the escape syntax$${bash_variable}. - When BIG-IP is launched, Terraform renders the template's
${secret_id}variable. For example, the rendered file on BIG-IP will contain:When Runtime Init runs, it will fetch the value for that secret namedruntime_parameters: - name: ADMIN_PASS type: secret secretProvider: environment: azure type: KeyVault vaultUrl: 'https://key-vault-1cqephd9-bigip.vault.azure.net' secretId: secret-1cqephd9-bigIpPassword
secret-1cqephd9-bigIpPasswordand set the runtime variableADMIN_PASS. Runtime Init will then render any tool chain declarations with the mustache variable{{{ ADMIN_PASS }}}with the secret value (i.e. the actual admin password) in its POST payload to the Tool Chain service endpoint. For example:... 2022-03-16T22:19:06.147Z [10306]: info: Creating - do 1.27.0 {"schemaVersion":"1.0.0","class":"Device", ... "admin":{"class":"User","userType":"regular","partitionAccess":{"all-partitions":{"role":"admin"}},"password":"********","shell":"bash"}, ...
For similar AWS and GCP examples, see the examples/terraform directory.
"commandToExecute": "concat('mkdir -p /config/cloud; mkdir -p /var/log/cloud/azure; cp $(ls -v | tail -n1)/runtime-init-conf.yaml /config/cloud/runtime-init-conf.yaml; curl -L https://cdn.f5.com/product/cloudsolutions/f5-bigip-runtime-init/v2.0.3/dist/f5-bigip-runtime-init-2.0.3-1.gz.run -o f5-bigip-runtime-init-2.0.3-1.gz.run && bash f5-bigip-runtime-init-2.0.3-1.gz.run -- ', variables('singleQuote'), '--cloud azure', variables('singleQuote'), ' 2>&1')",
"fileUris": [
"https://example.com/runtime-init-conf.yaml"
]"commandToExecute": "[concat('mkdir -p /config/cloud; mkdir -p /var/log/cloud/azure; echo -e ', variables('singleQuote'), parameters('runtimeConfig'), variables('singleQuote'), ' > /config/cloud/runtime-init-conf.yaml; curl -L https://cdn.f5.com/product/cloudsolutions/f5-bigip-runtime-init/v2.0.3/dist/f5-bigip-runtime-init-2.0.3-1.gz.run -o f5-bigip-runtime-init-2.0.3-1.gz.run; bash f5-bigip-runtime-init-2.0.3-1.gz.run -- ', variables('singleQuote'), '--cloud azure', variables('singleQuote'), ' 2>&1; f5-bigip-runtime-init --config-file /config/cloud/runtime-init-conf.yaml 2>&1')]"For more native template examples, see:
For example runtime-init configurations, go to the /bigip-configurations directory in a solution folder. For example, see failover examples in examples/failover/bigip-configurations.
F5 BIG-IP Runtime Init supports making HTTP/HTTPS requests through a proxy server for most features. It looks at the BIG-IP proxy configuration defined in system db variables. These must be set before installing Runtime Init and can be viewed by running this command: tmsh list sys db proxy.
| Feature | Uses Proxy? | Notes |
|---|---|---|
| pre_onboard_enabled | Yes* | Must manually specify proxy for inline type commands (curl, etc.) |
| runtime_parameters | Yes | Metadata type runtime parameters do not use the proxy. |
| bigip_ready_enabled | Yes* | Must manually specify proxy for inline type commands. |
| extension_packages | Yes | None |
| extension_services | Yes* | Resources using external URLs must be dowloaded locally via runtime parameters and referenced as file URLs. |
| post_onboard_enabled | Yes* | Must manually specify proxy for inline type commands. |
| post_hook | Yes | None |
| telemetry | Yes | None |
Configuring BIG-IP proxy settings via startup script:
# Set REST performance variables immediately before MCPD starts
/usr/bin/setdb provision.extramb 1000 || true
/usr/bin/setdb provision.restjavad.extramb 1384 || /usr/bin/setdb restjavad.useextramb true || true
/usr/bin/setdb iapplxrpm.timeout 300 || true
/usr/bin/setdb icrd.timeout 180 || true
/usr/bin/setdb restjavad.timeout 180 || true
/usr/bin/setdb restnoded.timeout 180 || true
# Set proxy variables
/usr/bin/setdb proxy.host 192.0.2.10
/usr/bin/setdb proxy.port 3128
/usr/bin/setdb proxy.username proxyuser
/usr/bin/setdb proxy.password apassword
/usr/bin/setdb proxy.protocol https
# Download
for i in {1..30}; do
curl -fv --retry 1 --connect-timeout 5 --proxy https://192.0.2.10:3128 --proxy-user proxyuser:apassword -L "${package_url}" -o "/var/config/rest/downloads/f5-bigip-runtime-init.gz.run" && break || sleep 10
done
# Install
bash /var/config/rest/downloads/f5-bigip-runtime-init.gz.run -- "--cloud aws --telemetry-params templateName:f5-bigip-runtime-init/examples/terraform/aws/main.tf"
# Run
f5-bigip-runtime-init --config-file /config/cloud/runtime-init-conf.yamlSome environments may not allow BIG-IPs to have any access to the Internet. In these cases, startup scripts can be customized to download packages and/or config files from locally hosted URLs. By default, the installer makes calls to the Internet to download a GPG key to verify RPM signatures, find the latest Automation Tool Chain packages, and send usage data. To disable calls to the Internet, you can use the examples below:
Example (secure) of hosting the GPG key locally and disabling checking for latest Automation Tool Chain packages.
curl https://myprivatehost/f5-bigip-runtime-init-2.0.3-1.gz.run -o f5-bigip-runtime-init-2.0.3-1.gz.run && bash f5-bigip-runtime-init-2.0.3-1.gz.run -- '--cloud aws --key https://mylocalhost/gpg.key --skip-toolchain-metadata-sync'
Example (thisisinsecure) of skipping downloading the GPG key entirely and checking for latest Automation Tool Chain packages, using a local copy of the metadata instead.
curl https://myprivatehost/f5-bigip-runtime-init-2.0.3-1.gz.run -o f5-bigip-runtime-init-2.0.3-1.gz.run -o f5-bigip-runtime-init-2.0.3-1.gz.run && bash f5-bigip-runtime-init-2.0.3-1.gz.run -- '--cloud aws --skip-verify --skip-toolchain-metadata-sync'
To disable the f5-bigip-runtime-init command from sending usage reporting, you can include the '--skip-telemetry' parameter.
f5-bigip-runtime-init -c /config/cloud/runtime-init-conf.yaml --skip-telemetry
If using the extension_services feature to send Declarative Onboarding declarations, you can alternatively set the autoPhonehome property to disabled. Using the autoPhonehome property will prevent BIG-IP from sending telemetry data globally. For more information on how to disable Automatic Phone Home, see this Overview of the Automatic Update Check and Automatic Phone Home features.
Here is an example of the payload that is sent by F5 TEEM
"telemetryRecords": [
{
"platform": "BIG-IP",
"platformVersion": "14.1.4.6",
"nicConfiguration": "multi",
"cloudAccountId": "<REDACTED>",
"regkey": "<REDACTED>",
"platformDetails": {
"platform": "BIG-IP",
"platformVersion": "14.1.4.6",
"platformId": "Z100",
"system": {
"cpuCount": 4,
"memory": 15753,
"diskSize": 77824
},
"nicCount": 2,
"modules": {
"ltm": "nominal"
},
"packages": {},
"environment": {
"pythonVersion": "Python 2.7.5",
"pythonVersionDetailed": "2.7.5 (default, Aug 12 2021, 23:00:20) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]",
"nodeVersion": "v6.9.1",
"libraries": {
"ssh": "OpenSSH_7.4p1, OpenSSL 1.0.2s-fips 28 May 2019"
}
}
},
"templateInfo": {
"install": "All operations finished successfully",
"templateName": "bigip-standalone.yaml",
"templateVersion": "v1.1.0.0",
"nicCount": 2,
"cloud": "aws",
"region": "us-west-2",
"localization": "en-US"
},
"product": {
"version": "1.3.2",
"locale": "en-US",
"installDate": "2021-11-18T19:29:45.486Z",
"installationId": "f4573f4e-dcd7-4a91-8a0a-3704fca5255f",
"installedComponents": {
"commander": "^4.1.0",
"winston": "^3.3.3",
"get-user-locale": "^1.4.0",
"uuid": "^8.2.0",
"@f5devcentral/f5-teem": "^1.4.6",
"js-yaml": "^3.13.1",
"mustache": "^4.0.0",
"request": "^2.88.0",
"jmespath": "^0.15.0",
"netmask": "^2.0.2",
"lodash.where": "^3.1.0"
}
},
"operation": {
"clientRequestId": "0a1bad90-2feb-4eb8-afd6-c851b3b4ffce",
"rawCommand": "f5-runtime-init -c /config/cloud/onboard_config.yaml",
"pre_onboard_enabled": {
"commands": 1
},
"runtime_params": {
"secrets": 3,
"metadata": 6
},
"vaults": {
"aws": 1,
"azure": 0,
"gcp": 0,
"hashicorp": 2
},
"userAgent": "f5-bigip-runtime-init/1.3.2",
"extension_packages": {
"do": "1.23.0",
"fast": "1.11.0",
"ilx": "0.1.0"
},
"extension_services": {
"do": true,
"as3": true
},
"post_onboard_enabled": {
"commands": 3,
"postHooks": 0
},
"result": "SUCCESS",
"resultSummary": "All operations finished successfully",
"startTime": "2021-11-18T19:29:43.325Z",
"endTime": "2021-11-18T19:29:43.387Z",
"installParams": [
{
"key": "templateName",
"value": "v1.1.0.0/examples/modules/bigip-standalone/bigip-standalone.yaml"
}
]
}
}
]F5 recommends adding the following to the beginning of your startup scripts to ensure that the iControl REST framework has the necessary resources to successfully provision the BIG-IP instance. These settings must be applied as early as possible in the boot process:
/usr/bin/setdb provision.extramb 1000 || true
/usr/bin/setdb provision.restjavad.extramb 1384 || /usr/bin/setdb restjavad.useextramb true || true
/usr/bin/setdb iapplxrpm.timeout 300 || true
/usr/bin/setdb icrd.timeout 180 || true
/usr/bin/setdb restjavad.timeout 180 || true
/usr/bin/setdb restnoded.timeout 180 || true
Note: Not all of these settings apply to all versions of BIG-IP, so the examples above will return true if the command is unsuccessful. F5 recommends using the BIG-IP versions listed in the Validated BIG-IP versions table above for best results.
F5 recommends adding the following to the beginning of your startup scripts to log the Serial Console as well as provide a consistent logging location.
mkdir -p /var/log/cloud
LOG_FILE=/var/log/cloud/startup-script.log
npipe=/tmp/$$.tmp
trap "rm -f $npipe" EXIT
mknod $npipe p
tee <$npipe -a $LOG_FILE /dev/ttyS0 &
exec 1>&-
exec 1>$npipe
exec 2>&1
Some environments have additional resource requirements for the instance to enable the Serial Console/Logging. For instance, see this Azure example.
If the BIG-IP or Service is not reachable, first review the console logs (see your cloud provider for details) for any errors.
If possible, try to log in to the BIG-IP instance via SSH (mgmt interface) to examine the logs. Serial console login may also be possible but only for partial onboarding successes where passwords were configured successfully (ex. with Declarative Onboarding). To verify the BIG-IP deployment, perform the following steps:
-
Check the
startup-scriptsent to user_data to make sure it was installed/interpolated correctly:- AWS:
cat /opt/cloud/instance/user-data.txt
- Azure:
cat /var/lib/waagent/CustomData | base64 -d
- GCP:
- Option Not Available
- AWS:
-
Check the logs (in order of invocation):
- waagent logs:
- /var/log/waagent.log (Azure Only)
- cloud-init logs:
- /var/log/boot.log
- /var/log/cloud-init.log
- /var/log/cloud-init-output.log
- runtime-init logs:
- /var/log/cloud/startup-script.log: This file contains events that happen prior to execution of f5-bigip-runtime-init. For example, if the Runtime Init package failed to download, the installer failed to download a file, etc.
- /var/log/cloud/bigIpRuntimeInit.log: This file contains events logged by the f5-bigip-runtime-init onboarding utility. If the configuration is invalid causing onboarding to fail, you will see those events logged here. If the deployment is successful, you will see an event with the body "All operations completed successfully".
- Automation Tool Chain logs:
- /var/log/restnoded/restnoded.log: This file contains events logged by the BIG-IP Automation Toolchain components. If an Automation Toolchain declaration fails to deploy, you will see more details for those events logged here.
- waagent logs:
-
GENERAL LOG TIP: Search for the most critical error level errors first (for example,
egrep -i err /var/log/[log name]).
Help with troubleshooting individual Automation Toolchain components can be found at F5's Public Cloud Docs:
- DO: https://clouddocs.f5.com/products/extensions/f5-declarative-onboarding/latest/troubleshooting.html
- AS3: https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/userguide/troubleshooting.html
- FAST: https://clouddocs.f5.com/products/extensions/f5-appsvcs-templates/latest/userguide/troubleshooting.html
- TS: https://clouddocs.f5.com/products/extensions/f5-telemetry-streaming/latest/userguide/troubleshooting.html
- CFE: https://clouddocs.f5.com/products/extensions/f5-cloud-failover/latest/userguide/troubleshooting.html
BIG-IP Runtime Init uses the "extension metadata" file (JSON document) to identify package delivery URL for each BIG-IP Automation Toolchain extension. Each Runtime Init build includes an extension metadata file and it is stored under the following directory: src/lib/bigip/toolchain/toolchain_metadata.json
The latest "extension metadata" file is published on F5 CDN under the following location: https://cdn.f5.com/product/cloudsolutions/f5-extension-metadata/latest/metadata.json
As a part of the installation workflow, by default, Runtime Init will attempt to fetch the latest available version of the extension metadata from the Internet and will replace the built-in file; however, providing --skip-toolchain-metadata-sync flag to the Runtime Init installation allows you to skip extension metadata sync and fall back to the built-in extension metadata file.
In a situation, when custom extension_metadata file needs to be used, Runtime Init installation allows to override delivery URL for the "extension metadata" file using the --toolchain-metadata-file-url parameter. See the Installer section for more details.
For more information on BIG-IP cloud solutions, including manual configuration procedures for some deployment scenarios, see F5's Public Cloud Docs.
The example declarations in this document are intended to provide reference onboarding configurations for BIG-IP Virtual Editions. Read more about Support Policies.
If you find an issue, we would love to hear about it.
- Use the Issues link on the GitHub menu bar in this repository for items such as enhancement or feature requests and non-urgent bug fixes. Tell us as much as you can about what you found and how you found it.
Copyright 2014-2022 F5 Networks Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.