v0.23.3

Pipeline Enhancements

Notes:

• The documenation has been updated in several areas.
• The OpenStack Install Pipeline was not tested in this release; use the pipeline with caution

Install PCF Pipelines

• (ALL) Remove unused garden_network_pool_cidr param from install params story
• (ALL) Fix typos in the terraform scripts where we wrote opsmman instead of opsman story
• (ALL) Params for opsman and pas versions have been updated to point to 2.0.x story
• (ALL) Fix params.yml for opsman admin and password order to be more natural story
• (ALL) Errand names in params and bootstrap have been updated story
• (ALL) configure-bosh has been replaced with configure-director due to deprecations in omstory
• (ALL) Removed a number of unused params from params.yml for various IAAS'sstory
• (ALL) Added the frontend_idle_timeout param to all IAAS's story
• (Azure) terraform_azure_account_name has been changed to terraform_azure_storage_account_name to be more explicit story
• (AWS) In IAM, we ensure that PcfAdminPolicy has ec2:CopyImage permission story
• (Azure) The Azure pipeline will now trigger the next job after config-opsman-auth has completed story
• (GCP) db_cloudsqldb_tier is now configurable story
• (Azure) The Azure pipeline now splits cidr in terraform template correctly (story)[https://www.pivotaltracker.com/story/show/156352337]
• (AWS, vSphere, GCP) Updated/removed Autoscaler errand names in documentation (story)[https://www.pivotaltracker.com/story/show/156706226]
• (ALL) Fixed typos in README story
• (vSphere) No longer need to provide an NSX cert when nsx_networking_enabled is set to false story

Upgrade Pipelines

• (vSphere) upgrade-ops-manager will now honor the opsman_vm_folder story
• (vSphere) When upgrading OpsMan, a 3 second sleep was added when waiting for VM to start story
• (Azure, AWS, GCP) If an image region is chosen that does not exist, the pipeline will choose an available region (if available) or fail with a meaningful message story
  *Note: For AWS and vSphere, the original disk size is preserved; you cannot set the disk size using the new parameter. For AWS, we've added this param in error it will be removed in the next release.

Param changes introduced in pcf-pipelines v0.23.3

Pipeline	Param	Notes
install-pcf	garden_network_pool_cidr	ALL; removed unused param
install-pcf	configure-director	ALL; replaces configure-bosh
install-pcf	configure-bosh	ALL; replaced by configure-director
install-pcf	frontend_idle_timeout	ALL; new param
install-pcf	terraform_azure_storage_account_name	Azure; changed from terraform_azure_account_name to be more explicit
install-pcf	terraform_azure_account_name	Azure; changed to terraform_azure_storage_account_name to be more explicit
install-pcf	db_cloudsqldb_tier	GCP; new param

v0.23.1

Pipeline Enhancements

Notes:

• The Norm team (pcf-pipelines team) created a new docker image called pcfnorm/rootfs which we have started using with v23.1 (replacing czero/rootfs with pcfnorm/rootfs). We will also started versioning this image.
• The OpenStack Install Pipeline was not tested in this release; use the pipeline with caution

Install PCF Pipelines

• (ALL) Self-signed certs generated by the pipelines are now being signed with SHA-256 signatures story
• (ALL) Enhanced the pipelines to make system_domain and apps_domain configurable story
• (ALL) Removed erroneous params opsman_client_id and opsman_client_secret from the install pipelines story
• (ALL) Changed the Point of Entry ssl params to poe_ssl_name1, poe_ssl_cert1, and poe_ssl_key1, allowing up to three param entries story
• (ALL) Changed the Credhub Encryption Key params to credhub_encryption_key_name_1, credhub_encryption_key_1, allowing up to three param entries story
• (AWS) Fix such that the Terraform scripts for the load balancer have more sensible defaults for health_check: interval and health_check: healthy_thresholdstory
• (AWS) Fix to Terraform scripts - now the load balancer health_check targets the correct protocol, endpoint, and port story
• (Azure) Make configurable Ops Manager's private IP configurable; introduced a new param called azure_opsman_priv_ip story
• (Azure) Locked the Azure Provider version in our Terraform scripts to use 1.0.0 story
• (vSphere) Fixed a minor bug whereby the params.yml contained inverted commas which could cause issues when deploying the pipeline story
• (vSphere) Fix to accept the spaces in a resource pool name or the Ops Manager vm name story
• (vSphere, GCP) Fix such that the Ops Manager --> Create Networks --> Service Network checkbox is only checked for Dynamic Services network story

Upgrade Pipelines

• (upgrade-tile) Added a new param enable_errand which allows you to disable, enable, or leave the state of errands as-is when doing a upgrade to a patch release story

• (upgrade-ops-man; GCP and Azure) Fixes a Ops Manager disk size limitation by adding a new param pcf_opsman_disk_size_in_gb, allowing you to set the disk size for the new Ops Man vm* story

  *Note: For AWS and vSphere, the original disk size is preserved; you cannot set the disk size using the new parameter. For AWS, we've added this param in error it will be removed in the next release.

Param changes introduced in pcf-pipelines v0.23.1

Pipeline	Param	Notes
install-pcf	opsman_client_id	ALL; removed erroneous param
install-pcf	opsman_client_secret	ALL; removed erroneous param
install-pcf	poe_ssl_name1	ALL; up to three entries; renamed from networking_poe_ssl_certs
install-pcf	poe_ssl_cert1	ALL; up to three entries; renamed from networking_poe_ssl_certs
install-pcf	poe_ssl_key1	ALL; up to three entries; renamed from networking_poe_ssl_certs
install-pcf	credhub_encryption_key_name_1	ALL; up to three entries; renamed from credhub_encryption_keys
install-pcf	credhub_encryption_key_1	ALL; up to three entries; renamed from credhub_encryption_keys
install-pcf	azure_opsman_priv_ip	Azure; new param
install-pcf	system_domain	AWS, GCP; new param
install-pcf	apps_domain	AWS, GCP; new param
install-pcf	is_service_network	vSphere; removed param
upgrade-ops-manager	pcf_opsman_disk_size_in_gb	GCP and Azure; new param
upgrade-tile	enable_errands	new param

v0.23.0

Pipeline Enhancements

Notes:

• (all install-pcf pipelines) This release supports PCF 2.0. A few notes about this release:
  • PCF 2.0.x introduced functionality to support multiple certificates for HAProxy and GoRouter. This release currently only supports multi-cert functionality for vSphere, Azure, and OpenStack.
  • this release currently only supports using the internal MySQL database for CredHub. For AWS and GCP, in order to opt in to using CredHub you must manually configure the instance count to 2 in PAS --> Resource Config. For vSphere, Azure, and OpenStack you can configure the instance count directly in the params.yml using credhub_instances.
• (all install-pcf pipelines) opsman_client_id and opsman_client_secret were added to the pipelines however note functionality has not been tested on install-pcf pipelines and will be removed in subsequent releases. Ensure both params are blank when deploying the pipelines story
• (vSphere, Azure, OpenStack) new resources syslog_scheduler_instances and credhub_instances were added to the pipelines; and one resource param renamed: loggregator_trafficcontroller_instances

Install PCF Pipelines

• (ALL) Renamed ssl cert params across all IAASs (see Param Changes below) in favour of networking_poe_ssl_certs story
• (ALL) Adds a new param routing_custom_ca_certificates which is an optional param to configure GoRouter and HAProxy to trust additional CAs story
• (ALL) Adds a new param routing_tls_termination to configure how PCF handles the X-Forwarded-Client-Cert header story
• (ALL) Includes a new param credhub_encryption_keys to configure CredHub integration. Currently the pipelines only support use of the internal MySQL database story
• (GCP) fix for TCP Firewall rule whereby there was a mismatch in naming tags for the Load Balancer and the forwarding rule story
• (OpenStack) Updated our docker image for OpenStackClient to use pyOpenSSL==17.4.0 story

Param changes introduced in pcf-pipelines v23

Pipeline	Param	Notes
install-pcf	networking_poe_ssl_certs	AWS; replaces ERT_SSL_CERT
install-pcf	networking_poe_ssl_certs	Azure, GCP; replaces pcf_ert_ssl_cert
install-pcf	networking_poe_ssl_certs	OpenStack, vSphere; replaces ssl_cert
install-pcf	credhub_encryption_keys	ALL; new param
install-pcf	routing_custom_ca_certificates	ALL; new param
install-pcf	routing_tls_termination	ALL; new param
install-pcf	credhub_instances	vSphere, Azure, OpenStack; new resource param
install-pcf	loggregator_trafficcontroller_instances	vSphere, Azure, OpenStack; replaces loggregator_traffic_controller_instances
install-pcf	syslog_scheduler_instances	vSphere, Azure, OpenStack; new resource param`

v0.22.0

Pipeline changes

Important Notes:

• (all install-pcf pipelines) opsman_client_id and opsman_client_secret were added to the pipelines however note functionality has not been tested on install-pcf pipelines and will be removed in subsequent releases. Ensure both params are blank when deploying the pipelines story
• (SRT) the SRT pipeline is still a work-in-progress; we hope to release support for this pipeline soon

Install PCF Pipelines

• (ALL) Pipelines now use a new docker image czero/rootfs which utilizes terraform 0.11.0; older versions of the pipeline will continue using czero/cflinuxfs2 story
• (AWS and GCP) includes a new security_acknowledgement param, which ensures you have set the appropriate Application Security Groups that control application network policies story
• (AWS) Changes S3 Signature v2 to v4, allowing for supportability across new regions story
• (Azure) Bug fix for network security group not attaching to a load balancer that has a public IP story
• (Azure) Bug fix for the bootstrap-terraform-state job whereby the job would complete successfully even though bootstrapping the terraform state failed story
• (Azure) Removal of extraneous param azure_vm_password, which was not being invoked by the pipeline story
• (GCP) Bug fix whereby when pcf_ert_ssl_cert: generate was set in the params, a cert is generated for the create-infrastructure job and a redundant one was generated during the configure-ert job story
• (GCP) check-opsman-dns now the performs dns retries
• (vSphere, Azure, OpenStack) includes a new mysql_static_ips param, allowing for configuring IP address(es) for the MySQL proxy instances configured on an external load balancer story
• (vSphere, Azure, OpenStack) adds support for configuring ERT --> Email Notifications --> SMTP Enable Automatic STARTTLS story
• (vSphere) includes new mysql_nsx params, allowing configuration of a load balancer in front of pools of MySQL story
• (vSphere) adds support for configuring a new param om_vm_folder for using a vCenter folder for Ops Manager story

Upgrade Pipelines

• (upgrade-tile) Adds support to use opsman_client_id and opsman_client_secret to authenticate with Ops Manager if it is configured to use SAML story
• (upgrade-ops-manager; ALL) Adds support to use opsman_client_id and opsman_client_secret to authenticate with Ops Manager if it is configured to use SAML story
• (upgrade-ops-manager; ALL) now the pipeline checks again to see if there are any pending changes after exporting Ops Manager settings story

Param Changes introduced in pcf-pipelines v22

Pipeline	Param	Notes
install-pcf	security_acknowledgement	GCP and AWS; new param
install-pcf	mysql_nsx_security_group	vSphere; new param
install-pcf	om_vm_folder	vSphere; new param
install-pcf	mysql_nsx_lb_edge_name	vSphere; new param
install-pcf	mysql_nsx_lb_pool_name	vSphere; new param
install-pcf	mysql_nsx_lb_security_group	vSphere; new param
install-pcf	mysql_nsx_lb_port	vSphere; new param
install-pcf	mysql_static_ips	vSphere, Azure, OpenStack; new param
install-pcf	smtp_enable_starttls_auto	vSphere, Azure, OpenStack; new param
install-pcf	smtp_auth_mechanism	vSphere, Azure, OpenStack; new param
install-pcf	smtp_address	vSphere, Azure, OpenStack; new param
install-pcf	smtp_from	vSphere, Azure, OpenStack; new param
install-pcf	smtp_port	vSphere, Azure, OpenStack; new param
install-pcf	smtp_user	vSphere, Azure, OpenStack; new param
install-pcf	smtp_pwd	vSphere, Azure, OpenStack; new param
install-pcf	opsman_client_id	ALL; new param however functionality does not work and will be removed
install-pcf	opsman_client_secret	ALL; new param however functionality does not work and will be removed
install-pcf	azure_vm_password	Azure; removed param
upgrade-ops-manager and upgrade-tile	opsman_client_id	AWS, Azure, GCP, vSphere, Azure; new param
upgrade-ops-manager and upgrade-tile	opsman_client_secret	AWS, Azure, GCP, vSphere, Azure; new param

v0.21.2

This release is the same as pcf-pipelines v22 and was released accidentally. Although both releases are the same, it is preferable to use v22.

v0.21.1

Pipeline changes

Along with the enhancements and bug fixes described below, this release:

• adds the airgapped pipelines to the PivNet release for pipelines
• adds a new required param for the upgrade-tile pipeline
• makes the pipelines more consistent and user-friendly; select parameters have been renamed for vSphere, GCP, AWS, Azure. See the full list of param changes below.

Install PCF Pipelines

• (ALL) Fix for the stage-product task so that now it now correctly queries and returns the correct number of unstaged products story
• (AWS) Adds support for S3 buckets in multi-regions story
• (AWS) Bug fix to the opsman_allow_ssh_cidr_ranges to correctly quote all cidrs in the list story
• (Azure) Minor fix for the wipe-env job to update the list of resources to destroy commit
• (GCP) Updated the check-opsman-dns task to use opsman_domain_or_ip_address for consistency story
• (GCP) Fix for the create-infrastructure job such that it can be re-run without encountering a ssl cert deletion issue story
• (vSphere) Adds new params vcenter_insecure and vcenter_ca_cert which allows you configure vCenter cert validation story
• (vSphere, Azure, OpenStack) Fixes a bug whereby the config-ert task attempts to set a property when route_services is set to disable story

Upgrade Pipelines

• (upgrade-tile) Adds a new param product_globs which defaults to *pivotal; if you are using this pipeline to upgrade service tiles, leave this property set to the default. If you are using this pipeline to upgrade ERT, change this parameter to cf*pivotal to prevent the task from downloading the unsupported Small Footprint Runtime product; story
• (upgrade-tile) Fix for the stage-product task so that now it now correctly queries and returns the correct number of unstaged products story
• (upgrade-ops-manager vSphere) Add new params vcenter_insecure and vcenter_ca_cert allowing you to configure vCenter cert validation story

Param Changes introduced in pcf-pipelines v21

View the full list in a spreadsheet.

See this directory for a list of scripts to help you more easily update existing install-pcf parameters.

upgrade-tile

Pipeline	Param	Notes
upgrade-tile	product_globs	new param; see instructions above

vSphere

Pipeline	Param	Notes
install-pcf	opsman_ssh_password	renamed; formerly om_ssh_pwd
install-pcf	vcenter_datacenter	renamed; formerly vcenter_data_center
install-pcf	vcenter_datastore	renamed; formerly om_data_store
install-pcf	vcenter_insecure	new param
install-pcf	vcenter_ca_cert	new param

GCP

Pipeline	Param	Notes
install-pcf	opsman_admin_username	renamed; formerly pcf_opsman_admin_username
install-pcf	opsman_admin_password	renamed; formerly pcf_opsman_admin_password
install-pcf	opsman_trusted_certs	renamed; formerly pcf_opsman_trusted_certs
upgrade-ops-manager	gcp_project_id	renamed; formerly opsman_gcp_project
upgrade-ops-manager	gcp_zone	renamed; formerly opsman_gcp_zone
upgrade-ops-manager	gcp_service_account_key	renamed; formerly opsman_gcp_credfile_contents

AWS

Pipeline	Param	Notes
install-pcf	aws_access_key_id	renamed; formerly TF_VAR_aws_access_key
install-pcf	aws_region	renamed; formerly TF_VAR_aws_region
install-pcf	aws_secret_access_key	renamed; formerly TF_VAR_aws_secret_key
install-pcf	amis_nat	renamed; formerly TF_VAR_amis_nat
install-pcf	aws_az1	renamed; formerly TF_VAR_az1
install-pcf	aws_az2	renamed; formerly TF_VAR_az2
install-pcf	aws_az3	renamed; formerly TF_VAR_az3
install-pcf	aws_cert_arn	renamed; formerly TF_VAR_aws_cert_arn
install-pcf	dynamic_services_subnet_cidr_az1	renamed; formerly TF_VAR_dynamic_services_subnet_cidr_az1
install-pcf	dynamic_services_subnet_cidr_az2	renamed; formerly TF_VAR_dynamic_services_subnet_cidr_az2
install-pcf	dynamic_services_subnet_cidr_az3	renamed; formerly TF_VAR_dynamic_services_subnet_cidr_az3
install-pcf	ert_subnet_cidr_az1	renamed; formerly TF_VAR_ert_subnet_cidr_az1
install-pcf	ert_subnet_cidr_az2	renamed; formerly TF_VAR_ert_subnet_cidr_az2
install-pcf	ert_subnet_cidr_az3	renamed; formerly TF_VAR_ert_subnet_cidr_az3
install-pcf	infra_subnet_cidr_az1	renamed; formerly TF_VAR_infra_subnet_cidr_az1
install-pcf	nat_ip_az1	renamed; formerly TF_VAR_nat_ip_az1
install-pcf	nat_ip_az2	renamed; formerly TF_VAR_nat_ip_az2
install-pcf	nat_ip_az3	renamed; formerly TF_VAR_nat_ip_az3
install-pcf	opsman_ip_az1	renamed; formerly TF_VAR_opsman_ip_az1
install-pcf	public_subnet_cidr_az1	renamed; formerly TF_VAR_public_subnet_cidr_az1
install-pcf	public_subnet_cidr_az2	renamed; formerly TF_VAR_public_subnet_cidr_az2
install-pcf	public_subnet_cidr_az3	renamed; formerly TF_VAR_public_subnet_cidr_az3
install-pcf	rds_subnet_cidr_az1	renamed; formerly TF_VAR_rds_subnet_cidr_az1
install-pcf	rds_subnet_cidr_az2	renamed; formerly TF_VAR_rds_subnet_cidr_az2
install-pcf	rds_subnet_cidr_az3	renamed; formerly TF_VAR_rds_subnet_cidr_az3
install-pcf	services_subnet_cidr_az1	renamed; formerly TF_VAR_services_subnet_cidr_az1
install-pcf	services_subnet_cidr_az2	renamed; formerly TF_VAR_services_subnet_cidr_az2
install-pcf	services_subnet_cidr_az3	renamed; formerly TF_VAR_services_subnet_cidr_az3
install-pcf	vpc_cidr	renamed; formerly TF_VAR_vpc_cidr
install-pcf	opsman_admin_password	renamed; formerly OPSMAN_PASSWORD
install-pcf	opsman_admin_username	enamed; formerly OPSMAN_USER
install-pcf	aws_key_name	enamed; formerly TF_VAR_aws_key_name

Azure

Pipeline	Param	Notes
install-pcf	azure_client_id	renamed; formerly azure_service_principal_id
install-pcf	azure_client_secret	renamed; formerly azure_service_principal_password
install-pcf	azure_storage_container_name	renamed; formerly terraform_statefile_container
install-pcf	opsman_admin_username	renamed; formerly pcf_opsman_admin_username
install-pcf	opsman_admin_password	renamed; formerly pcf_opsman_admin_password

v0.19.2

Pipeline changes

Due to versioning issues, we are publishing v19.2 instead of v20; the next release will be v21.

Install PCF Pipelines

• (ALL) Introduces a new internet_connected parameter, which allows you to disable or enable internet connected resources story
• (ALL) Adds a new param called container_networking_nw_cidr which allows you to override the overlay subnet to avoid ip collision story
• (AWS and OpenStack) Bug fix whereby dynamic services were not enabled as a service network story
• (AWS) Bug fix to correct subnets assigned to dynamic service subnets story
• (AWS) Enhancement to make the pipeline.yml consistently use opsman_domain_or_ip_address story
• (AWS) Fix which allows for the S3 region to work with more regions other than the default us-east story
• (AZURE) Adds a new param called pcf_opsman_disk_size_in_gb so that you can configure the OM disk size

Upgrade Pipelines

Bug Fixes

• Enhances the apply-updates pipeline so that if there is is no stemcell upgrade available, the upload-stemcell job displays a message saying it is skipping the upgrade story

Param Changes introduced in pcf-pipelines v19

Pipeline	Param	Notes
install-pcf	internet_connected	all pipelines; new param
install-pcf	container_networking_nw_cidr	all pipelines; new param
install-pcf	pcf_opsman_disk_size_in_gb	Azure pipeline; new param

v0.19.1

Pipeline changes

Note: The v19 pcf-pipelines release was pulled due to bug found in the install-pcf pipeline for vSphere, whereby the wipe-env job did not work as expected. v19 was pulled from Github and Pivotal Network.

Install PCF Pipelines

• (OpenStack) Includes support for the OpenStack install-pcf pipeline on PCF 1.12.x. Please note there is a known issue whereby the deploy-ert smoke errand may fail, even though the pipeline will actually succeed, and the deployment completes successfully. Running the smoke tests manually works; but there is a known issue with running the smoke errand on OpenStack. story
• (All Pipelines) Includes a minor fix to the pipelines introduced by ERT 1.12.4/5. story
• (vSphere) Includes a fix to the wipe-env job for the vSphere install-pcf pipeline (when used in conjunction with the upgrade-ops-manager pipeline); now the job, when triggered, identifies the active Ops Man vm, and deletes it. The old Ops Manager vm must be deleted manually. story
• (GCP) Updates the Load Balancer health check timeouts to more sensible defaults story
• (GCP) Increases the timeout_sec to 30 for the timeout value on GCP Load Balancers, or the amount of time the backend service will wait before considering the request a failure. story

Upgrade Pipelines

Bug Fixes

• Fixes a bug for the upgrade-ops-manager pipeline whereby if there is more than one Ops Manager with the same IP, the pipeline selects the incorrect Ops Manager vm to shutdown story
• Includes a fix for upgrade-ops-manager pipeline on vSphere; previously when the new Ops Manager was created, the old Ops Manager vm did not remain powered off. story
• Includes a minor fix to the pipelines introduced by ERT 1.12.4/5.
• Bug fix for the upgrade-ops-manager pipeline for Azure where it pulled attempted to pull the aws stemcell instead of azure story
• Further enhancements to the pipeline include: eliminating the need for theproduct-name param in the upgrade-tile pipeline; a new job added to upgrade-tile pipeline called delete-unused-products which can be triggered optionally; and removal of the regulator job in the apply-updates pipeline

Param Changes introduced in pcf-pipelines v19

Pipeline	Param	Notes
upgrade-tile	product_name	removed

v0.18.0

Pipeline changes

This release makes pcf-pipelines compatible with Ops Manager v1.12.x and Elastic Runtime v1.12.0 (v1.12.1, 1.12.2, 1.12.3) for vSphere, Azure, AWS, and GCP.

Install PCF Pipelines

• In ERT v1.12 --> Networking, the ssl termination points were removed from the configuration
• The following jobs were removed from ERT v1.12: etcd, ccdb, uaadb

Param Changes introduced in ERT v1.12

Pipeline	Param	Notes
ALL	haproxy_forward_tls	new param
ALL	haproxy_backend_ca	new param
AWS, GCP	routing_disable_http	new param; this property already exists for vSphere and Azure but it's called disable_http_proxy
ALL	router_tls_ciphers	new param
ALL	haproxy_tls_ciphers	new param
vSphere	ssl_termination_point	removed

Upgrade Pipelines

• fixes a bug whereby the upgrade-ops-manager pipeline sporadically turns off the wrong VM if there are multiple Ops Managers in different networks running on the same IPs (upgrade-ops-manager pipeline)
• the wait-opsman-clear now only checks the running status of the last change (upgrade-tile and upgrade-ops-manager pipeline)

v0.15.4

Pipeline changes:

Install PCF on vSphere

• Pipeline should fix handling of NSX Manager CA certificate, so that it gets passed to Ops Mgr
• Note: NSX params have not been verified yet for the security group settings in ERT, even though they exist

Install PCF on Azure

• added the new azure CLI which uses az as the command

All Upgrade Pipelines

• Removed the job which performed a patch-upgrade-only check from these pipelines, because there were issues with the Ops Mgr API not returning product versions in all cases.

Rootfs changes (czero/cflinuxfs2):

• add the azure cli