terraform-azure-aks

Terraform module for deploying a Domino on AKS

Use

Create a Domino development AKS cluster

module "aks_cluster" {
  source  = "github.com/dominodatalab/terraform-azure-aks"

  cluster = "cluster-name"
}

Manual Deploy

For new projects, the following needs to be done only once for the workspace.

1. az login
2. terraform init
3. terraform workspace new [cluster-name]

Run the Terraform deployment

1. export TF_VAR_service_principal_name=<service-principal-appid>
2. export TF_VAR_service_principal_secret=<service-principal-password>
3. terraform apply -auto-approve

Access AKS cluster

1. az aks get-credentials --resource-group [cluster-name] --name [cluster-name]

Troubleshooting

1. No access to Azure backend store In this case you would need to override the backend configuration. This can be done via the command line:

terraform init -backend-config="storage_account_name=<YourAzureStorageAccountName>" -backend-config="container_name=tfstate" -backend-config="access_key=<YourStorageAccountAccessKey>" -backend-config="key=codelab.microsoft.tfstate"

Development

Please submit any feature enhancements, bug fixes, or ideas via pull requests or issues.

Requirements

Name	Version
terraform	>= 1.3.0
azurerm	~> 3.45
random	~> 3.1

Providers

Name	Version
azurerm	~> 3.45
random	~> 3.1

Modules

Name	Source	Version
domino_acr_ep	./modules/private_endpoint	n/a
domino_blob_ep	./modules/private_endpoint	n/a
domino_shared_ep	./modules/private_endpoint	n/a

Resources

Name	Type
azurerm_container_registry.domino	resource
azurerm_federated_identity_credential.hephaestus	resource
azurerm_federated_identity_credential.importer	resource
azurerm_kubernetes_cluster.aks	resource
azurerm_kubernetes_cluster_node_pool.aks	resource
azurerm_log_analytics_solution.logs	resource
azurerm_log_analytics_workspace.logs	resource
azurerm_monitor_diagnostic_setting.control_plane	resource
azurerm_private_dns_zone.acr_private_dns_zone	resource
azurerm_private_dns_zone.aks_private_dns_zone	resource
azurerm_private_dns_zone.blob_private_dns_zone	resource
azurerm_private_dns_zone.shared_private_dns_zone	resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_acr_vnet_link	resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_aks_vnet_link	resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_blob_vnet_link	resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_shared_vnet_link	resource
azurerm_role_assignment.aks_domino_acr	resource
azurerm_role_assignment.aks_domino_private_acr	resource
azurerm_role_assignment.aks_domino_shared	resource
azurerm_role_assignment.aks_file_share_contributor	resource
azurerm_role_assignment.aks_network	resource
azurerm_role_assignment.hephaestus_acr	resource
azurerm_role_assignment.identity_assign_pdnsz	resource
azurerm_role_assignment.identity_assign_rg	resource
azurerm_role_assignment.identity_assign_vnet	resource
azurerm_storage_account.domino	resource
azurerm_storage_account.domino_shared	resource
azurerm_storage_account_network_rules.domino_blob_rules	resource
azurerm_storage_account_network_rules.domino_shared_rules	resource
azurerm_storage_container.domino_containers	resource
azurerm_storage_share.shared_store	resource
azurerm_user_assigned_identity.aks_assigned_identity	resource
azurerm_user_assigned_identity.hephaestus	resource
random_id.log_analytics_workspace_name_suffix	resource
azurerm_kubernetes_service_versions.selected	data source
azurerm_resource_group.aks	data source
azurerm_subnet.aks_subnet	data source
azurerm_subscription.current	data source
azurerm_virtual_network.aks_vnet	data source

Inputs

Name	Description	Type	Default	Required
additional_node_pools	additional node pools	map(object({ enable_node_public_ip = optional(bool, false) vm_size = string zones = list(string) node_labels = map(string) node_os = optional(string, "AzureLinux") node_taints = optional(list(string), []) enable_auto_scaling = optional(bool, true) min_count = optional(number, 0) max_count = number initial_count = optional(number, 0) max_pods = optional(number, 30) os_disk_size_gb = optional(number, 128) }))	{}	no
aks_subnet_name	Subnet name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true.	string	null	no
aks_vnet_name	VNet name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true.	string	null	no
aks_vnet_rg_name	VNet Resource Groupe name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true.	string	null	no
api_server_authorized_ip_ranges	The IP ranges to whitelist for incoming traffic to the masters	list(string)	null	no
cluster_sku_tier	The Domino cluster SKU (defaults to Free)	string	null	no
cni_overlay_enabled	Flag to determine whether to use overlay network settings	bool	false	no
containers	storage containers to create	map(object({ container_access_type = string }))	{ "backups": { "container_access_type": "private" }, "projects": { "container_access_type": "private" } }	no
deploy_id	Domino Deployment ID.	string	n/a	yes
dns_service_ip	IP address assigned to the Kubernetes DNS service, used when CNI Overlay is enabled	string	"100.97.0.10"	no
kubeconfig_output_path	kubeconfig path	string	n/a	yes
kubernetes_nat_gateway	Managed NAT Gateway configuration	object({ idle_timeout_in_minutes = optional(number, 4) managed_outbound_ip_count = number } )	null	no
kubernetes_version	Optional Kubernetes version to provision. Allows partial input (e.g. 1.18) which is then chosen from azurerm_kubernetes_service_versions.	string	null	no
log_analytics_workspace_sku	log analytics sku	string	"PerGB2018"	no
namespaces	Namespace that are used for generating the service account bindings	object({ platform = string, compute = string })	n/a	yes
node_pools	default node pools	object({ compute = object({ enable_node_public_ip = optional(bool, false) vm_size = optional(string, "Standard_D8s_v4") zones = optional(list(string), ["1", "2", "3"]) node_labels = optional(map(string), { "dominodatalab.com/node-pool" = "default" }) node_os = optional(string, "AzureLinux") node_taints = optional(list(string), []) enable_auto_scaling = optional(bool, true) min_count = optional(number, 0) max_count = optional(number, 10) initial_count = optional(number, 1) max_pods = optional(number, 30) os_disk_size_gb = optional(number, 128) }), platform = object({ enable_node_public_ip = optional(bool, false) vm_size = optional(string, "Standard_D8s_v4") zones = optional(list(string), ["1", "2", "3"]) node_labels = optional(map(string), { "dominodatalab.com/node-pool" = "platform" }) node_os = optional(string, "AzureLinux") node_taints = optional(list(string), []) enable_auto_scaling = optional(bool, true) min_count = optional(number, 1) max_count = optional(number, 3) initial_count = optional(number, 1) max_pods = optional(number, 60) os_disk_size_gb = optional(number, 128) }), gpu = object({ enable_node_public_ip = optional(bool, false) vm_size = optional(string, "Standard_NC6s_v3") zones = optional(list(string), []) node_labels = optional(map(string), { "dominodatalab.com/node-pool" = "default-gpu" "nvidia.com/gpu" = "true" }) node_os = optional(string, "AzureLinux") node_taints = optional(list(string), [ "nvidia.com/gpu=true:NoExecute" ]) enable_auto_scaling = optional(bool, true) min_count = optional(number, 0) max_count = optional(number, 1) initial_count = optional(number, 0) max_pods = optional(number, 30) os_disk_size_gb = optional(number, 128) }) system = object({ enable_node_public_ip = optional(bool, false) vm_size = optional(string, "Standard_DS4_v2") zones = optional(list(string), ["1", "2", "3"]) node_labels = optional(map(string), {}) node_os = optional(string, "AzureLinux") node_taints = optional(list(string), []) enable_auto_scaling = optional(bool, true) min_count = optional(number, 1) max_count = optional(number, 6) initial_count = optional(number, 1) max_pods = optional(number, 60) os_disk_size_gb = optional(number, 128) }) })	{ "compute": {}, "gpu": {}, "platform": {}, "system": {} }	no
pod_cidr	CIDR block for Kubernetes pods, used when CNI Overlay is enabled	string	"192.168.0.0/16"	no
private_acr_enabled	Flag to determine whether to deploy a private ACR	bool	false	no
private_cluster_enabled	Flag to determine whether to deploy a private AKS	bool	false	no
private_cluster_public_fqdn_enabled	Flag to determine whether to use a public FQDN when deploying a private AKS cluster	bool	null	no
registry_tier	registry tier	string	"Standard"	no
resource_group	Name or id of optional pre-existing resource group to install AKS in	string	n/a	yes
service_cidr	CIDR block for Kubernetes services, used when CNI Overlay is enabled	string	"100.97.0.0/16"	no
storage_account_replication_type	storage replication	string	"LRS"	no
storage_account_tier	storage account tier	string	"Standard"	no
tags	Tags to apply to resources	map(string)	{}	no

Outputs

Name	Description
aks_identity	AKS managed identity
blob_dns_zone_name	blob dns zone name
containers	storage details
domino_acr	Azure Container Registry details
oidc_issuer_url	OIDC issuer url
private_cluster_enabled	Flag to determine if AKS is private or public
shared_storage_account	shared storage account
storage_account	storage account
workload_identities	service identities