Unable to create cluster if Service-Linked Role already exists

[johnowennixon]

My account already has another Elasticsearch cluster. When applying my project with your module, Terraform reports:

module.elasticsearch.aws_iam_service_linked_role.default: 1 error(s) occurred:

aws_iam_service_linked_role.default: Error creating service-linked role with name es.amazonaws.com: InvalidInput: Service role name AWSServiceRoleForAmazonElasticsearchService has been taken in this account, please try a different suffix.

[aknysh]

@johnowennixon thanks for testing!
we'll look how to fix it

[aknysh]

@johnowennixon

In the latest release
#6

we added

variable "create_iam_service_linked_role" {
  type        = "string"
  default     = true
  description = "Whether to create `AWSServiceRoleForAmazonElasticsearchService` service-linked role. Set it to `false` if you already have an ElasticSearch cluster created in the AWS account and AWSServiceRoleForAmazonElasticsearchService already exists. See https://github.com/terraform-providers/terraform-provider-aws/issues/5218 for more info"
}

which could be set to "false" to disable the service-linked role creation.
It's probably the easiest way to do it instead of trying to check if the role already exists using TF.
Let us know if it works for you.

[myles-mcdonnell]

Unless I've got the wrong end of the stick this breaks the TF model in that I the TF author have to decide true or false for this variable depending on the state of the target account, whereas the promise of TF is that it will make these decisions at runtime, no?

[moosahmed]

yeah so for example if you have the service role already created in your dev environment but not in your stg environment.

you need to set it to true or false depending on the env.

[unfor19]

Another option would be providing custom_suffix, as stated in the official terraform docs - https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role#argument-reference

If we were able to provide iam_service_linked_role_custom_suffix that would solve a lot of problems. I'll create a PR, and hopefully it will be merged

Update- I've just realized it's not possible - Custom suffix is not allowed for es.amazonaws.com

[yash-kalwani]

Does setting "create_iam_service_linked_role" variable solve this issue? I tried doing this and it did not work and eventually had to set the count to 0 manually depending upon the env i am in.

Do the 9 downvotes on the solution denote that it is not working or that is not a good solution?

[aknysh]

There is NO way to determine in Terraform if the Service Linked Role is already created on AWS in the account.
So yes, you have to check it manually, and if it does exist, set create_iam_service_linked_role to false.
Otherwise, set it to true.
We have not found a better way of doing this.

Also, one a ES cluster is created in the account manually from the AWS console, the role will be created automatically.
This complicates the matter even more.

If somebody has a better option please chime in.

[galeaspablo]

@aknysh Thanks for asking for options. I have an idea :)

There is NO way to determine in Terraform if the Service Linked Role is already created on AWS in the account.

The name and ARN of the role are always the same. This could be used to determine if the role exists.

[David3Ar]

@aknysh Thanks for asking for options. I have an idea :)

There is NO way to determine in Terraform if the Service Linked Role is already created on AWS in the account.

The name and ARN of the role are always the same. This could be used to determine if the role exists.

Has anyone done this yet ? Would like to do this directly in cloudformation seems to be much nicer then adding a boolean value to make the creation optional.

[simonlewandowski]

maybe:

data "aws_iam_roles" "role" {
  name_regex = "AWSServiceRoleForAmazonOpenSearchService"
}

locals {
  role_exists = can(data.aws_iam_roles.role.arns)
}

resource "aws_iam_service_linked_role" "opensearch" {
  count = local.role_exists ? 0 : 1

  aws_service_name = "opensearchservice.amazonaws.com"
}

[avoidik]

@simonlewandowski how do you define depends_on = [...] between aws_elasticsearch_domain and aws_iam_service_linked_role in your case?

[nitrocode]

It's probably best to keep the new variable false and manage the service role creation from its own root terraform module (component)

See https://github.com/cloudposse/terraform-aws-components/tree/main/modules/iam-service-linked-roles

[cl0udf0x]

@nitrocode I like the idea of removing the dependency on the service linked role from OpenSearch deployment by using the module, however when doing a quick test using just the resource it fails if the service linked role has already been created. Perhaps I've misunderstood what you were suggesting?

Error: 
creating IAM Service Linked Role (opensearchservice.amazonaws.com): InvalidInput: Service role name AWSServiceRoleForAmazonOpenSearchService has been taken in this account, please try a different suffix.

NOTE: Opensearch service linked role doesn't support custom suffix.

The tactical fix I've used is:

#main.tf
module "opensearch" {
...
...
local.create_opensearch_service_linked_role[var.environment]


#locals.tf
locals {
  create_opensearch_service_linked_role = {
    prod    = "true" #AWS prod account
    qa      = "true" #AWS non-prod account
    uat     = "false" #AWS non-prod account
    test    = "false" #AWS non-prod account
    dev2    = "false" #AWS non-prod account
    dev     = "false" #AWS non-prod account
  }
}

This isn't ideal. Do you know if its possible to raise a request with the AWS team to enable custom suffix on the OpenSearch service linked role and if its something they would consider fixing? Thanks.

[nitrocode]

The service role has to be created once per account.

If you instantiate this module more than once and it tries to create the same service role, it will fail. For this reason, is why I suggest setting create_iam_service_linked_role = false and creating the service linked role separately.

Contrived example of separating the service linked role from the creation of elasticsearch instances

# components/terraform/service-linked-roles
# In a separate root dir or component
resource "aws_iam_service_linked_role" "default" {
  aws_service_name = "es.amazonaws.com"
  description      = "${local.service_linked_role_name} Service-Linked Role"
}

# components/terraform/elasticsearch
# In a separate root dir or component
module "elasticsearch" {
  source = "cloudposse/elasticsearch/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version     = "x.x.x"

  for_each = toset([
    "es1",
    "es2",
    "es3",
  ])

  name = each.key

  # set to false since this is already created outside of the module
  create_iam_service_linked_role = false

  # ...
}