Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Can not create aws_bedrock_custom_model resource with fine tuning job #39956

Open
jovanamarko opened this issue Oct 30, 2024 · 2 comments
Labels
bug Addresses a defect in current functionality. service/bedrock Issues and PRs that pertain to the bedrock service.

Comments

@jovanamarko
Copy link

jovanamarko commented Oct 30, 2024

Terraform Core Version

Terraform v1.9.2 on darwin_amd64

AWS Provider Version

hashicorp/aws

Affected Resource(s)

aws_bedrock_custom_model

Expected Behavior

Running terraform apply should deploy the resources which includes S3 bucket, IAM roles and Bedrock custom model with fine tuning job.

Actual Behavior

The terraform apply fails with following error:

│ Error: Provider produced invalid plan

│ Provider "registry.terraform.io/hashicorp/aws" planned an invalid value for aws_bedrock_custom_model.project_name_model.base_model_identifier: planned value cty.UnknownVal(cty.String) does not match config
│ value cty.StringVal("amazon.titan-image-generator-v2:0").

│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

provider.tf file looks like the following:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }

  backend "s3" {
    bucket  = "project-terraform-s3-state"
    key     = "terraform.tfstate"
    region  = "us-east-1"
    profile = "project"
  }
}

provider "aws" {
  region                   = var.region
  profile                  = var.profile
}

and the creation of the custom model with fine tuning job is:

# S3 Bucket for training data
resource "aws_s3_bucket" "training_data_bucket" {
  bucket = "bedrock-training-data"
  force_destroy = true
}

# IAM Role for Bedrock Fine-Tuning
resource "aws_iam_role" "bedrock_fine_tuning_role" {
  name = "BedrockFineTuningRole"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [{
      Effect = "Allow",
      Principal = {
        Service = "bedrock.amazonaws.com"
      },
      Action = "sts:AssumeRole"
    }]
  })
}

# Attach S3 permissions to the role
resource "aws_iam_role_policy" "bedrock_fine_tuning_policy" {
  name = "BedrockFineTuningPolicy"
  role = aws_iam_role.bedrock_fine_tuning_role.id

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [{
      Effect = "Allow",
      Action = ["s3:GetObject", "s3:PutObject"],
      Resource = [
        "${aws_s3_bucket.training_data_bucket.arn}/*"
      ]
    }]
  })
}



data "aws_bedrock_foundation_model" "titan_image" {
  model_id = "amazon.titan-image-generator-v2:0"
}

resource "aws_bedrock_custom_model" "project_name_model" {
  custom_model_name     = "project-name-model"
  job_name              = "project-name-model-job-1"
  base_model_identifier = data.aws_bedrock_foundation_model.titan_image.model_id
  role_arn              = aws_iam_role.bedrock_fine_tuning_role.arn

  hyperparameters = {
    "stepCount"               = "4000"
    "epochCount"              = "5"
    "batchSize"               = "32"
    "learningRate"            = "0.005"
    "learningRateWarmupSteps" = "0"
  }

  training_data_config {
    s3_uri = "s3://${aws_s3_bucket.training_data_bucket.id}/train.jsonl"
  }

  output_data_config {
    s3_uri = "s3://${aws_s3_bucket.training_data_bucket.id}/fine-tuned-output/"
  }
}

Steps to Reproduce

  • match the provider.tf file as stated and have an other file for the custom model with fine tuning job as stated before and run terraform init then terraform plan and/or terraform apply

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

@jovanamarko jovanamarko added the bug Addresses a defect in current functionality. label Oct 30, 2024
Copy link

Community Note

Voting for Prioritization

  • Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

  • If you are interested in working on this issue, please leave a comment.
  • If this would be your first contribution, please review the contribution guide.

@github-actions github-actions bot added service/bedrock Issues and PRs that pertain to the bedrock service. service/iam Issues and PRs that pertain to the iam service. service/s3 Issues and PRs that pertain to the s3 service. needs-triage Waiting for first response or review from a maintainer. labels Oct 30, 2024
@jovanamarko jovanamarko changed the title [Bug]: [Bug]: Can not create aws_bedrock_custom_model resource with fine tuning job Oct 30, 2024
@justinretzolk justinretzolk removed service/iam Issues and PRs that pertain to the iam service. service/s3 Issues and PRs that pertain to the s3 service. needs-triage Waiting for first response or review from a maintainer. labels Oct 31, 2024
@acwwat
Copy link
Contributor

acwwat commented Nov 1, 2024

@jovanamarko Could you please try using the ARN for the base model (data.aws_bedrock_foundation_model.titan_image.model_arn) instead of the ID for the base_model_identifier argument in the aws_bedrock_custom_model resource to see if it works around the problem?

I suspect there's a data type conversion in the code since it's expecting a strict ARN but the AWS API is more flexible and allows the ID.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Addresses a defect in current functionality. service/bedrock Issues and PRs that pertain to the bedrock service.
Projects
None yet
Development

No branches or pull requests

3 participants