(cloudformation-include): Cannot use computed outputs from nested stack #27233

ozeebee · 2023-09-21T08:43:56Z

Describe the bug

Hi,
I must integrate some pre-built components (provided as CFN templates) in my app as nested stacks.
When I try to use computed outputs (whose value use an intrinsic function for instance) exposed by the nested stack in my CDK stack, deployment fails with message:

❌ Deployment failed: Error [ValidationError]: Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.

It looks like the generated CFN is incorrect.
This is confirmed by running cloudformation validate-template with the CLI.

Expected Behavior

To be able to use nested stack's outputs in my CDK constructs.

Current Behavior

Invalid CFN template generates an error during deployment:

❌ Deployment failed: Error [ValidationError]: Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.

Reproduction Steps

For instance, I have a CFN template creating some VPC Endpoint, exposing outputs like this:

  "Outputs": {
    "SimpleOutput": {
      "Description": "Simple Output",
      "Value": {
        "Ref": "VpcEndpoint"
      }
    },
    "ComputedOutput": {
      "Description": "Computed Output",
      "Value": {
        "Fn::Join": [
          ",",
          {
            "Fn::GetAtt": "VpcEndpoint.DnsEntries"
          }
        ]
      }
    }
  }

Now consider this stack:

interface MyNestedStackProps extends NestedStackProps {
  VpcId: string,
  SubnetIDs: string[]
}

class MyNestedStack extends NestedStack {
  readonly cfnInc: cfninc.CfnInclude
  constructor(scope: Construct, id: string, props: MyNestedStackProps) {
    super(scope, id)
    // Include CFN template
    this.cfnInc = new cfninc.CfnInclude(this, 'CfnTemplate', {
      templateFile: 'cfn-template.json',
      parameters: {
        'VpcId': props.VpcId,
        'SubnetIDs': props.SubnetIDs
      }
    })
  }
}

export interface CdkbugreproducerStackProps extends StackProps {
  vpcName: string
}

export class CdkbugreproducerStack extends Stack {
  constructor(scope: Construct, id: string, props: CdkbugreproducerStackProps) {
    super(scope, id, props)

    const vpc = Vpc.fromLookup(this, 'MyVPC', { vpcName: 'My-VPC' })
    const privateSubnetIds = vpc.privateSubnets.map(_ => _.subnetId)

    // Include CFN template as a NESTED STACK in the stack
    const nested = new MyNestedStack(this, 'MyNestedStack', {
      VpcId: vpc.vpcId,
      SubnetIDs: privateSubnetIds
    })
    const simpleOutput = nested.cfnInc.getOutput('SimpleOutput').value
    const computedOutput = nested.cfnInc.getOutput('ComputedOutput').value

    new ssm.StringParameter(this, 'MyParamSimpleOutput', {
      parameterName: 'MyParamFromStackSimpleOutput',
      stringValue: simpleOutput
    })
    new ssm.StringParameter(this, 'MyParamComputedOutput', {
      parameterName: 'MyParamFromStackComputedOutput',
      stringValue: computedOutput
    })
  }
}

So, in this simple example, I just want to get the DNS entries of the VPC Endpoint (created in the nested stack) and create an SSM parameter with its value, which fails.
In practice, we have similar error with other stacks exposing outputs with Fn::Sub, like for instance in:

"DomainEndpoint": {
    "Description": "ElasticSearch Service domain endpoint",
    "Value": {
        "Fn::Sub": "https://${ElasticSearch.DomainEndpoint}"
    }
},

Possible Solution

No idea. I don't even have work-around for this.

Additional Information/Context

No response

CDK CLI Version

2.96.2 (build 3edd240)

Framework Version

2.96.2

Node.js Version

v18.12.0

OS

macos

Language

Typescript

Language Version

3.9.10

Other information

I have a full reproducer project available in case it can help (can push it to GitHub or GitLab if needed).

Some observations after testing various things:

This error does not happen when including the CFN template directly into the CDK stack, but I must add them as nested stacks (for compliance reasons)
In the code above, If I comment the SSM parameter for "ComputedOutput", it works.

Here's the output of the synthesis for this "ComputedOutput" SSM parameter:

...
  "MyParamComputedOutput8C461DF5": {
   "Type": "AWS::SSM::Parameter",
   "Properties": {
    "Name": "MyParamFromStackComputedOutput",
    "Type": "String",
    "Value": {
     "Fn::Join": [
      ",",
      {
       "Fn::GetAtt": [
        "MyNestedStackNestedStackMyNestedStackNestedStackResource9C617903",
        "Outputs.cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11DnsEntries"
       ]
      }
     ]
    }
   },
...

And the generated CF for the outputs in the included nested stack looks like this:

 "Outputs": {
  "SimpleOutput": {
   "Description": "Simple Output",
   "Value": {
    "Ref": "VpcEndpoint"
   }
  },
  "ComputedOutput": {
   "Description": "Computed Output",
   "Value": {
    "Fn::Join": [
     ",",
     {
      "Fn::GetAtt": "VpcEndpoint.DnsEntries"
     }
    ]
   }
  },
  "cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11Ref": {
   "Value": {
    "Ref": "VpcEndpoint"
   }
  },
  "cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11DnsEntries": {
   "Value": {
    "Fn::GetAtt": "VpcEndpoint.DnsEntries"
   }
  }
 }

The text was updated successfully, but these errors were encountered:

ozeebee · 2023-09-22T08:03:03Z

I continued experimenting on this issue.
I found out that actually, this is issue is not related to cfn-include but to nested stacks in CDK.
If I re-write this purely in CDK, this fails too:

interface VpcEndpointNestedStackProps extends NestedStackProps {
  vpc: ec2.IVpc
}

/** A simple nested stack that creates a VPC Endpoint and tries to export some of its properties */
export class VpcEndpointNestedStack extends NestedStack {
  readonly vpcEndpointId: string
  readonly vpcEndpointDnsEntries: string[]
  readonly vpcEndpointDnsEntriesStr: string

  constructor(scope: Construct, id: string, props: VpcEndpointNestedStackProps) {
    super(scope, id)

    const vpce = new ec2.InterfaceVpcEndpoint(this, 'VpcEndpoint', {
      vpc: props.vpc,
      service: ec2.InterfaceVpcEndpointAwsService.S3,
      privateDnsEnabled: false
    })

    this.vpcEndpointId = vpce.vpcEndpointId
    this.vpcEndpointDnsEntries = vpce.vpcEndpointDnsEntries
    this.vpcEndpointDnsEntriesStr = vpce.vpcEndpointDnsEntries.join(',')
  }
}

export interface CdkbugreproducerStackProps extends StackProps {
  vpcName: string
}

export class CdkbugreproducerStack extends Stack {
  constructor(scope: Construct, id: string, props: CdkbugreproducerStackProps) {
    super(scope, id, props)

    const vpc = Vpc.fromLookup(this, 'MyVPC', { vpcName: 'My-VPC' })

    const nested = new VpcEndpointNestedStack(this, 'MyNestedStack', { vpc })

    new CfnOutput(this, 'CdkSimpleOutput', { value: nested.vpcEndpointId }) // OK, the "simple" prop works as expected
    // new CfnOutput(this, 'CdkComputedOutput', { value: nested.vpcEndpointDnsEntries.join(',') }) // Error: Found an encoded list token string in a scalar string context. Use 'Fn.select(0, list)' (not 'list[0]') to extract elements from token lists.
    // new CfnOutput(this, 'CdkComputedOutput', { value: Fn.join(',', nested.vpcEndpointDnsEntries) }) // Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.
    // new CfnOutput(this, 'CdkComputedOutput', { value: nested.vpcEndpointDnsEntriesStr }) // Error: Found an encoded list token string in a scalar string context. Use 'Fn.select(0, list)' (not 'list[0]') to extract elements from token lists.
    // new CfnOutput(this, 'CdkComputedOutput', { value: Fn.select(0, nested.vpcEndpointDnsEntries) }) // Template error: Fn::Select requires a list argument with two elements: an integer index and a list
    // new CfnOutput(this, 'CdkComputedOutput', { value: nested.vpcEndpointDnsEntries[0] }) // Error: Found an encoded list token string in a scalar string context. Use 'Fn.select(0, list)' (not 'list[0]') to extract elements from token lists.
  }
}

In the stack above, there is no way to get the DNS entries from the VPC Endpoint created in the nested stack.

indrora · 2023-09-22T18:14:48Z

Thank you for your report.

Can you include the generated clouformation from your reproduction here to help debug?

github-actions · 2023-09-24T20:04:36Z

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

ozeebee · 2023-09-25T08:30:36Z

Sure,
Here's the synthesized root stack CFN:

{
 "Resources": {
  "MyNestedStackNestedStackMyNestedStackNestedStackResource9C617903": {
   "Type": "AWS::CloudFormation::Stack",
   "Properties": {
    "TemplateURL": {
     "Fn::Join": [
      "",
      [
       "https://s3.eu-west-1.",
       {
        "Ref": "AWS::URLSuffix"
       },
       "/cdk-hnb659fds-assets-123456789012-eu-west-1/fde4170827a9cd567630c317b8970bc9fe5bcd4754144208c276d15a5cb56856.json"
      ]
     ]
    }
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/MyNestedStack.NestedStack/MyNestedStack.NestedStackResource",
    "aws:asset:path": "cdkbugreproducercfMyNestedStackDDCA32DC.nested.template.json",
    "aws:asset:property": "TemplateURL"
   }
  },
  "MyParamSimpleOutputD4138D08": {
   "Type": "AWS::SSM::Parameter",
   "Properties": {
    "Name": "MyParamFromStackSimpleOutput",
    "Type": "String",
    "Value": {
     "Fn::GetAtt": [
      "MyNestedStackNestedStackMyNestedStackNestedStackResource9C617903",
      "Outputs.cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11Ref"
     ]
    }
   },
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/MyParamSimpleOutput/Resource"
   }
  },
  "MyParamComputedOutput8C461DF5": {
   "Type": "AWS::SSM::Parameter",
   "Properties": {
    "Name": "MyParamFromStackComputedOutput",
    "Type": "String",
    "Value": {
     "Fn::Join": [
      ",",
      {
       "Fn::GetAtt": [
        "MyNestedStackNestedStackMyNestedStackNestedStackResource9C617903",
        "Outputs.cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11DnsEntries"
       ]
      }
     ]
    }
   },
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/MyParamComputedOutput/Resource"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {
    "Analytics": "v2:deflate64:H4sIAAAAAAAA/zPSszTTM1JMLC/WTU7J1s3JTNKrDi5JTM7WCUotzi8tSk7VcU7Lg4gAFcUXF+eCFBRl5qUHJBYl5qaWpBaBVMA5tSAeTG+tTl5+SqpeVrF+maGFnqGRnoFiVnFmpm5RaV5JZm6qXhCEBgAz2BWJhQAAAA=="
   },
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/CDKMetadata/Default"
   }
  }
 },
 "Parameters": {
  "BootstrapVersion": {
   "Type": "AWS::SSM::Parameter::Value<String>",
   "Default": "/cdk-bootstrap/hnb659fds/version",
   "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
  }
 },
 "Rules": {
  "CheckBootstrapVersion": {
   "Assertions": [
    {
     "Assert": {
      "Fn::Not": [
       {
        "Fn::Contains": [
         [
          "1",
          "2",
          "3",
          "4",
          "5"
         ],
         {
          "Ref": "BootstrapVersion"
         }
        ]
       }
      ]
     },
     "AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."
    }
   ]
  }
 }
}

And the nested stack CFN :

{
 "Description": "reproducer for CDK bug related to cfn outputs",
 "AWSTemplateFormatVersion": "2010-09-09",
 "Resources": {
  "VpcEndpoint": {
   "Type": "AWS::EC2::VPCEndpoint",
   "Properties": {
    "ServiceName": {
     "Fn::Sub": "com.amazonaws.${AWS::Region}.s3"
    },
    "SubnetIds": [
     "subnet-0ab01234556ccc123",
     "subnet-0b3482397dd0080d"
    ],
    "VpcEndpointType": "Interface",
    "VpcId": "vpc-0d2be234324ede232"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {
    "Analytics": "v2:deflate64:H4sIAAAAAAAA/yWLOw7CMBAFz5LeXogLBHVEQQMoSLSRsTfS5rOO7DUUiLtjSPXmjTQGDjswlX0l7fyoJ3rA+4xJ0N/EulG5KWTfhzhbocAdsZuyR2h6Pq2oStqhMz91vzZH9ksgFlXuJcuS/9RiCjk6/CgOpR7S5lnvoTawrYZEpGNmoRmhXfcLRYNAEZUAAAA="
   },
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/MyNestedStack/CDKMetadata/Default"
   }
  }
 },
 "Outputs": {
  "SimpleOutput": {
   "Description": "Simple Output",
   "Value": {
    "Ref": "VpcEndpoint"
   }
  },
  "ComputedOutput": {
   "Description": "Computed Output",
   "Value": {
    "Fn::Join": [
     ",",
     {
      "Fn::GetAtt": "VpcEndpoint.DnsEntries"
     }
    ]
   }
  },
  "cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11Ref": {
   "Value": {
    "Ref": "VpcEndpoint"
   }
  },
  "cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11DnsEntries": {
   "Value": {
    "Fn::GetAtt": "VpcEndpoint.DnsEntries"
   }
  }
 }
}

Hope this helps !

… invalid outputs (aws#27233) Referencing a list attribute of a resource defined within a NestedStack synthesizes successfully but the nested stack will fail deployment with the error: ``` Template format error: Every Value member must be a string. ``` This prevents deploying resources such into NestedStack instances if a reference to that resource's list attribute needs to be referenced in a cross-stack context. For example, deploying a `InterfaceVpcEndpoint` instance in a nested stack and attempting to reference its `vpcEndpointDnsEntries` property within a different stack will cause this error. To fix this issue, a similar strategy to `exportStringListValue` is used to join the reference's values into a string and expose that value as the output from the nested stack. The reference to the serialized value is then re-exported as normally needed to hoist it to the top-level parent stack. The final reference that imports the value is then re-written to also deserialize the imported string back to the original list. fixes aws#27233 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

… invalid outputs Referencing a list attribute of a resource defined within a NestedStack synthesizes successfully but the nested stack will fail deployment with the error: ``` Template format error: Every Value member must be a string. ``` This prevents deploying resources such into NestedStack instances if a reference to that resource's list attribute needs to be referenced in a cross-stack context. For example, deploying a `InterfaceVpcEndpoint` instance in a nested stack and attempting to reference its `vpcEndpointDnsEntries` property within a different stack will cause this error. To fix this issue, a similar strategy to `exportStringListValue` is used to join the reference's values into a string and expose that value as the output from the nested stack. The reference to the serialized value is then re-exported as normally needed to hoist it to the top-level parent stack. The final reference that imports the value is then re-written to also deserialize the imported string back to the original list. fixes aws#27233 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

ozeebee added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 21, 2023

github-actions bot added the @aws-cdk/cloudformation-include Issues related to the "CFN include v.20" package label Sep 21, 2023

indrora added p1 @aws-cdk/aws-cloudformation Related to AWS CloudFormation feature/coverage-gap Gaps in CloudFormation coverage by L2 constructs and removed needs-triage This issue or PR still needs to be triaged. labels Sep 22, 2023

khushail added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Sep 22, 2023

github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label Sep 24, 2023

github-actions bot removed closing-soon This issue will automatically close in 4 days unless further comments are made. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels Sep 25, 2023

github-actions bot mentioned this issue Oct 1, 2023

Monthly issue metrics report - Sep 2023 #27367

Closed

pahud added p2 and removed p1 labels Jun 11, 2024

brandondahler linked a pull request Dec 18, 2024 that will close this issue

fix(core): cross-stack references to NestedStack list values produces invalid outputs #32575

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(cloudformation-include): Cannot use computed outputs from nested stack #27233

(cloudformation-include): Cannot use computed outputs from nested stack #27233

ozeebee commented Sep 21, 2023

ozeebee commented Sep 22, 2023

indrora commented Sep 22, 2023

github-actions bot commented Sep 24, 2023

ozeebee commented Sep 25, 2023 •

edited

Loading

(cloudformation-include): Cannot use computed outputs from nested stack #27233

(cloudformation-include): Cannot use computed outputs from nested stack #27233

Comments

ozeebee commented Sep 21, 2023

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

CDK CLI Version

Framework Version

Node.js Version

OS

Language

Language Version

Other information

ozeebee commented Sep 22, 2023

indrora commented Sep 22, 2023

github-actions bot commented Sep 24, 2023

ozeebee commented Sep 25, 2023 • edited Loading

ozeebee commented Sep 25, 2023 •

edited

Loading