DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Coding
  3. Languages
  4. CloudFormation Update: YAML, Cross-Stack References, and Simplified Substitution

CloudFormation Update: YAML, Cross-Stack References, and Simplified Substitution

Amazon has updated their CloudFormation templates that now includes YAML, references across stacks, multiline support, and a simpler way to substitute strings.

Michael Wittig user avatar by
Michael Wittig
·
Oct. 26, 16 · Tutorial
Like (2)
Save
Tweet
Share
9.59K Views

Join the DZone community and get the full member experience.

Join For Free

In mid-September, AWS released a big update to CloudFormation.

The update contained:

  • YAML support: You can now write your CloudFormation templates in YAML.
  • Cross stack references: You can now export values from one stack and use them in another.
  • Simplified substitution: You can more easily embed variables in strings.

After one month of using the new features, I want to share my learnings with you.

YAML Support

During the last four weeks, I discovered three main advantages of using YAML over JSON to describe my CloudFormation templates:

  1. Support for multi-line strings.
  2. It is possible to use comments within a template.
  3. YAML is more compact than JSON.

Let me explain them in more detail.

Support for Multi-Line Strings

The UserData property usually consists of many lines. In JSON, there was no elegant way to express this. Instead, you used the Fn::Join function of CloudFormation.

{
  "Resources": {
    "MyLC": {
      "Type": "AWS::AutoScaling::LaunchConfiguration",
      "Properties":
        "UserData": {"Fn::Join": ["\n"], [
          "#!/bin/bash -x",
          "/opt/aws/bin/cfn-init -v --stack my-stack --resource MyLC --region eu-west-1",
          "/opt/aws/bin/cfn-signal -e $? --stack my-stac --resource MyASG --region eu-west-1"
        ]}
      }
    }
  }
}

In YAML, this looks much nicer:

---
Resources:
  MyLC:
    Type: 'AWS::AutoScaling::LaunchConfiguration'
    Properties:
      UserData: !Base64 |
        #!/bin/bash -x
        /opt/aws/bin/cfn-init -v --stack my-stack --resource MyLC --region eu-west-1
        /opt/aws/bin/cfn-signal -e $? --stack my-stack --resource MyASG --region eu-west-1

In YAML, you can also add comments.

---
Resources:
  MyLC:
    Type: 'AWS::AutoScaling::LaunchConfiguration'
    Properties:
      # This how a comment looks like
      UserData: !Base64 |
        #!/bin/bash -x
        /opt/aws/bin/cfn-init -v --stack my-stack --resource MyLC --region eu-west-1
        /opt/aws/bin/cfn-signal -e $? --stack my-stack --resource MyASG --region eu-west-1

Nothing fancy, but very helpful.

YAML Is More Compact Than JSON

I converted all our Free Templates for AWS CloudFormation from JSON to YAML. See how the number of lines changed:

templateJSON linesYAML lines
ec2/ec2-auto-recovery460403 (-13%)
jenkins/jenkins2-ha-agents16361599 (-3%)
jenkins/jenkins2-ha747656 (-13%)
security/account-password-policy160161 (+0%)
security/cloudtrail134101 (-25%)
security/config9277 (-17%)
static-website/static-website147183 (+24%)
vpc/vpc-2azs289253 (-13%)
vpc/vpc-3azs356306 (-15%)
vpc/vpc-4azs423359 (-16%)
vpc/vpc-nat-gateway3951 (+30%)
vpc/vpc-nat-instance518446 (-14%)
wordpress/wordpress-ha670602 (-11%)

On average, the templates get smaller.

Ouch

What I do not like is that there is more than one way to represent strings:

  • Sometimes you need quotes; sometimes they are optional.
  • Sometimes you need single quotes; sometimes you can use double quotes.

The rules seem to be more complicated than just using double quotes all the time like in JSON. I finally ended up with using single quotes all the time unless the string contains only [a-zA-Z0-9]. However, that is just my personal style. It ends up in a total mess if multiple people work on a single template.

Cross Stack References

Instead of putting everything in a single template, it can make sense to split them up. One very common example is the VPC. You create one CloudFormation stack that contains your VPC. Each application that you run is also a stack, but they depend on the VPC stack. Before Cross Stack References you could solve this problem with Parameters.

Application Template

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Parameters": {
    "Subnet": {
      "Description": "Use Subnet output from vpc stack.",
      "Type": "AWS::EC2::Subnet::Id"
    }
  },
  "Resources": {
    "VirtualMachine": {
      "Type": "AWS::EC2::Instance",
      "Properties": {
        "SubnetId": {"Ref": "Subnet"},
      }
    }
  }
}

With Cross Stack References, you can export values in one stack and import them in another stack. Let’s see how this works.

VPC Template

---
AWSTemplateFormatVersion: '2010-09-09'
Resources:
  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: !Sub '10.0.0.0/16'
  SubnetAPublic:
    Type: 'AWS::EC2::Subnet'
    Properties:
      AvailabilityZone: !Select [0, !GetAZs '']
      CidrBlock: '10.0.0.0/20'
      VpcId: !Ref VPC

Outputs:
  Subnet:
    Description: 'Subnet.'
    Value: !Ref SubnetAPublic
    Export:
      Name: 'vpc-subnet'

The VPC stack now exports the subnet ID under the name vpc-subnet.

Application Template

---
AWSTemplateFormatVersion: '2010-09-09'
Resources:
  VirtualMachine:
    Type: 'AWS::EC2::Instance'
    Properties:
      SubnetId: !ImportValue 'vpc-subnet'

The application stack imports the subnet id.

With Cross Stack References you can pass data from one stack to another.

Ouch

It is not possible to export and import a list of values at the moment. You can export a comma separated string, but you are not able to import that string as a list.

Keep in mind that you can not change exported values.

Let’s again have a look at the UserData property. It usually not only consists of many lines, but it also references some resources. In JSON, I used Fn::Join and Ref for this like crazy:

{
  "Resources": {
    "MyLC": {
      "Type": "AWS::AutoScaling::LaunchConfiguration",
      "Properties":
        "UserData": {"Fn::Join": [""], [
          "#!/bin/bash -x\n",
          "/opt/aws/bin/cfn-init -v --stack ", {"Ref": "AWS::StackName"}, " --resource ", {"Ref": "MyLC"}, " --region ", {"Ref": "AWS::Region"}, "\n",
          "/opt/aws/bin/cfn-signal -e $? --stack ", {"Ref": "AWS::StackName"}, " --resource ", {"Ref": "MyASG"}, " --region ", {"Ref": "AWS::Region"}, "\n"
        ]}
      }
    }
  }
}

In YAML, this looks cleaner:

---
Resources:
  MyLC:
    Type: 'AWS::AutoScaling::LaunchConfiguration'
    Properties:
      UserData:
        'Fn:Base64': !Sub |
          #!/bin/bash -x
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ${MyLC} --region ${AWS::Region}
          /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ${MyASG} --region ${AWS::Region}

So the magic is the Fn::Sub function or shorter !Sub. It substitutes references inside ${} automatically with the value.

Ouch

Unfortunately, it is not possible to write !Base64 !Sub |. The documentation tells us:

If you use the short form and immediately include another function in the valueToEncode parameter, use the full function name for at least one of the functions. For example, the following syntax is invalid:

!Base64 !Sub string

Instead, use the full function name for at least one of the functions, as shown in the following examples:

'Fn::Base64':
  !Sub string

I hope this is fixed shortly.

The CloudFormation update makes our lives easier:

  • We can now write the template in YAML which makes them shorter.
  • We can add comments and deal with multi-line strings without struggles.
  • The simplified substitution removes complex Fn::Joins from our templates.
  • We can pass data between stacks with Cross Stack References.

As always, new features have rough edges:

  • It is not possible to write !Base64 !Sub |. Instead, we need to write: 'Fn::Base64': !Sub |
  • It is not possible to export and import a list values with Cross Stack References.
  • YAML and strings are complicated. Many options and many ways to do express the same in different ways.

All in all, I can recommend the new YAML format.

I started with an automatic conversion from JSON to YAML with:

ruby -ryaml -rjson -e 'puts YAML.dump(JSON.parse(STDIN.read))' < template.json > template.yaml

Then, that hard manual work begins when you want to use simplified substitution and YAML multi-line support.

YAML Template Strings JSON

Published at DZone with permission of Michael Wittig. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • 7 Awesome Libraries for Java Unit and Integration Testing
  • Kubernetes vs Docker: Differences Explained
  • Easy Smart Contract Debugging With Truffle’s Console.log
  • An Introduction to Data Mesh

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: