AWS Serverless Application Model (SAM) is a framework for building serverless applications on AWS. One of the components of SAM is a template specification. SAM templates would look and feel familiar to anyone who has used AWS CloudFormation to define their infrastructure as code, however they are not completely interchangeable. There are multiple reasons why you might want to convert from SAM to native CloudFormation:

  • You want to deploy the app using CloudFormation StackSets. SAM uses the AWS::Serverless transform in its templates and transforms are not supported by stack sets.
  • You want to deploy the app as part of an AWS Landing Zone (ALZ) account baseline. ALZ uses stack sets as the mechanism to deploy baseline resources and so suffers from the same constraint as the point above.
  • Your operating system of choice isn't documented in the SAM installation instructions and you're uncertain how to install from source or doubtful it will work at all (I'm looking at fellow OpenBSD and FreeBSD users here).

This post will show you how to take an existing SAM application and convert it to a CloudFormation template (CFT). As a CFT, the challenges listed above can be avoided.

Satisfy pre-requisites⌗

These pre-reqs need to be met before continuing:

  1. Install the SAM CLI
  2. Create an Amazon S3 bucket to store the serverless code artifacts that the SAM template generates. At a minimum, you will need permission to put objects into the bucket.
  3. The permissions applied to your IAM identity must include iam:ListPolicies.
  4. You must have AWS credentials configured either via the AWS CLI or in your shell's environment via the AWS_* environment variables.
  5. Git installed.
  6. Python 3.x installed.
  7. (Optional) Install Python's virtualenvwrapper.

Sorry BSD users, yes, this process requires the SAM CLI be installed. Boot a Linux VM/instance?

Do the template conversion⌗

I'm going to use a sample SAM app from the SAM documentation to demonstrate the conversion. In a real scenario, you would already have your own SAM app and would skip the step below where a new project is initialized.

Per the SAM documentation for this example app, use the sam command to initialize the project:

sam init \
  --location https://github.com/aws-samples/cookiecutter-aws-sam-s3-rekognition-dynamodb-python \
  --no-input

Open the template.yaml file in the app and look for any serverless functions that have been defined to use the Python runtime. Change the runtime value to match a version of Python that is installed on your system. For example, if you have Python 3.8 installed, change the parameter to look like this:

Runtime: python3.8

I don't know if this step is required for other runtimes. Python is my jam. 🐍

At this point, I recommend you create a Python virtual environment to isolate the packages that you're about to install. If you happen to have multiple versions of Python installed, be sure to specify the version that matches what you configured in the template.yaml file.

mkvirtualenv -p /usr/local/opt/python@3.8/bin/python3.8 sam2cfn

Jump into the SAM app's directory and run the sam package command. This will take the serverless function code from the local file system, zip it up, and stash it in your Amazon S3 bucket. The template file that gets output from this command will contain references to the bucket and object that was uploaded for the function which is how the function's code would be deployed into AWS Lambda if you were to run the sam deploy command.

sam package --output-template-file packaged.yaml --s3-bucket <BUCKET_NAME>

If you're using credentials from the AWS CLI and need to specify a profile name, run sam package --profile <PROFILE_NAME> .... Otherwise the default profile will be used or the AWS_* environment variables will be consulted (if set).

The packaged.yaml file could be fed to CloudFormation as a template, but at this point the template still uses the AWS::Serverless transform so there is more work still to do. Fortunately, the SAM source code contains a script for converting a SAM template to a CFT. Clone the SAM repo and then install the Python dependencies.

git clone https://github.com/aws/serverless-application-model.git
pip install -r serverless-application-model/requirements/base.txt
pip install pyyaml

The conversion script does not let you specify an AWS credential profile name as one of its arguments. If you're not already using the AWS_* environment variables to pass credentials to these tools, set the AWS_PROFILE variable to the name of the credential profile you're using. The conversion script makes some AWS API calls and will throw an exception if it can't determine your credentials.

Run the conversion script to convert packaged.yaml to a CFT:

python serverless-application-model/bin/sam-translate.py \
  --template-file=packaged.yaml \
  --output-template=cft.json

The resulting CFT will be in JSON format, so if you're at all sane, you'll now convert this JSON file to YAML 😄. The cfn-flip tool was written to do exactly this.

pip install cfn-flip
cfn-flip -i json -o yaml cft.json cft.yaml
rm cft.json

A CloudFormation template that does not use any transforms is now ready as cft.yaml.

AWSTemplateFormatVersion: '2010-09-09'
Description: SAM app that uses Rekognition APIs to detect text in S3 Objects and stores
  labels in DynamoDB.
Resources:
  SourceImageBucket:
    Type: AWS::S3::Bucket
    DependsOn:
      - DetectTextInImageBucketEvent1Permission
    Properties:
      NotificationConfiguration:
        LambdaConfigurations:
          - Function: !GetAtt 'DetectTextInImage.Arn'
            Event: s3:ObjectCreated:*
  DetectTextInImage:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket: BUCKET_NAME
        S3Key: 915a69e82ff6d649e804e681769cf9b3
      Description: Uses Rekognition APIs to detect text in S3 Objects and stores the
        text and labels in DynamoDB.
      Handler: src/app.lambda_handler
      MemorySize: 512
      Role: !GetAtt 'DetectTextInImageRole.Arn'
      Runtime: python3.8
      Timeout: 30
      Environment:
        Variables:
          TABLE_NAME: !Ref 'ResultsTable'
      Tags:
        - Key: lambda:createdBy
          Value: SAM
  DetectTextInImageRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Action:
              - sts:AssumeRole
            Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: DetectTextInImageRolePolicy0
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - s3:GetObject
                Resource: arn:aws:s3:::*
              - Effect: Allow
                Action:
                  - rekognition:DetectText
                  - rekognition:DetectLabels
                Resource: '*'
              - Effect: Allow
                Action:
                  - dynamodb:GetItem
                  - dynamodb:PutItem
                  - dynamodb:Scan
                  - dynamodb:UpdateItem
                Resource: !Join
                  - ''
                  - - 'arn:aws:dynamodb:'
                    - !Ref 'AWS::Region'
                    - ':'
                    - !Ref 'AWS::AccountId'
                    - :table/
                    - !Ref 'ResultsTable'
      Tags:
        - Key: lambda:createdBy
          Value: SAM
  DetectTextInImageBucketEvent1Permission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Ref 'DetectTextInImage'
      Principal: s3.amazonaws.com
      SourceAccount: !Ref 'AWS::AccountId'
  ResultsTable:
    Type: AWS::DynamoDB::Table
    Properties:
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: id
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST

I highly recommend you review the template for accuracy before you deploy it. You may also want to clean up little things such as in the template above, there are references to SAM in some tags and the template's description.

Package code with CloudFormation⌗

Just because SAM is no longer needed to deploy this app, doesn't mean the code can't be packaged up into S3 like sam package was doing. The AWS CLI command aws cloudformation package will do the same thing. The template has to be prepared to work with this command by pointing the template to the local path where the code resides. The package command will then zip the local code, push it to S3, and output a new template with the proper references to the bucket and newly-uploaded object (precisely how SAM did).

Open cft.yaml and for each Lambda function, find the Code stanza and delete the S3Bucket and S3Key properties. Set the Code property to the local file system path where the code is. For example, for the YAML template above, you would find this Lambda function:

  DetectTextInImage:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket: BUCKET_NAME
        S3Key: 915a69e82ff6d649e804e681769cf9b3

You would change the Code property to this:

  DetectTextInImage:
    Type: AWS::Lambda::Function
    Properties:
      Code: local_directory/

Where local_directory/ is a relative path to the directory that contains the Lambda code.

You now call cloudformation package to package the Lambda, push it to S3, and output a new template file.

aws cloudformation package --template-file cft.yaml \
  --output-template-file deploy.yaml \
  --s3-bucket <BUCKET_NAME>

You end up with two template files from here on out:

  • cft.yaml - This is the working copy of the template. Development of the app should happen on this template and it should be checked into version control.
  • deploy.yaml - This is used to deploy the app. Once a deployment is done, this file can be deleted. It can always be created again with the cloudformation deploy command.

Whenever changes are made to the Lambda code or cft.yaml, cloudformation package must be run as part of deploying those changes.

Find more information⌗

If you work a lot with CloudFormation, check out Rain and these other CloudFormation tools which make working with CFTs almost fun!

Disclaimer: The opinions and information expressed in this blog article are my own and not necessarily those of Amazon Web Services or Amazon, Inc.