Deploy to multiple environments with git and CircleCI

Easily deploying to multiple environments in a simple manner using GitHub, CircleCI and Heroku.

Continuous Integration is awesome, but sometimes you need a buffer between auto-deploying things on merge and the production release. To do that with CircleCI requires some git branch-wrangling and a few lines of bash scripting. We’ll imagine a scenario where a deploy is trivial (ie. we’ll pretend we’re using Heroku). For more complicated build steps we should still be able to follow similar principles. This is not a CircleCI 2.0 workflows tutorial, it’s more of a git-flow/CircleCI hybrid to have 2 (or more) environments being released to and automatically deployed by CircleCI.

I would like to thanks Chris Fidao, and this tweet:

We’ll go through how to use GitHub + CircleCI for deployment automation and release management.

This was sent out on the Code with Hugo newsletter last Monday. Subscribe to get the latest posts right in your inbox (before anyone else).

A branch setup 🌳

We’ll want a develop and a master branch that get auto-deployed. Our default branch should be develop (ie. all pull requests should get merged into. Thats as simple as running:

$ git checkout -b develop
$ git push -u origin develop
# There's usually already a master branch otherwise:
$ git checkout -b master
$ git push -u origin master

We’re using branches because that’s the only primitive that CircleCI understands. On TravisCI or GoCD you would be able to set up “pipelines” for each environment but CircleCI workflows can’t be triggered for different environments manually, so it’s easiest to use git branches.

The workflow 🏞

1. Create a feature/task branch
2. Complete the task, get the code in a state to be merged
3. Open a PR from the feature/task branch to develop
4. CircleCI runs tests/lint whatever else (not covered in this post)
5. Automated checks are all green ✅
6. Review
7. The PR is merged into develop
8. CircleCI runs automated checks again
9. CircleCI deploys to development/staging environment if all checks are green
10. To deploy to production, the release has to be manual
11. Merge develop into master
12. CircleCI runs automated checks again
13. CircleCI deploys to production environment if all checks are green

To make this process easier, we’ll have some release scripts to automate step 6 (merging correctly is easy to do wrong) and some CircleCI config to do steps 5a-b and 6b-c.

Release scripts 🛫

The following is release-production.sh, we can use it to merge changes from develop → master:

#!/bin/bash
set -e
set -u
RELEASE_FROM="develop"
RELEASE_TO="master"
CURRENT_BRANCH="`git branch | grep \* | cut -d ' ' -f2`"
echo "Checking out to '${RELEASE_FROM}' branch and pulling latest"
git checkout ${RELEASE_FROM} 
git pull
echo "Checking out to '${RELEASE_TO}' branch and pulling latest"
git checkout ${RELEASE_TO} 
git pull
read -p "Are you sure you want to merge '${RELEASE_FROM}' into '${RELEASE_TO}'? (y/n)" -n 1 -r
echo

if [[ $REPLY =~ ^[Yy]$ ]]
then
    git merge ${RELEASE_FROM} --ff-only
    git push
fi

git checkout ${CURRENT_BRANCH}

Here’s a breakdown of the steps of what it does:

• Save current branch name
• checkout to the branch we are releasing from (develop)
• pull latest
• checkout to the branch we are releasing to (master)
• pull latest
• prompt before merge
• merge
  • --ff-only, means we run all merges with “fast-forward” which means we won’t get a merge commit, this means there won’t be a merge commit
• prompt before release
• push
• reset to branch we were initially on

Logging in to Heroku (optional) 🔑

To store secrets we’ll use CircleCI environment variables setting, and set HEROKU_EMAIL and HEROKU_TOKEN through the UI (Settings → Build Settings → Environment Variables). To get your Heroku token run heroku auth:token. To log in to Heroku, use the following in login-heroku.sh:

cat > ~/.netrc << EOF
    machine api.heroku.com
        login $HEROKU_EMAIL
        password $HEROKU_TOKEN
    machine git.heroku.com
        login $HEROKU_EMAIL
        password $HEROKU_TOKEN
EOF

# Add heroku.com to the list of known hosts
mkdir ~/.ssh
ssh-keyscan -H heroku.com >> ~/.ssh/known_hosts

12(ish) factor app 🏗

We want to manage configuration somehow, for all the environments as described by https://12factor.net/

Injecting config and secrets 💉

setup-env.sh

• Switch on CIRCLE_BRANCH, set some variables conditionally (ENVIRONMENT, HEROKU_APP, others not (NODE_ENV):

case $CIRCLE_BRANCH in
    "develop")
        export ENVIRONMENT="dev"
        export HEROKU_APP="some-app"
        ;;
    "master")
        export ENVIRONMENT="production"
        export HEROKU_APP="some-other-app"
        ;;
esac
export NODE_ENV="production"

If we had to set some secrets around here, we would do something like the following:

case $CIRCLE_BRANCH in
    "develop")
        export MY_SECRET=${MY_SECRET_DEV}
        export HEROKU_APP="some-app"
        ;;
    "master")
        export MY_SECRET=${MY_SECRET_PRODUCTION}
        ;;
esac

Where MY_SECRET_DEV and MY_SECRET_PRODUCTION are set through CircleCI environment variables (Settings → Build Settings → Environment Variables).

Run that deploy 🛬

deploy-heroku.sh:

• Read setup from setup-env, add Heroku remote and push current branch to master on Heroku

set -e
set -u
source ./setup-env.sh
echo "Pushing branch ${CIRCLE_BRANCH} to app ${HEROKU_APP}"
git remote add heroku https://git.heroku.com/${HEROKU_APP}.git
git push heroku ${CIRCLE_BRANCH}:master

• To have some sort of record of what’s deployed and what’s not, we want to set the COMPARE_URL and version number (BUILD_NUM) on Heroku, that requires the Heroku CLI:

if [ ! -L /usr/local/bin/heroku ];
then
    wget https://cli-assets.heroku.com/branches/stable/heroku-linux-amd64.tar.gz
    sudo mkdir -p /usr/local/lib /usr/local/bin
    sudo tar -xvzf heroku-linux-amd64.tar.gz -C /usr/local/lib
    sudo ln -s /usr/local/lib/heroku/bin/heroku /usr/local/bin/heroku
fi
source infra/scripts/setup-env.sh
heroku config:set BUILD_NUM=${CIRCLE_BUILD_NUM} COMPARE_URL=${CIRCLE_COMPARE_URL} -a ${HEROKU_APP}

All together we end up with the following .circleci/config.yml:

version: 2
jobs:
    deploy:
    docker:
        - image: circleci/node:10.5.0 # replace with the image you need
    steps:
        - checkout
        - run:
            name: Log in to Heroku
            command: bash ./login-heroku.sh
        - run:
            name: Install Heroku CLI
            command: |
            wget https://cli-assets.heroku.com/branches/stable/heroku-linux-amd64.tar.gz
            sudo mkdir -p /usr/local/lib /usr/local/bin
            sudo tar -xvzf heroku-linux-amd64.tar.gz -C /usr/local/lib
            sudo ln -s /usr/local/lib/heroku/bin/heroku /usr/local/bin/heroku            
        - run:
            name: Deploy heroku app
            command: bash infra/deploy-heroku.sh
        - run:
            name: Set BUILD_NUM and COMPARE_URL on Heroku to CIRCLECI values
            command: |
            source ./setup-env.sh
            heroku config:set BUILD_NUM=${CIRCLE_BUILD_NUM} COMPARE_URL=${CIRCLE_COMPARE_URL} -a ${HEROKU_APP}            

workflows:
    version: 2
    ci:
    jobs:
        - deploy:
            filters:
            branches:
                only:
                - develop
                - master
            # You should probably be running
            # some checks before you deploy
            # requires:
            #  - test
            #  - lint

This isn’t an exhaustive description of how to set up your CI, but it’s a start.

Feel free to drop me a line at [email protected], or Twitter @hugo__df.

Oliver Roos