Deploying Phoenix with CircleCI

Deploying Phoenix with CircleCI

CircleCI is a continuous integration service that also works for continuous delivery.

Let's look at a simple CircleCI configuration file used to build, test, and deploy a simple Phoenix-based application.


Prerequisites

For the sake of simplicity, assume that we have provisioned the target server. We built this under the assumption that the target server is Ubuntu 16.04 LTS based, with Elixir 1.9 and OTP 22. There is nothing particularly specific to these, so it should be simple to modify this tutorial to suit other versions.

Also, assume that you already have followed the Phoenix getting started guide.


Getting Started

By the end of this article, we will have a simple Phoenix application that will run tests using CircleCI, then can be deployed by pushing a Git tag to your repository.

Start out by creating your new project:

mix phx.new hello_circle
cd hello_circle
git init .

Creating an OTP Release

Elixir 1.9 introduced official support for OTP releases. The first thing we need to do is configure our project for this.

Open the mix.exs file in the project root and update the project/0 function with a new releases key:

def project do
  [
    app: :hello_circle,
    version: "0.1.0",
    elixir: "~> 1.9",
    elixirc_paths: elixirc_paths(Mix.env()),
    compilers: [:phoenix, :gettext] ++ Mix.compilers(),
    start_permanent: Mix.env() == :prod,
    aliases: aliases(),
    deps: deps(),
    releases: releases()
  ]
end
Add the releases key

and create that new function:

defp releases do
  [
    hello_circle: [
      include_executables_for: [:unix],
      applications: [runtime_tools: :permanent]
    ]
  ]
end
Configure the release

This function returns a list of releases with their configurations. We need to create a single release called hello_circle for this guide. (You can name your release whatever you want, but the rest of this article assumes you named the release hello_circle).


The release contains the following options:

include_executables_for: [:unix]

Which tells the release task to include ERTS in the release. That option allows your target server to not require Erlang or Elixir installed.

applications: [runtime_tools: :permanent]

The applications option tells the release task to include runtime_tools in the release.

There are other configuration entries for the release, but these are all the requirements we need for now.

You can now run:

mix release

then

_build/dev/rel/hello_circle/bin/hello_circle start_iex

to  ensure your release boots.

Building the release with Docker and CircleCI

Now, how do we deploy this release to our server? Building the release manually and uploading it is one option, but this can become tedious when we start encountering issues related to differences between your development machine and the target server. Enter CircleCI and Docker. CircleCI doesn't require using Docker, but Docker allows much faster builds and deployment pipelines with simpler crafting than other platforms.

Let's go over section by section, creating a configuration file:

First, create a new file in your project

touch .circleci/config.yml

The configuration sets up 2 jobs:

  • build - The default job name. This job must exist. The build is the perfect place to run tests.
  • deploy - The job that deploys our new codebase to our server.

Open the configuration file in your editor .circleci/config.yml and add the following data to it:

The test job

version: 2
jobs:
  build:
    docker:
      - image: binarynoggin/elixir-release-builder:latest
        environment:
          MIX_ENV: test 
          POSTGRES_USER: ubuntu
          POSTGRES_DB: hello_circle_test

      - image: circleci/postgres:9.6.2-alpine
        environment:
          POSTGRES_USER: ubuntu
          POSTGRES_DB: hello_circle_test
    working_directory: ~/app
    steps:
      - checkout
      - run: echo ${MIX_ENV} >> MIX_ENV
      - restore_cache:
          keys:
            - v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
      - restore_cache:
          keys:
            - v0-npm-dependency-cache-{{ checksum "assets/package-lock.json" }}
      - run: psql -U ${POSTGRES_USER} ${POSTGRES_DB} -h localhost < priv/repo/structure.sql
      - run: npm install --prefix assets
      - run: mix deps.get
      - run: mix test
      - save_cache:
          key: v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
          paths:
            - deps
            - _build
      - save_cache:
          key: v0-npm-dependency-cache-{{ checksum "assets/package-lock.json" }}
          paths:
            - assets/node_modules

Now we'll go over each part and explain what it does:

version: 2

version tells CircleCI what version of the runner we need. This document explains the syntax of v2.

jobs:
  build:

jobs contains a list of jobs to run on CircleCI. build is a
required job name, but remember we use this job to run our tests.

    docker:
      - image: binarynoggin/elixir-release-builder:latest
        environment:
          MIX_ENV: test 
          POSTGRES_USER: ubuntu
          POSTGRES_DB: hello_circle_test

      - image: circleci/postgres:9.6.2-alpine
        environment:
          POSTGRES_USER: ubuntu
          POSTGRES_DB: hello_circle_test

The docker configuration option tells CircleCI how to setup our build environment. This job will use two images:

  • binarynoggin/elixir-release-builder:latest
    This image contains all the necessary packages to build and test our application
  • circleci/postgres:9.6.2-alpine
    This image starts an instance of postgres that our app to complete database tests
    steps:
      - checkout
      - run: echo ${MIX_ENV} >> MIX_ENV
      - restore_cache:
          keys:
            - v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
      - restore_cache:
          keys:
            - v0-npm-dependency-cache-{{ checksum "assets/package-lock.json" }}
      - run: npm install --prefix assets
      - run: mix deps.get
      - run: mix ecto.setup
      - run: mix test
      - save_cache:
          key: v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
          paths:
            - deps
            - _build
      - save_cache:
          key: v0-npm-dependency-cache-{{ checksum "assets/package-lock.json" }}
          paths:
            - assets/node_modules

The CircleCI job runner runs each step in order.
First, we checkout the code. We use git to clone the repo and the commit that is being tested/deployed.

Next, echo ${MIX_ENV} >> MIX_ENV writes the current
Elixir Mix environment to a file on the filesystem. We use the MIX_ENV to configure the caches to use the correct environment and be stored separately from each other.

Next, a couple of restore_cache steps configure where to get caches that were saved with save_cache. No cache exists on the first run, but having it drastically speeds up future builds.

Then, we fetch dependencies with NPM and Mix:

npm install --prefix assets
mix deps.get

Before we stop, we set up the database and run tests:

mix ecto.setup
mix test

Finally, we save all the dependencies with save_cache regardless of our previous lines passing or failing.

The deploy job

Next, we need to add the deploy job. This job is responsible for building and uploading our new release to the remote server. Add the following under the build job:

  deploy:
    docker:
      - image: binarynoggin/elixir-release-builder:latest
        environment:
          MIX_ENV: prod 
          DEPLOYMENT_NAME: hello_circle
          DEPLOY_USER: some_user
          DEPLOY_HOST: SOME_URL.com
    working_directory: ~/app

    steps:
      - checkout
      - run: echo ${MIX_ENV} >> MIX_ENV
      - add_ssh_keys:
          fingerprints:
            - "SO:ME:FIN:G:ER:PR:IN:T"
      - restore_cache:
          keys:
            - v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
      - restore_cache:
          keys:
            - v0-npm-dependency-cache-{{ checksum "assets/package-lock.json" }}
      - run: mix deps.get
      - run: npm install --prefix assets
      - run: npm run deploy --prefix assets
      - run: mix phx.digest
      - run: mix release
      - run: tar -zcvf ${DEPLOYMENT_NAME}-${MIX_ENV}.tar _build/${MIX_ENV}/rel/${DEPLOYMENT_NAME}
      - run: 
          name: Upload release
          command: |
            scp -o "StrictHostKeyChecking no" \
            ${DEPLOYMENT_NAME}-${MIX_ENV}.tar \
            ${DEPLOY_USER}@${DEPLOY_HOST}:/home/${DEPLOYMENT_NAME}/${DEPLOYMENT_NAME}-${MIX_ENV}.tar
      - run:
          name: Untar release
          command: |
            ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_HOST} \
            "tar xf ${DEPLOYMENT_NAME}-${MIX_ENV}.tar"
      - run:
          name: Stop running instance
          command: |
            ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_HOST} \
            "_build/prod/rel/${DEPLOYMENT_NAME}/bin/${DEPLOYMENT_NAME} stop; echo \"Ignore if this command fails\""
      - run:
          name: Start a new instance
          command: |
            ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_HOST} \
            "bash --login -c \"_build/prod/rel/${DEPLOYMENT_NAME}/bin/${DEPLOYMENT_NAME} daemon\""
      - run:
          name: Check the new instance is running
          command: |
            ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_HOST} \
            "_build/prod/rel/${DEPLOYMENT_NAME}/bin/${DEPLOYMENT_NAME} pid"
      - save_cache:
          key: v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
          paths:
            - deps
            - _build
      - save_cache:
          key: v0-npm-dependency-cache-{{ checksum "assets/package-lock.json" }}
          paths:
            - assets/node_modules

Let's go over what exactly this does:

First, we checkout and write the MIX_ENV file again just like the build job.

Next, we have a new command:

      - add_ssh_keys:
          fingerprints:
            - "SO:ME:FIN:G:ER:PR:IN:T"

This command tells the CircleCI job runner to use a particular ssh key. We previously authorized this key on our server.
See the official docs for adding ssh keys.

Next, we do the same setup as the build task, install dependencies from NPM and Hex.

After that, we package up our web assets with:

npm run deploy --prefix assets
mix phx.digest

Finally, we get to the good stuff:

mix release
tar -zcvf ${DEPLOYMENT_NAME}-${MIX_ENV}.tar _build/${MIX_ENV}/rel/${DEPLOYMENT_NAME}

The mix release command creates an OTP release with everything needed to serve our web app.  We then zip it up, and it is ready to upload to your server. The next few commands use SSH to upload, unpack, and execute our new release:

# upload the tar file to the server. 
# You can put this wherever you want on the server.
scp -o "StrictHostKeyChecking no" ${DEPLOYMENT_NAME}-${MIX_ENV}.tar ${DEPLOY_USER}@${DEPLOY_HOST}:/home/${DEPLOYMENT_NAME}/${DEPLOYMENT_NAME}-${MIX_ENV}.tar

# Unpack the tar file.
ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_HOST} "tar xf ${DEPLOYMENT_NAME}-${MIX_ENV}.tar"

# stop an existing instance. 
# (this should be ignored if no instance is running)
ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_HOST} "_build/prod/rel/${DEPLOYMENT_NAME}/bin/${DEPLOYMENT_NAME} stop"

# start a new instance. 
ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_HOST} "bash--login -c \"_build/prod/rel/${DEPLOYMENT_NAME}/bin/${DEPLOYMENT_NAME} daemon\""

Everything else in the deploy job sets up caches to speed up future builds.

Deployment Workflow

Next we configure CircleCI's job runner to run our jobs based on specific events.

Add the final new section to .circleci/config.yml:

workflows:
  version: 2
  deploy_production:
    jobs:
      - build
      - deploy:
          filters:
            tags:
              only: /.*/
            branches:
              ignore: /.*/
          requires:
            - build

This simple configuration tells CircleCI how to order our jobs. We name our workflow deploy_production. It contains two jobs:

  • build
    Remember, this is the default name. We use this step to test our code
  • deploy
    This is our job that uploads packages and uploads new releases

build uses the default settings, which run for every commit. However, we only want deploy to run when we push a git tag, and it build (our tests) completes before running. Requiring build prevents code with failing tests from being deployed.

Conclusions

This configuration is just a jumping-off point. Every application will be unique, but this provides a simple starting point for future modification.

Here is the final circle ci config:

version: 2

jobs:
  deploy:
    docker:
      - image: binarynoggin/elixir-release-builder:latest
        environment:
          MIX_ENV: prod 
          DEPLOYMENT_NAME: hello_circle
          DEPLOYMENT_USER: some_linux_user
          DEPLOYMENT_HOST: some-server-url
    working_directory: ~/app

    steps:
      - add_ssh_keys:
          fingerprints:
            - "SO:ME:FIN:G:ER:PR:IN:T"
      - checkout
      - run: echo ${MIX_ENV} >> MIX_ENV
      - restore_cache:
          keys:
            - v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
      - restore_cache:
          keys:
            - v0-npm-dependency-cache-{{ checksum "assets/package.json" }}
      - run: mix deps.get
      - run: npm install --prefix assets
      - run: npm run deploy --prefix assets
      - run: mix phx.digest
      - run: mix release
      - run: tar -zcvf ${DEPLOYMENT_NAME}-${MIX_ENV}.tar _build/${MIX_ENV}/rel/${DEPLOYMENT_NAME}
      - run: 
          name: Upload release
          command: |
            scp -o "StrictHostKeyChecking no" \
            ${DEPLOYMENT_NAME}-${MIX_ENV}.tar \
            ${DEPLOY_USER}@${DEPLOY_URL}:/home/${DEPLOYMENT_NAME}/${DEPLOYMENT_NAME}-${MIX_ENV}-${CIRCLE_TAG}.tar
      - run:
          name: Untar release
          command: |
            ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_URL} \
            "tar xf invoyer-${MIX_ENV}.tar"
      - run:
          name: Stop running instance
          command: |
            ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_URL} \
            "_build/prod/rel/${DEPLOYMENT_NAME}/bin/${DEPLOYMENT_NAME} stop; echo \"Ignore if this command fails\""
      - run:
          name: Start a new instance
          command: |
            ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_URL} \
            "bash --login -c \"_build/prod/rel/${DEPLOYMENT_NAME}/bin/${DEPLOYMENT_NAME} daemon\""
      - run:
          name: Check the new instance is running
          command: |
            ssh -o "StrictHostKeyChecking no" ${DEPLOY_USER}@${DEPLOY_URL} \
            "_build/prod/rel/${DEPLOYMENT_NAME}/bin/${DEPLOYMENT_NAME} version"
      - save_cache:
          key: v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
          paths:
            - deps
            - _build
      - save_cache:
          key: v0-npm-dependency-cache-{{ checksum "assets/package.json" }}
          paths:
            - assets/node_modules

  build:
    docker:
      - image: binarynoggin/elixir-release-builder:latest
        environment:
          MIX_ENV: test 
          POSTGRES_USER: ubuntu
          POSTGRES_DB: invoyer_test

      - image: circleci/postgres:9.6.2-alpine
        environment:
          POSTGRES_USER: ubuntu
          POSTGRES_DB: invoyer_test
    working_directory: ~/app
    steps:
      - checkout
      - run: echo ${MIX_ENV} >> MIX_ENV
      - restore_cache:
          keys:
            - v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
      - restore_cache:
          keys:
            - v0-npm-dependency-cache-{{ checksum "assets/package.json" }}
      - run: psql -U ${POSTGRES_USER} ${POSTGRES_DB} -h localhost < priv/repo/structure.sql
      - run: npm install --prefix assets
      - run: mix deps.get
      - run: mix test
      - save_cache:
          key: v0-mix-dependency-cache-{{ checksum "mix.lock" }}-{{ checksum "MIX_ENV" }}
          paths:
            - deps
            - _build
      - save_cache:
          key: v0-npm-dependency-cache-{{ checksum "assets/package.json" }}
          paths:
            - assets/node_modules

workflows:
  version: 2
  test:
    jobs:
      - build:
          filters:
            tags:
              ignore: /.*/
  deploy_production:
    jobs:
      - deploy:
          filters:
            tags:
              only: /^v.*/
            branches:
              ignore: /.*/

Try this out on your project. We have a docker file all ready for you, here.

Tags :