John Vincent

Configure Travis-CI for Docker Multi-Container Deployment to AWS

Configure Travis-CI for Docker Multi-Container Deployment to AWS

by John Vincent

Posted on June 1, 2021

Travis-CI is used to build and deploy the application to production at AWS.

This is part of a series of discussions regarding Deploying TaskMuncher, a React and Node application, to a Multi-Container Docker Environment at AWS using Dockerhub and Travis CI

For more details, please see Overview of Create Multi-Container Docker TaskMuncher Application at AWS

Please review AWS Production Architecture.

Overview

The responsibility of .travis.yml is to build, test and save the Docker images to Dockerhub, and then to notify AWS to perform a redeploy to Elastic Beanstalk.

There are some following tasks that require information from tasks that have not yet been performed. Links to documents describing these tasks are provided. It is understood this document requires of the reader a leap of faith.

Please note that the building of custom images during deployment with a Dockerfile is not supported by the multi-container Docker platform on Elastic Beanstalk.

Thus it is necessary to build the images and deploy them to an online repository, in this case Dockerhub, before deploying to an Elastic Beanstalk environment.

Configure Dockerhub for Private Repositories

Login to Dockerhub or create an account.

  • Click username, click Account Settings from the dropdown.
  • From left menu, click Default Privacy.
  • Select Private

This ensures that my Dockerhub repositories will be private.

Provide Github access for Travis-CI

Notice Travis CI

  • Select Configure Travis CI

Notice Repository access

  • Select Only select repositories
  • Select Repositories
  • Add repository taskmuncher
  • Save

Add AWS Keys to Travis-CI

Go to your Travis Dashboard and find the taskmuncher repository.

  • On the repository page, click More Options and then Settings
  • Go down to the Environment variables section
  • Create an AWS_ACCESS_KEY variable and paste your IAM access key id
  • Create an AWS_SECRET_KEY variable and paste your IAM secret access key

To keep them secret, ensure Display value in build log is disabled.

Add Dockerhub Id to Travis-CI

Go to your Travis Dashboard and find the taskmuncher repository.

  • On the repository page, click More Options and then Settings
  • Go down to the Environment variables section
  • Create a DOCKER_ID variable and paste your Docker login id
  • Create a DOCKER_PASSWORD variable and paste your Docker password

To keep them secret, ensure Display value in build log is disabled.

Build Travis yaml

This requires another large leap of faith but if you have been following the documents in order you will at least understand the deploy section as it refers to deploying to the AWS environment that has already been created.

For more details, see

Create file .travis.yml is base folder of the repository.

language: generic
sudo: required

branches:
  only:
  - master

services:
  - docker

before_install:
  - docker build -t johnvincentio/taskmuncher-client -f ./config/client/dev/Dockerfile .
  - docker build -t johnvincentio/taskmuncher-server -f ./config/server/dev/Dockerfile .

script:
  - docker run -e CI=true johnvincentio/taskmuncher-client npm run test -- --coverage
  - docker run -e CI=true johnvincentio/taskmuncher-server npm run test -- --coverage

after_success:
  - docker build -t johnvincentio/taskmuncher:client -f ./config/client/aws/Dockerfile .
  - docker build -t johnvincentio/taskmuncher:server -f ./config/server/aws/Dockerfile .
  - docker build -t johnvincentio/taskmuncher:nginx -f ./config/nginx/Dockerfile .
  - docker build -t johnvincentio/taskmuncher:mongodb -f ./config/mongodb/Dockerfile .
  - docker build -t johnvincentio/taskmuncher:mongo-client -f ./config/mongo-client/Dockerfile .

  # Log in to the docker CLI
  - echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin

  # Take those images and push them to docker hub
  - docker push johnvincentio/taskmuncher:client
  - docker push johnvincentio/taskmuncher:server
  - docker push johnvincentio/taskmuncher:nginx
  - docker push johnvincentio/taskmuncher:mongodb
  - docker push johnvincentio/taskmuncher:mongo-client

deploy:
  provider: elasticbeanstalk
  region: us-east-1
  app: taskmuncher-docker
  env: Taskmuncherdocker-env
  bucket_name: elasticbeanstalk-us-east-1-971716655829
  bucket_path: taskmuncher-docker
  on:
    branch: master
  access_key_id: $AWS_ACCESS_KEY
  secret_access_key: $AWS_SECRET_KEY

  skip_cleanup: true
  edge: true

  • sudo is required

  • services lists docker as the platform that will be used.

  • branches - only rebuild on a push to the master branch.

  • script lists unit tests that must run without error for after_success to be executed.

  • before_install lists the builds that are required for the script section.

  • after_success lists the tasks to be performed if the unit tests succeed.

    • build the docker images
    • login to Dockerhub
    • push the docker images to Dockerhub

  • johnvincentio/taskmuncher is the private Dockerhub repository

  • Set the region. The region code can be found by clicking the region in the toolbar next to your username us-east-1

  • app should be set to the EBS Application Name taskmuncher-docker

  • env should be set to your EBS Environment name Taskmuncherdocker-env

  • Set the bucket_name. This can be found by searching for the S3 Storage service. Click the link for the elastic beanstalk bucket that matches your region code and copy the name elasticbeanstalk-us-east-1-971716655829

Set the bucket_path.

  • List S3 buckets

  • Select your S3 bucket.

  • Notice the folders.

    • When first created the application folder will not exist.
    • Set to the default.
    • Default folder is the name of the application taskmuncher-docker

  • Set the bucket_path to taskmuncher-docker

  • Set access_key_id to $AWS_ACCESS_KEY

  • Set secret_access_key to $AWS_SECRET_KEY

Workflow of Travis Based Builds

Let's discuss a few options

Perform a build on a successful pull request

Go to TaskMuncher Travis Settings

  • Build pushed branches: off
  • Limit concurrent jobs: off
  • Build pushed pull requests: on

Rebuilds will only be performed on a github pull request.

The following shows how that is done.

Create Feature Branch

  • Create your feature branch git checkout -b feature
  • Commit your changes git commit -am 'Add some feature'
  • Push to the branch git push origin feature

Edit on Feature Branch

  • make changes to feature branch
  • git status to show the changes
  • git add .
  • git commit -m "feature changed"
  • git push origin feature

Create Pull Request

From the Github repository

  • Pull Requests

Should now see feature had recent pushes less than a minute ago.

  • Click Compare and pull request

which opens a pull request.

  • Add a comment
  • Click Create pull request

Notice some pending checks.

Go to Travis dashboard, notice the project is being rebuilt.

Eventually the build of the pull request is successful.

The github pull request:

  • will now show all checks have passed.
  • Add a comment to the pull request.
  • Click Comment
  • Click Merge pull request
    • Click Confirm merge

Should now see Pull request successfully merged and closed

Go to Dockerhub

  • select Repositories
  • select johnvincentio/taskmuncher
  • notice the 5 docker images have been created and stored as tags.

Notice in Travis-CI log

Skipping a deployment with the elastic beanstalk provider because the current build is a pull request.

which is obviously a problem when want to deploy to AWS.

Perform a build on a push to a branch

Go to TaskMuncher Travis Settings

  • Build pushed branches: on
  • Limit concurrent jobs: off
  • Build pushed pull requests: off

Rebuilds will only be performed on a github push.

Note that builds will be run on branches that are not explicitly excluded in your .travis.yml.

The following shows how that is done.

Create Feature Branch

  • Create your feature branch git checkout -b feature
  • Commit your changes git commit -am 'Add some feature'
  • Push to the branch git push origin feature

Edit on Feature Branch

  • make changes to feature branch
  • git status to show the changes
  • git add .
  • git commit -m "feature changed"
  • git push origin feature

Notice a Travis-CI rebuild has NOT been triggered.

Merge to master

git checkout master
git merge feature
git push origin master

Go to your Travis dashboard and notice a rebuild has now been triggered.

Check the status of the build. It should eventually return with a green check mark and show "build passing"

Check AWS Rebuild

  • Go to AWS and use Find Services to search for Elastic Beanstalk

  • Select Environments

  • Select Taskmuncherdocker-env

  • It should say "Elastic Beanstalk is updating your environment".

  • Recent events should details a recent update.

  • It should eventually show a green check mark under "Health". You will now be able to access your application at the external URL provided under the environment name.