Dockerize a Rails App with MySQL and Sidekiq

Dockerize a Rails App with MySQL and Sidekiq

Dockerising an application is creating an isolated environment with containers for different moving parts in an application which makes the development and deployment easier and you don’t have to repeat yourself everytime when you need to deploy it locally for development or in production. It’s one command away and it’s that’s easy when you dockerize it. Dockerising is fun and tricky in the beginning when you define volumes for containers and working out an architecture that’ll suit your application. Okay, let’s just stop the talking and dive right into it. I’ll first start with the introduction to Docker and what you’ll need to dockerize an application. I take up a Rails App for this blog post but you can dockerize literally anything.

The prerequisites are to install Docker and make sure it’s running. Install Ruby & Rails. Don’t worry we’re not gonna deal with ruby.

”Hey, Docker. Who are you?”

I’m not gonna describe it to you. I’ll ask Docker to do that part. It’ll be so good if he does that by himself. So here he is.

“Hey! I’m a tool designed to make it easier to create, deploy, and run applications by using containers. A container is a part of me. A container is a standard unit of software that packages up the code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings. By doing so, thanks to my container, you can rest assured that the application will run on any other Linux machine regardless of any customized settings that machine might have that could differ from the machine used for writing and testing the code. The important thing to note is that I’m Open source and you can contribute to making me better. See how awesome I’m! :D”

So, how was his introduction? Okay, Let’s move to the other aspects in detail. Let’s talk about Docker images and Containers.

Volumes, Docker Images & Containers

Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure of the host machine, volumes are completely managed by Docker.

Docker containers are based on Docker images. A Docker image is a binary that includes all of the requirements for running a single Docker container, as well as metadata describing its needs and capabilities. You can think of it as a packaging technology. Docker containers only have access to resources defined in the image, unless you give the container additional access when creating it.

Dockerfile and docker-compose.yml

A Dockerfile is used to build an image. A Compose file is used to deploy a container from an image.

If you’re using a public image, such as Nginx or MySQL, then there’s no need for a Dockerfile since you don’t need to build it yourself - it’s already built and accessible via Docker Hub, so your Compose file can just pull it from there.

You usually don’t need a Dockerfile unless you’re creating a custom container image from scratch, or customizing a public image for some reason.

So, we’ll need Dockerfile to build our custom images for the web-app, sidekiq for our ruby application. Since we have pre-built images for MySQL, Ruby, and Redis, we’ll just pull the image from Docker Hub. We’ll talk in detail about everything.

We’re half-way through and let’s get started in dockerizing a rails application.

rails new app_name

As we discussed earlier, we’ll work on a rails application. Let’s create a new application with the command rails new rails-mysql-docker where rails-mysql-docker is the name of the project. This will create a scaffold and once it’s complete, open the folder in a code editor. Now, let’s get the basics done. We’ll install the MySQL gem as we’ll be using MySQL as the database and sidekiq for background processing. At the later point of the blog, we’ll talk about sidekiq. Now, add the required gems to the Gemfile and run bundle install in the command line.

Now, I assume that you’ve installed Docker on your computer and made sure that it’s up and running. We now created a rails app and installed mysql gem in it.

”It’s Docker Time!!”

We’ll now be writing the Dockerfile and the docker-compose.yml file. As I said previously, A Dockerfile is used to build an image. A Compose file is used to deploy a container from an image. We’ll first write the Dockerfile. The Dockerfile contains commands that need to be executed while building the image. That may include system libraries and other stuff. Create a file named Dockerfile at the root of the project and add the following to it.

Terminal window
FROM ruby:2.5-alpine


RUN apk update && apk upgrade && apk add ruby ruby-json ruby-io-console ruby-bundler ruby-irb ruby-bigdecimal tzdata postgresql-dev && apk add nodejs && apk add curl-dev ruby-dev build-base libffi-dev && apk add build-base libxslt-dev libxml2-dev ruby-rdoc mysql-dev sqlite-dev


RUN mkdir /app
WORKDIR /app


COPY Gemfile Gemfile.lock ./
RUN gem install ovirt-engine-sdk -v '4.3.0' --source 'https://rubygems.org/'
RUN bundle install --binstubs


COPY . .


EXPOSE 3000


ENTRYPOINT ["sh", "./config/docker/startup.sh"]

The above Dockerfile contains some commands that are run to build a Docker image. We’ll now break out the above docker file.

Terminal window
FROM ruby:2.5-alpine

We’re asking it to use the base image ruby for the 2.5-alpine distribution.

Terminal window
RUN apk update && apk upgrade && \
apk add ruby ruby-json ruby-io-console ruby-bundler ruby-irb ruby-bigdecimal tzdata && \
apk add nodejs && \
apk add curl-dev ruby-dev build-base libffi-dev && \
apk add build-base libxslt-dev libxml2-dev ruby-rdoc mysql-dev sqlite-dev

This will install the required system libraries. The above has some extra libraries which are not quite needed for a small rails application.

Terminal window
RUN mkdir /app
WORKDIR /app

We now run a mkdir command which is used to create a new directory. The WORKDIR line specifies a new default directory within the image’s file system which is the app directory.

Terminal window
COPY Gemfile Gemfile.lock ./


RUN bundle install --binstubs


COPY . .

Now, copy the Gemfile, lock file and the current directory and run bundle install. When you use COPY it will copy the files from the local source, in this case . meaning the files in the current directory, to the location defined by WORKDIR. In the above example, the second. refers to the current directory in the working directory within the image.

Terminal window
EXPOSE 3000


ENTRYPOINT ["sh", "./config/docker/startup.sh"]

This will export the port 3000 and run a startup.sh shell script. We use some shell scripts to build the application. We’ll talk about it more in some time.

An ENTRYPOINT allows you to configure a container that will run as an executable. In our case, it will run startup.sh that will build our app.

We also don’t want unwanted files inside a container. Take git tracking for example. We don’t need it to run our app. Thus, it is not needed. So, we create a .dockerignore file that contains the following,

Terminal window
.dockerignore
.git
logs/
tmp/

docker-compose.yml

The Compose file is a YAML file which defines services, networks, and volumes. It usually helps us defining the containers and, volumes with ENV variables. We’ll break out every container.

Terminal window
version: "3.7"
services:
 db:
   image: "mysql:5.7"
   environment:
     MYSQL_ROOT_PASSWORD: root
     MYSQL_USERNAME: root
     MYSQL_PASSWORD: root
   ports:
      - "3307:3306"
 redis:
   image: "redis:4.0-alpine"
   command: redis-server
   volumes:
      - "redis:/data"


 website:
   depends_on:
      - "db"
      - "redis"
   build: .
   ports:
      - "3000:3000"
   environment:
     DB_USERNAME: root
     DB_PASSWORD: root
     DB_DATABASE: sample
     DB_PORT: 3306
     DB_HOST: db
     RAILS_ENV: production
     RAILS_MAX_THREADS: 5
   volumes:
      - ".:/app"
      - "./config/docker/database.yml:/app/config/database.yml"


 sidekiq:
   depends_on:
      - "db"
      - "redis"
   build: .
   command: sidekiq -C config/sidekiq.yml
   volumes:
      - ".:/app"
   environment:
     REDIS_URL: redis://redis:6379/0


volumes:
 redis:
 db:

“Breaking it up is better than breakup”. Ok, I get it. It’s so dumb. Now we’ll break out our docker-compose.yml.

In the docker-compose.yml file, the version depends on the docker release. There is a table in the docker documentation that maps different version with their respective docker releases. The services are what that makes up your application. It consists of all the services that will be built as different containers that act as the organs of your application. We’ll have about 4 services namely - db, redis, website, sidekiq.

Terminal window
db:
  image: "mysql:5.7"
  environment:
    MYSQL_ROOT_PASSWORD: root
    MYSQL_USERNAME: root
    MYSQL_PASSWORD: root
  ports:
    - "3307:3306"

This container is for installing our MySQL and running it as a container. We pull an already built image mysql:5.7 from docker hub. We also pass the required environment variables. Ports mentioned in docker-compose.yml will be shared among different services started by the docker-compose. Ports will be exposed to the host machine to a random port or a given port.

Terminal window
redis:
  image: "redis:4.0-alpine"
  command: redis-server
  volumes:
    - "redis:/data"

Same as MySql, we pull a redis image from docker hub. Compose includes the ability to attach volumes to any service that has persistent storage requirements. We’ll have volumes to contain our persistent data like the above. The command will run after the container is built. Docker will create the volume for you in the /var/lib/docker/volumes folder. This volume persists as long as you are not typing docker-compose down -v.

Terminal window
website:
  depends_on:
    - "db"
    - "redis"
  build: .
  ports:
    - "3000:3000"
  environment:
    DB_USERNAME: root
    DB_PASSWORD: root
    DB_DATABASE: sample
    DB_PORT: 3306
    DB_HOST: db
    RAILS_ENV: production
    RAILS_MAX_THREADS: 5
  volumes:
    - ".:/app"
    - "./config/docker/database.yml:/app/config/database.yml"

The website service is where our rails application resides. docker-compose up will start services in dependency order as defined with depends_on. In the following example, db and redis will be started before the website. The build key tells to build with the docker-compose.yml present in the current directory.

Terminal window
sidekiq:
  depends_on:
    - "db"
    - "redis"
  build: .
  command: sidekiq -C config/sidekiq.yml
  volumes:
    - ".:/app"
  environment:
    REDIS_URL: redis://redis:6379/0

Sidekiq is a simple, efficient background processing for Ruby. Sidekiq uses threads to handle many jobs in the same process simultaneously. It does not require Rails but will integrate tightly with Rails to make background processing dead simple. A rails app will definitely need redis and sidekiq for background processing.

Docker Utility Folder

We’ll have a docker utility folder inside config that’ll contain a database.yml which will be later mounted into the config folder that’ll be used by the application. We’ll have four shell scripts.

startup.sh

If you remember the Dockerfile (If not, you can scroll back though :p), we would’ve referred startup.sh in our ENTRYPOINT. Yes, you guessed it right. We’re nearly at the end of the tutorial.

#! /bin/sh


# Wait for DB services
sh ./config/docker/wait-for-services.sh


# Prepare DB (Migrate - If not? Create db & Migrate)
sh ./config/docker/prepare-db.sh


# Pre-comple app assets
sh ./config/docker/asset-pre-compile.sh


# Start Application
bundle exec puma -C config/puma.rb

This script will run three more scripts and starts our application. We’ll discuss that below.

wait-for-services.sh

This script polls and waits for the MySQL to be up and running with the help of the host and the port it is running.

#! /bin/sh


# Wait for MySQL
until nc -z -v -w30 $DB_HOST $DB_PORT; do
 echo 'Waiting for MySQL...'
 sleep 1
done
echo "MySQL is up and running!"

prepare-db.sh

This script handles the database migrations for the application. It’ll create the database and migrate if the migrations failed in the first try.

#! /bin/sh


# If the database exists, migrate. Otherwise setup (create and migrate)
bundle exec rake db:migrate 2>/dev/null || bundle exec rake db:create db:migrate
echo "Done!"

asset-pre-compile.sh

We use rake assets:precompile to precompile our assets before pushing code to production. This command precompiles assets and places them under the public/assets directory in our Rails application.

#! /bin/sh


# Precompile assets for production
bundle exec rake assets:precompile


echo "Assets Pre-compiled!"

database.yml

As said before we’ll have the database configurations in our docker folder which will then be mounted into the config folder that’ll be then used by the rails application.

Terminal window
default: &default
 adapter: mysql2
 encoding: utf8mb4
 collation: utf8mb4_bin
 reconnect: false
 pool: 50
 username: <%= ENV['DB_USERNAME'] %>
 password: <%= ENV['DB_PASSWORD'] %>
 port: <%= ENV['DB_PORT'] %>
 host: <%= ENV['DB_HOST'] %>
 socket: /var/run/mysqld/mysqlx.sock


development:
 <<: *default
 database: <%= ENV['DB_DATABASE'] %>_development


production:
 <<: *default
 database: <%= ENV['DB_DATABASE'] %>

#ItsDone

Now, all we need to do is run docker-compose up -d. The -d is Detached mode which will run the containers in the background. The app will be built with all the containers and you can view the containers with docker ps -a. You can also find the GitHub Repo or follow me on Twitter for some retweet spam. In my next article, we’ll talk about CI/CD with Jenkins . Until next time, have this Gif for free.