How we migrated our Rails 6 app from AWS RDS to Planetscale

We help thousands of companies manage customer feedback. As featureOS grows to accommodate the incoming traffic, we knew we had to rethink our infrastructure. We came at this from two directions, like a pincer attack! One, we had to have reliable infrastructure that could scale without regular human interventions and two, being bootstrapped, it had to be affordable. The first leg of this transformation was to realign our database infrastructure to this new vision.

When we built the first version of featureOS we were strictly adhering to Paul Graham’s “Do Things that Don’t Scale” philosophy. Our Rails server, database and a couple other tools were all sitting in just one EC2 server. While this worked great we soon hit a point where we had to move out our database to a managed service. We chose Amazon’s RDS, it made sense at the time since we were already in Amazon’s ecosystem and the learning curve was a gentle slope. This again worked great until it got too expensive and made us do too much work. You see, we’re a bunch of lazy devs. Just can’t get that out of our DNA!

This is when we decided to move to Planetscale. It’s developer friendly, new age tech and had sleek design (this is a vanity metric, but honestly Planetscale’s UI is a breath of fresh air compared to AWS). The one decision that tipped the balance was how hard it is to get RDS to autoscale. Yes, we could theoretically use Aurora clusters but the work required did not justify the large amount of time a small team like ours would have to expend. Planetscale uses Vitess clusters to autoscale with fantastic scale for connection limits and pooling. All at a really great cost.

It does seem too good to be true, and that’s what we thought as well. Planetscale does have one limitation. It forces you to restructure your deployment strategy. For most Rails deployments it’s as simple as creating migration files in your local machine and then pushing them to origin. Now in the server all you have to do is,

rails db:migrate

This ensures all migrations are up-to date with the production database. This strategy did not change from when we had a local database to when we moved to RDS. The only change required for the RDS migration was the config/database.yml update. Everything else stayed the same.

When you use Planetscale you are forced to use their database branching workflow. Basically you sign up for a Planetscale account, configure your database and you get a main branch. All the developers use a branch of the main to configure their local databases. Each branch gets its own config. The important thing to remember here is that when you create a branch only the schema gets copied over, not the data. This worked great in local development. All of us were essentially on the same “page”. (Though Rails more or less accomplishes the same thing.)

However it caused a hindrance when it came to deploy migrations to the production DB. Here the main branch is “protected”. No migrations can happen on it directly. The workflow is,

1. Create a branch off of main
2. Make your migrations
3. Raise a deploy request from the new branch to the main brach
4. Merge the deploy request

Exhausting. But we made it work!

The first thing we had to do was tick this box that copies over the schema_migrations table data (not just the schema) to every new branch that is created.

This is required else Rails will perform every single migration all over again. The data within the schema_migrations table allows Rails to keep in sync with the database schema.

After that we had to write a script that essentially automates the 4 steps of the workflow to enable us to just do something similar to rails db:migrate and not change too much of our deployment workflow.

For folks who use CD Planetscale provides a fantastic Github action to mitigate all of this drama for you. Unfortunately (fortunately) we don’t do CI/CD just yet, just CI. So we, like any sane developer, wrote a script. 🎉

Planetscale provides a CLI (we wish it was a gem, but that’s unmaintained right now) that lets you automate the entire workflow. Here’s the overview of the script.

1. Create a branch off of main

pscale branch create #{DBNAME} #{name}

When you do this Planetscale takes awhile to create a branch. So you need to poll until the branch is “ready”. You can get that information using,

pscale branch show #{DBNAME} #{name}

Which will output this,

NAME   PARENT BRANCH   PRODUCTION   READY   CREATED AT       UPDATED AT
 ------ --------------- ------------ ------- ---------------- ----------------
  tiki   main            No           No      23 seconds ago   23 seconds ago

2. Setup a password for the new database branch

When you create a branch in Planetscale’s web interface it will prompt you to generate a password and save it elsewhere. Planetscale does not keep a copy of the database passwords.

So now in CLI, that’s what we will have to imitate. First let’s check if any passwords have been created.

pscale password list #{DBNAME} #{branch}

=> No passwords exist in chiki/tiki.

Nope! So now we have to create a password. To do that we need to pass in a name (can be anything).

pscale password create #{DBNAME} #{branch} #{name}
=> Password wanda was successfully created in chiki/tiki.
Please save the values below as they will not be shown again

  NAME    BRANCH   USERNAME       ACCESS HOST URL                     ROLE    ROLE DESCRIPTION               PASSWORD
 ------- -------- -------------- ----------------------------------- ------- ------------------------------ -------------------------------------------------------
  wanda   tiki     87spreefxvpy   forn47n5lck0.us-east-4.psdb.cloud   admin   Can Read, Write & Administer   pscale_pw_6i3Y7v3lHHYEzVVhrIyekYkXLzFd9gQtQI1Nb-usfs0

3. Use the new branch credentials to perform the migration

Now that you have your new branch simply perform the migration but pass in your new branch’s credentials. Otherwise, Planetscale will attempt to run the migration on the main branch.

rake db:migrate PS_DATABASE={database} PS_DATABASE_HOST={host} PS_DATABASE_PASSWORD={password} PS_DATABASE_USERNAME={username}

4. Create a deploy request

Now you’ve only performed the migration on the new branch. You need to port these over to the main branch so that your production database is in sync.

You can raise a deploy request like this,

pscale deploy-request create #{DBNAME} #{branch}
=> Deploy request #1 successfully created.

View this deploy request in the browser: https://app.planetscale.com/hnxt/chiki/deploy-requests/1

Planetscale will now create the deploy request and perform validations in the background. You can only merge the request once those checks are over. You can check for the status of the deploy request using (where ID is the ID of the request),

pscale deploy-request show #{DBNAME} #{id}
=> ID             NUMBER   BRANCH   INTO BRANCH   APPROVED   STATE   DEPLOY STATE   DEPLOYABLE   QUEUED AT        STARTED AT       FINISHED AT   CREATED AT   UPDATED AT   CLOSED AT
 -------------- -------- -------- ------------- ---------- ------- -------------- ------------ ---------------- ---------------- ------------- ------------ ------------ -----------
  h2hv8w46svfj   1        tiki     main          No         open    ready          Yes          31 seconds ago   21 seconds ago

Once the deploy request is ready, merge it.

pscale deploy-request deploy #{DBNAME} #{id}
=> Successfully queued h2hv8w46svfj from tiki for deployment to main.

That’s it! Your main branch now has the new migrations. You can continue deploying your code to production as you normally would.

5. Delete the merged branch

Just like in the kitchen, you need to clean as you cook. So don’t forget to delete the branch you recently created.

pscale branch delete #{DBNAME} #{name} --force

But before you do this you will have to ensure that the branch has finished getting deployed. You can use the same CLI command we used in step 1 to check on the branch status.

We’ve now been on Planetscale for just over a week and we absolutely love the performance and the insights it’s been giving us. Not to mention the huge load of work it did for us! We ❤️ you, Planetscale!