As you might have recently read, we’re not very fond of taking risks at Harvest. But what about a major upgrade, like upgrading Rails?

The first version of Harvest was deployed on Rails 0.14 — so we’ve gone through a few Rails upgrades over the years. Our most recent upgrade was to Rails 4.0, and although it was a big change for us, we were able to break it down and deploy many parts of the update before the actual Rails version update went out.

Our actual deploy that upgraded Rails to 4.0 was very small — a few gem version updates and really minor code changes. Here’s the story of how we did it.

Break Everything

So, how did we start? Well, some things are obvious just by reading the official Rails Upgrade Guide. Other things you find by virtue of trying to boot your application and running the tests. Upgrading a Rails app is a lot like climbing a mountain: you keep moving upwards, fixing one thing after the other, solving obstacles until you reach the top.

Release Small Fixes

Many of the changes required were compatible with the previous version of Rails as well — so as we fixed issues on our Rails 4.0 branch, we were able to merge the changes back into our master branch and release them, keeping each release small and easy to understand.

• Strong Parameters. This is one of the bigger changes in Rails 4. Luckily the Rails team released strong_parameters, a gem that let us add this feature while still on Rails 3. Even better, when set up properly, it lets us convert slowly model by model, so the granularity of deploys can be really small.
• match routes requiring the verb. We had tons of routes that needed fixing, but again, this is something we could do beforehand, and in our particular case it was done in 16 different pull requests, all deployed safely and independently.
• Gem updates. One of the first things you have to do when updating a Rails version is update a bunch gems. Most of the newer gem versions (except the Rails ones themselves) worked just as well with Rails 3.
• Autoloading. We had a bunch of errors running our test suite — most of the changes required we reference classes or modules by their full name (External::Export instead of Export while you’re inside the External module). These are changes that can be merged and deployed at any time.
• Undigested assets. Rails 4 stopped generating undigested assets, so if you depended on them you had to do something. The recommended solutions were to make sure you reference digested versions by using one of the various rails helpers or just move those assets to /public. Again, a small, simple change that we released before the upgrade of Rails.

Besides that, there were 6 other pull requests with a variety of tiny tweaks in the way we did things that could be reimplemented in a way that worked both with Rails 3 and Rails 4. Every week, after getting a few more tests green on the rails-4 branch, I’d list the commits and see if there was something that could go back to master. You’d be surprised how much stuff can be backported.

Don’t drop the discipline of single meaningful commits. That will help you clearly see which commits can be backported and deployed to production right away. Imagine you usually work off master and you’re working on a rails-4 branch. You can very easily create a backport branch with:

git checkout master
git checkout -b rails-4-backmerge

# Repeat for every commit you think you can backport
git cherry-pick <sha> 
rake # run tests

Once you’re done and your backport is merged into master, make sure to merge master into rails-4 (or if you’re feeling adventurous, rebase rails-4 off of master).

Ignore Deprecation Warnings

We completely ignored deprecation warnings from Rails 4. They’re warnings for a reason — they don’t need to be dealt with immediately. Remember, our goal is to go live with as few changes as possible.

We dealt with all our deprecation warnings after the initial launch — over 18 pull requests in a few weeks, touching more than 1,700 lines of code.

Rehearse a Rollback Plan

With an update of a core gem like Rails, we couldn’t simply rely on our normal cap production deploy:rollback. While we were preparing for the release, we developed a rollback plan, and rehearsed the release and rollback on one of our test machines.

This turned out to be great practice, because the first time we attempted the Rails 4.0 release we discovered something wrong through our checkup plan and quickly rolled back the release without any issues.

Balance Risk Versus Effort

Some people have gone the extra mile and made their apps dual bootable. GitHub has a nice story about it and Discourse had it for a while. That would have given us great flexibility and let us slowly release this upgrade one server at the time.

We considered dual booting, but decided it wasn’t worth the effort for our application. Both for the extra complexity in actually implementing it, and the fact that a whole new set of problems arise from the fact that two different versions coexist at the same time led us to go with backports instead.

After merging most of our changes back into master we realized that what we needed to deploy was actually quite minimal. So we all agreed we’d try a deploy with a well-thought checkup plan and a rehearsed revert strategy instead.

Release With Confidence

Although upgrading to Rails 4.0 was a major change for us, we were able to break down the changes and release them slowly over the course of a few weeks. We merged as many changes back into our master branch (Rails 3) as possible, keeping the changes required for the Rails 4.0 release as small and simple as possible.

We developed a rollback and checkup plan, rehearsed them, and used them successfully — and followed up our release with a series of changes which removed our reliance on deprecated methods.

After following these techniques we were able to painlessly upgrade Harvest, a pretty large application, without scheduling a downtime or any disruptive customer outage. As a developer, that feels fantastic.