← Home

Now or Never: Upgrading a payroll monolith (from Ruby 2.7 to Ruby 4)

I work on a fairly large payroll platform — the kind of app where the domain model has to know about tax brackets, collective bargaining agreements, and the fact that "March" means something completely different depending on which country's fiscal calendar you're dealing with. It's a Rails monolith that's been running in production for years, and until recently it was sitting on Ruby 2.7 and Rails 6.1. Comfortable, battle-tested, and quietly turning into a liability nobody wanted to touch.

This is the writeup of how we took it from Ruby 2.7.8 / Rails 6.1.7 to Ruby 4.0.3 and Rails 8.1.2. It took longer than I expected going in, mostly because a payroll codebase is not a place where "probably fine" is an acceptable standard of evidence.

Where we actually started

Some numbers: north of 600 models, over 600 controllers, close to 1,700 specs. Heavy Sidekiq usage for background processing — payroll runs, bulk imports, report generation, anything that can't block a web request. A handful of gems doing real structural work: acts_as_paranoid for soft deletes, audited for the audit trail (which, in payroll, isn't a nice-to-have — someone will eventually ask "who changed this person's salary and when," and "I don't know" is not an answer you get to give), discard, draper, blueprinter.

One detail that tells you everything about how these upgrades get postponed: we were running Rails 6.1, but config.load_defaults in application.rb was still pinned at 6.0. Nobody had gone back and done the boring, deprecation-by-deprecation work of bumping it. That's what happens on a monolith with a lot of surface area and zero appetite for "let's touch every controller for no visible feature win" — it just piles up. By the time we started this project we already had two versions of framework-defaults debt sitting there before we'd written a single line of the actual upgrade.

The order of operations

We didn't jump straight to Ruby 4.0.3 + Rails 8.1 in one branch. That's how you end up with a PR nobody can review and a bug nobody can bisect. What actually worked:

  1. Bump Ruby only, keep Rails at 6.1, get CI green.
  2. Bump config.load_defaults to 6.1, deal with the fallout, ship it.
  3. Hop Rails one version at a time — 6.1 → 7.0 → 7.1 → 7.2 → 8.0 → 8.1 — bumping load_defaults and reading the framework defaults changelog at every single hop, not just the last one.

That last part matters more than it sounds like it should. It's tempting to grab the newest defaults file and diff it against what you had. We tried exactly that on a smaller service once and missed an intermediate default that existed for a single version before getting replaced by something else entirely. Going version by version is slower, and it doesn't matter — it's the difference between actually understanding what changed and just hoping your test suite would've caught it.

The Ruby jump: keyword arguments were the whole story

Going from 2.7 straight to 4.0 sounds like it should've been the scary part of this whole project. In practice, almost all of the actual pain traces back to one specific thing that happened years earlier, in 3.0: the separation of positional and keyword arguments. In 2.7 you could get away with sloppy method calls where a trailing hash got silently treated as keyword args. From 3.0 onward that ambiguity is just gone, and it doesn't always fail loud — sometimes a method just quietly receives the wrong arguments and does the wrong thing.

What this looks like in a real Rails app: a pile of internal service objects and query builders with signatures like

def initialize(params = {})

called from a dozen places with a mix of positional-looking hashes that used to work by accident. Fixing it is boring, mechanical work — the actual hard part is finding all of it, not fixing it once found. A few third-party gems needed bumping to versions that had already sorted this out; one or two smaller, barely-maintained gems needed a fork-and-patch because nobody upstream was going to do it for us.

The other thing worth flagging, carried over from the 3.x era: a handful of libraries that used to ship as part of the standard library (net-smtp, net-pop, a couple others) got carved out into default gems that eventually need an explicit Gemfile entry. Not dramatic, but it fails in the most annoying way possible — silently, and only on a code path nobody exercises day to day. For us that was an SMTP notification path. Of course it was.

Once the dust settled, Ruby 4.0 was genuinely nice to have. Hash shorthand syntax ({name:, age:}), pattern matching that's actually mature at this point, Data.define as a lighter Struct for simple value objects. Small stuff individually, but it adds up over a few months of code review.

The war story: the audit trail patch

We had a small monkey-patch sitting on top of the audited gem, written years ago to handle a payroll-specific requirement around how bulk changes get recorded. It had been stable for so long that basically nobody remembered it existed.

It broke during the Rails 7 hop, and it broke in the worst possible way for a payroll system: silently. No exception, no red test — the audit records for one specific bulk-import path just stopped capturing the "before" state correctly. We only caught it because a spec asserted on the actual content of an audit record instead of just checking that one got created. Thank god somebody bothered to write that assertion three years earlier instead of the lazy version.

Root cause: an internal method signature changed upstream in audited somewhere between the versions we skipped, and our patch was reaching into a private method that had quietly changed shape underneath it. The fix itself was tiny. The lesson wasn't: any monkey-patch on a third-party gem is debt that compounds silently across every major version you skip. It doesn't announce itself. It just sits there and waits for a spec you were smart enough to write a long time ago.

The Rails hops themselves

Zeitwerk-only autoloading was mostly a non-event — we were already on Zeitwerk in 6.1 — except for a handful of files leaning on classic-autoloader naming leniency that Rails 7 just stopped forgiving. Rename the file to match the constant, move on.

Asset pipeline churn was real but manageable. We were never on Webpacker, which saved us an entire category of pain other teams have written entire blog series about. Still had to work through the Sprockets/Propshaft transition for a couple of admin-only views nobody had touched in years and, frankly, nobody wanted to touch now either.

Rails 8's "Solid" trifecta (Solid Queue, Solid Cache, Solid Cable) is genuinely appealing, and we deliberately didn't touch it. Sidekiq Pro is doing real, boring, reliable work for us, and swapping the job backend in the same project as a Ruby/Rails major version bump is how you turn one risky change into two risky changes that are impossible to debug independently when something goes wrong at 2am. That's its own project, later, on its own terms — not something to bolt on because it happened to be the new default.

Same logic applied to deployment tooling. Rails 8's generators point you toward Kamal now. We already had a deploy pipeline that worked and nobody hated. Ripping it out because it wasn't the shiny new default would've been change for the sake of change, and "the framework has a new opinion" is not a reason to rewrite something that isn't broken.

Rails 8.1 itself was a smaller, calmer step than 8.0 — most of that generation's framework-defaults work had already been absorbed getting to 8.0. Mostly it meant clearing the last round of deprecation warnings and double-checking the Sidekiq/ActiveJob glue didn't care about anything new.

Testing and rollout

New Ruby versions change hash iteration and object-id behavior just enough to surface specs that were secretly relying on ordering they were never actually guaranteed. Found a handful of those. Good outcome, annoying week.

For rollout we leaned on feature flags for anything touching payroll calculation directly, and did a staged rollout by tenant instead of one global cutover — smaller blast radius if something in tax or benefits math behaved differently under the new Ruby's numeric or string handling, even though in practice it didn't. Watched error tracking closely for the first 48 hours after every Rails hop, not just the last one, because that's exactly the kind of thing people stop doing once the "big" version bump is behind them.

What I'd tell someone starting this

Do the Ruby jump on its own first. Don't conflate "new interpreter" bugs with "new framework" bugs — you want to know which one broke a given thing without guessing. Read every intermediate Rails version's framework defaults, not just the one you're aiming for. And audit every monkey-patch on a third-party gem before you start, not after it silently breaks something — if you can't explain in one sentence why the patch still needs to exist, that's your answer, and the patch loses.

None of this took cleverness. It took a few months of unglamorous, patient work from a team that also had a product to ship. That's most of what actually upgrading a monolith is, and anyone telling you otherwise is trying to sell you something.