“Let’s just rebuild it from scratch” is the most expensive sentence in software — when to modernize instead, and the rare case for starting over

Every business that runs on custom software eventually arrives at the same meeting. The system is old. It is ugly inside. The person who built it left two jobs ago. Every small change takes too long, and someone — usually the newest, most confident engineer in the room — says the words: "Honestly, it would be faster to just rebuild it from scratch." Everyone nods, because it feels obviously true. The old thing is a mess; a clean start would be so much better. That sentence is one of the most expensive things anyone can say in a software business, and it is almost always wrong.

Here is what the clean-start fantasy leaves out: that ugly old system is ugly because it works. Every strange conditional, every weird special case, every line nobody remembers writing is usually a bug that was found and fixed, an edge case a real customer hit at 2am, a rule the business genuinely needs but never wrote down. That knowledge is not in a document — it is in the code. Throw the code away and you throw the hard-won knowledge away with it. Then you get to rediscover every one of those buried bugs the slow way: in production, with real customers, one painful surprise at a time.

This piece is about the single most consequential decision you will make about a system you already own: telling the case where it has genuinely aged out and needs replacing apart from the far more common case where "rebuild from scratch" is a two-year detour that lands you roughly where you started — minus the money and the years. There is a third door most people in that meeting never see: modernize the thing in place, one piece at a time, without ever taking it offline. That door is almost always the right one.

The most famous version of this mistake has a name attached to it. In 2000, watching Netscape rebuild its browser from the ground up, the software writer Joel Spolsky called the decision "the single worst strategic mistake that any software company can make" ^[1]. The rewrite opened a roughly three-year gap in which Netscape shipped no competitive product, a rival swallowed the market it had once led, and the company effectively did not survive the detour. His core argument is the one that matters for any owner, not just a browser company: when you throw out working code and start over, there is no particular reason to believe the second version will come out better than the first — and while you build it, you ship nothing to the customers who are still paying you.

The deeper reason cuts against every instinct. That old code looks like a mess, but it is a mess that has been tested by reality for years. Each ugly patch is a scar over a real wound — a defect someone hit, an edge case a customer tripped, a tax rule or a refund path or a rounding quirk the business actually depends on. None of that is written down anywhere; it lives only in the lines you are about to delete. A rewrite does not start from the spec you have. It starts from a spec you no longer fully know, because most of what the system does was learned the hard way and never documented. That is why rewrites reliably rediscover old bugs: you are not building the same system again, you are re-learning it in production.

Even when a rebuild does happen, it tends to bloat. Fifty years ago Fred Brooks named the "second-system effect": the second system a team designs is the most dangerous one they will ever build, because they finally get to add every feature they had the discipline to leave out of the first ^[2]. The lean, focused original — spare precisely because its builders were still cautious — becomes an over-engineered monument to everything the team always wished it had. So the rewrite is not just late. It is often worse than what it replaced: heavier, slower, aimed at a wish-list spec that already moved while it was being built.

This is not a niche engineering worry — it is where the money already goes, and the clearest public numbers come from the one buyer forced to publish them. The US federal government spends about 79 percent of its planned IT budget — on the order of $83 billion a year — just operating and maintaining systems that already exist, rather than building anything new ^[3]. Some of those systems are more than fifty years old, written in languages almost nobody still learns, a few once running on hardware as old as 8-inch floppy disks ^[4]. And of the ten most critical legacy systems flagged for modernization in 2019, only three were finished six years later ^[4]. The lesson is not "government is slow." It is that replacing a load-bearing system is genuinely hard, big-bang replacement stalls often enough that even a near-unlimited budget cannot force it through, and the maintenance meter runs the entire time you argue about it.

The way out is not rewrite-or-suffer — there is a third path, and it has a name too. Martin Fowler calls it the "strangler fig," after the vine that grows around a tree and slowly replaces it until the original is gone ^[5]. You wrap the old system in a thin routing layer, move one slice of functionality at a time onto new code behind it, and shrink the old system piece by piece until there is nothing left to switch off ^[6]. The system stays live the entire time. Every step is small enough that being wrong costs a week, not a year, and each step ships something the business can use instead of a promise it will one day. It is less heroic than a grand rewrite and far more likely to actually finish — which is the only quality that matters.

None of this means never rebuild. Full replacement is the right call in a narrow set of cases: when the platform underneath is genuinely dead — unsupported, unsafe, and nobody left who can run it — when the business the system was built for no longer exists, or when the thing is small enough that a rewrite is a few weeks of work, not a few years. The honest tell is scale. A rewrite you can finish before the old system changes underneath you is a rewrite; one that takes years is a bet against a moving target, and you will probably lose it. For anything load-bearing, the default is to modernize in place — and to choose it deliberately, with the parity gap and the second-system effect on the table, not in the heat of the meeting where someone says it would be faster to start over.