Why software projects fail
Last Thursday I was reviewing a pull-request for a 200-line Swift UI tweak in the morning and helping untangle a decade-old PHP monolith by lunch. That’s the agency life in a nutshell — half a dozen unrelated codebases in a single day. A friend once joked that “one agency year equals three product years,” and I kind of believe him (though I haven’t figured out the exact exchange rate).
Switching context that often gives you a weird super-power: pattern spotting. After a while you start seeing the same potholes on very different roads. Most teams hit them, no matter the industry or tech stack. And when a project goes off the rails, you rarely get a neat post-mortem pointing at the culprit. It’s usually a cocktail of tiny, boring issues — the silent killers that look harmless right up until the deadline explodes.
I’ve skimmed roughly a dozen failure write-ups this month alone (Standish CHAOS 2020, a few Gartner case studies, couple of internal docs clients shared) and casually reviewed them against the war stories from our own backlog. The usual suspects — missed deadlines, runaway budget, key people leaving — show up every time. But I want to talk about the subtler stuff, the kind that doesn’t make the slide deck yet derails the project just the same. Failure isn’t only “we never shipped.” Retiring a system two years after launch because nobody uses it is failure too — the quieter, more expensive kind.
Source: commitstrip.com
Overconfident developers
Confidence keeps you moving; overconfidence makes you skip the map. I keep bumping into four flavours of it:
- “My code is solid.” We forget the outliers — that one Friday commit nobody reviewed because beer-o’clock was calling.
- “I’ll pick up this new framework over the weekend.” Serial tutorial watching ≠ production readiness.
- “This abstraction is future-proof.” Right until the future shows up with requirements you didn’t imagine.
- “The vendor API looks stable.” Famous last words before the breaking-change email at 2 a.m.
Ask ten developers whether they’ve ever shipped terrible code and you’ll get a mixed bag: some will laugh and confess immediately, a few will hesitate, one or two will insist their commits are pristine. (I used to be in that last camp — selective memory is powerful.) Surveys like the 2023 StackOverflow pulse back that up: plenty of engineers rate themselves “above average,” Lake Wobegon style.
On a normal day I probably can deliver a medium-size feature in three weeks. The trap is assuming every day is normal. We discard the tail-risks, the obscure bug that eats four days, the teammate who gets sick, the CI runner that suddenly needs an upgrade. That’s the Planning Fallacy in action — Kahneman & Tversky wrote about it long before Agile poker made it fashionable.
💡 I'm not entirely sure this scales beyond our use case, but tracking "tail events" separately from the happy path improved our estimates more than any planning ceremony.
The abstraction issue hurts even more. A lovely, generic interface looks smart on day one. Six months later a new requirement lands that almost fits. Someone wedges in an extra flag. Then a switch-case. Next thing you know every new feature adds one more branch because “refactor later” feels cheaper. I once tried deleting a barely-used reporting module to reduce that complexity — three departments vetoed the PR because the feature sat in somebody’s OKR, even though logs showed <20 monthly users. Organizational inertia keeps dead code alive long after its technical merit is gone, though it didn't fully fix things.
External dependencies deserve their own horror anthology. Last year a payment provider rolled out a mandatory TLS change with two weeks’ notice. Integration teams burned half the sprint just proving their stack even could support it. Multiply that by every library, SaaS, and docker image in the chain and “on average it works out” stops being comforting. Given a long enough timeline, there's a high likelihood of something exploding.
Inexperienced Managers
The other unplanned variable: the person steering the ship. Sometimes they’re inexperienced, sometimes competent but mis-incentivised, sometimes both (a fun combo). Their KPIs often reward visible activity over sustainable progress, so scope grows, risk shrinks (on paper), and everyone above them feels reassured — right until the deadline slips.
The classic example is promotion-driven delivery: manager jumps to a new role right after “launch,” leaving the maintenance phase to the next unlucky soul. Peter Principle, textbook case. But the quieter danger is the overly agreeable manager — the one who says yes to every executive whim. When “Login with TikTok” lands in the backlog of a banking app you know exactly who approved it.
Red flags I watch for:
- Information hoarding. “Trust me, we’re on track.” Spoiler: we’re not.
- Context-free task pushing. Deadline first, specs later.
- Credit monopolizing. Team delivers, manager demos, applause follows.
- Decision paralysis. Every minor choice escalates upward.
- Public blame shifting. Nothing kills morale faster.
I could be wrong, but the fastest way to salvage such a situation is exposing real cost early — show the burn rate, the staffing impact, the delay projections. Numbers travel upward better than complaints.
Mismanaged Stakeholders
A project is supposed to deliver value, not just survive stand-ups. The tricky part: value gets fuzzy once budgets are approved and slide decks printed. Confirmation bias sets in, nobody asks the scary question: “Should we kill this?” Early cancellation feels like failure, so the team keeps polishing — classic sunk-cost fallacy dressed up as perseverance.
I’ve watched projects cruise through the first 30 % looking healthy: roadmap aligned, cash reserves solid, tech chosen. Then reality intervenes — competitor ships first, key vendor shutters, hype shifts to AI and the funding reallocates. Momentum slows, departures start, a fresh tech lead wants to rewrite half the stack (I’ve been that tech lead, for the record) and direction drifts.
Nobody wants their name stamped on the tombstone, so the zombie product lingers. Budget lines get creative, milestones get redefined, success stays just one quarter away. Meanwhile every feature request triggers the “not my job” reflex. That mindset seeds a toxic loop: less collaboration, more politics, spiralling delay.
💡 This is what worked for us once: schedule a "red team" review with outsiders who have zero emotional stake. Their brutal honesty hurts, but it breaks the denial cycle.
Worth noting: civil engineering hits the same wall. Look at Berlin’s airport or Boston’s Big Dig — billions over budget, years late. So it’s not that software people are uniquely bad at estimates; humans in general are.
Once the “not my problem” culture takes hold, technical issues become social ones. You can’t patch them with a hotfix. Accountability fragments, trust erodes, and every stand-up turns into status theatre. By then rescue costs more than a greenfield rebuild — yet somehow the budget for that is even harder to get.
Conclusion
Projects fail for dull, human reasons: overconfidence, perverse incentives, collective denial. No silver bullet, just vigilance and the occasional willingness to say, “Stop, this isn’t working.” I still get it wrong more often than I’d like — but spotting the pattern early buys you options.
Got your own cautionary tale? Drop it in the comments — anonymised horror stories welcome.
Other Newsletter Issues:
Worried your codebase might be full of AI slop?
I've been reviewing code for 15 years. Let me take a look at yours and tell you honestly what's built to last and what isn't.
Learn about the AI Audit →No-Bullshit CTO Guide
268 pages of practical advice for CTOs and tech leads. Everything I know about building teams, scaling technology, and being a good technical founder — compiled into a printable PDF.
Get the guide →
15 Comments
Nice article
Jumping into projects thinking you’ll figure out the tech stack as you go can really mess things up. Always better to budget extra time at the start for learning. Ever noticed how overconfidence can trip up even the best of plans? Always overshoot how long learning something new will take—it saves stress later. And honestly, talking straight with stakeholders about what can go wrong and how long things will really take can save a lot of headaches. Anyone else had to learn this the hard way?
In over 30 years, I have never worked on a project that did not deliver. Original delivery date was moved back in a few cases, but deliver we did. In summary, this is down to having good project managers most of the time, hence planning was done properly most of the time, but the most important and crucial factor is having a clear objective at the beginning of the project, and deviation from this was never agreed to or kept to a bare minimum in all cases. As an engineer or a project manager, I have always ensured that I am clear on the objective before agreeing to work on any project.
Great article. I’d add another common failing , estimating development effort at the very beginning of a project when there are so many unknowns, setting these in stone as deadlines and never revising these deadlines as more is known as what is to be done
Dependency is a code smell for an over confidence in abstraction.
Good writing. These are all good insights indeed. In my experience the key fallacy continues to be thinking “we know it all” – and it concerns all stakeholders, not just developers. What’s worse non-tech management often thinks that software development is for the most part just coding, and coding in their eyes is as simple as getting an “obvious task” and “just get done with it”, which if course couldn’t be further from the truth. Managers in general also assume they know how to make developers go faster, mainly by meeting often and agreeing every detail upfront, which of course is mostly a bad idea.
Developers on the other hand assume they have it all figured out technically, and also they know best what to do. This concerns in particular adding third party dependencies and in tendency going after shiny new things instead of using well known libraries.
Exploring new tech is thrilling but can lead to delays if not carefully planned. It’s equally important to know when to end a failing project to save time and effort. Learning from such experiences can guide future decisions, making project management more effective.
I have been a software coder (engineer) for over 25 years. Recently retired. No problems. Mostly worked in FoxPro 2.6 and Visual FoxPro and traditioned to VS C# with some incursions into VB6.
I have worked as a company employee and as a contract employee. As a contract employee I had much less to do with company politics. As a company employee I was constantly exposed to company politics and fired sometimes due to company internal stresses. At one place I think I was a threat to the VP. Not that I ever thought of myself as a threat to his job. After leaving that employer I found other employments as a contract employee and was very happy and productive in that position. My last twelve years I was a company employee.
Projects fail because of three + main reasons. Programmers who are overconfident in their ability. Programmers who do not realize the scope/complexity of the project. Programmers and manager who do not know how the product works at all levels. Managers who do not understand what is needed (coder ability, skill sets, knowledge) or the scope of the project and the need to limit the scope creep that management always? will push for. Scope creep is a killer and needs a strong management team to oversee the project.
Thank you,
Gifford T Nicholson
In economic terms, a software project fails when the cost of implementing change exceeds the anticipated benefit of the change. That’s my insight from observing software projects over 30 years.
I see a continued failure to do a “crawl, walk, run” on projects. Come up with the “crawl” and say “I can do that for you in 2w, and then we can look at where we are and figure out what the walk looks like”. There’s no way you can accurately estimate what the “walk” or the “run” looks like in terms of time and expense until you’ve validated your assumptions and re-verified the goals and requirements of the project. Obviously this doesn’t work for all types of projects, but the vast majority in my 30y experience.
9 times out of 10, once you get to “crawl” you will have found 10 things that were wrong about your original assumptions or requirements, and a “crawl” or “MVP” will give you time to adjust before you continue on.
Hey Louis, very interesting concept, thanks for sharing
Jumping on new tech? Cool, but comes with a learning curve that can mess up timelines. Gotta weigh if it’s worth the hassle. Also, knowing when to kill a failing project saves $$$ and stress. Fail fast, learn faster. Cut the crap and move on. This ain’t rocket science, just common sense in the dev world.
Jumping on new tech without considering the ramp-up time is a common pitfall. It’s essential to realistically assess the learning curve and challenges. Also, recognizing when a project is no longer viable and having the courage to make tough decisions early can save resources and prevent bigger failures.
I once jumped too quickly into using a new programming language for a project, thinking it’d make us more efficient. Turned out, the steep learning curve actually slowed us down and stressed the team out. We learned the hard way that sticking to familiar tools can sometimes be the smarter move.
Jumped into a project with a new stack thinking it’d speed things up. Reality hit hard with the learning curve slowing us way down. Everyone got bogged down with tutorials instead of coding. Ended up switching back to our old stack to meet the deadline.