Are we getting worse at our jobs?
Posted by SimonSim211@reddit | ExperiencedDevs | View on Reddit | 46 comments
According to this article https://spectrum.ieee.org/it-management-software-failures we still have the same failure rate for IT projects as 20 years ago.
If we assume that is right. And also assume that we have access to better testing tools and more type-safety in mainstream languages, etc. give us some advantage.
If a project is done in 10 months instead of 12, due to new tooling, then political infighting that could kill the project should have a lower chance to kill it (as that takes time too).
So why are we failing at the same rate?
My only guesses are that:
- Fewer Devs build up domain knowledge.
- higher percentage of External devs that can't speak up if there are problems.
explodingfrog@reddit
We still work in the same way as before. The tooling has never been the issue.
When you've worked in a truly high performing team, you get drastically different results than scrum/ticket driven development. But doing so means you need to sacrifice your sacred cows - individual heroics, PRs, and actually working to increase team efficiency and effectiveness, not individuals.
Kseniya_ns@reddit
From management perspectives and such, all the modern advantages which programmers have is directed into having higher output more work done faster, not better. It will be the same with AI, so it would think, developer can work efficiently then is more time to test and refine, no, is just expectation of more quantity of bad quality being produced.
AlgeaSocialClub@reddit
I believe the problem is that there is no “quality code metric” anyone can point to. I work with plenty of people who get stuff done, but when you look at their code it’s obvious that it’s fragile and highly prone to bugs. Of course, how do we quantify this? We can’t. You can track bugs over time and map these back to certain features and then determine bug rates for each dev, but let’s be real nobody’s gonna do that. It’s also not that clean. Multiple people touch features. Anyway, yeah I think you’re right and I just wanted to add to what you said because one thing I’ve never heard in over a decade is “Let’s put a freeze on new features and tackle tech debt this quarter”.
Kseniya_ns@reddit
Aha yes indeed, I sometimes dream someone would have such a fantastical idea as that. An idea that is attractive to no one except the programmer themselves
Inner_Butterfly1991@reddit
I actually worked at a company that did this, it was a month not a quarter, but we had just lost a client due to a bug and we had convinced our CEO if we didn't carve out time to tackle tech debt we could have more. So we had a single month dedicated to nothing but tech debt, he pushed out feature estimates to our clients a month extra, and we barely were able to move the needle on the tech debt and a lot of the "solutions" to tech debt introduced new tech debt. I suspect even if we'd been given a year to do no feature work all tech debt that wouldn't have been enough either.
agumonkey@reddit
it's also related to customer relations. if they drink the "we're a continuous improvement shop, you file a bug we fix it, great !" kool aid, then you don't need as much quality
HearingNo8617@reddit
I do think there will be lower awareness of poor quality though with AI, since it is hyper-optimised for looking good at a glance. This will probably worsen things. At least with type safety and other productivity increasing technologies, the poor quality is quickly noticeable usually
cgt303@reddit
Sure the tools are better but IMO the overall business environment is worse. I don’t know about everyone else but it seems speed is the most important thing at my workplace. I’m bringing up the basics regularly (I.e observability and testability) just to get the brush off and told we don’t have time. The most important thing is features. It almost feels like in the AI age every company is scared of being “left behind”. I’m not proud of anything I’ve pushed to prod in awhile - everything feels rushed. Then again maybe it’s just my job…….
infinit100@reddit
Because there are the same fundamental challenges in software delivery today as there were 20 years ago (and 40 years ago).
The mythical man month told us in the 80s that adding more people to a behind schedule project would slow it down further, but I still hear people asking for more devs to help speed up a delivery.
Agile was proposed as a way to work better, but most places still expect committed timelines, scope and effort, so Agile becomes a more complicated waterfall.
There’s still a pressure to ship faster and beat the competitors, which means corners get cut and things get missed.
None of these are problems that tooling or typesafe languages will fix
__scan__@reddit
Maybe that’s the “right” failure rate, so we pull other levers (scope, complexity, timeline), so we end up landing there?
potatolicious@reddit
Ding ding ding. This is the winner.
First of all, the cited study is really vague about what project scope is entailed and what constitutes failure, so take the number with a giant grain of salt.
But more importantly, the optimal failure rate for projects is higher than zero.
More directly: we have different problems and failure modes now, largely as the result of scopes having drastically expanded over the past 20 years. You used to hire a sysadmin to host a literal static HTML page. Now you kick a few bucks to some provider and it works 99.9999% of the time.
Getting a trivial web form on a page to save stuff into a database used to be actual project work that you'd hire SWEs for. Now you can buy any number of off-the-shelf solutions that pretty much just work... or if you really want to DIY there are a bajillion open source things you can deploy basically trivially.
Twenty years ago only a tiny slice of internet traffic was delivered to mobile devices. Streaming was a fantasy. The idea of any computer system having live access to the user's location was science fiction. Sharding a database was something only the absolutely biggest players in the industry even thought about. Now it's all routine.
We grew scope, that's really all there is to it.
lunivore@reddit
If a project can be done in 10 months instead of 12, that means that all your organizations competitors are also doing their projects in 10 months instead of 12.
If your developers can go 4x faster with AI (for instance, not saying that's correct) then your organization's competitors are also going 4x faster.
That's the pressure that doesn't go away, and it leads to the same practices: unrealistic deadlines, failure to keep options open, and refusal to cut scope when needed - sometimes because of the need to achieve feature parity with competitors but more often because of organizational habit.
gurudennis@reddit
That's right on the money. I have seen many a sin committed in the name of nominal feature parity.
bwainfweeze@reddit
Management has decoded Scrum enough that they’ve reintroduced most of the problems it was meant to solve.
gurudennis@reddit
That's a backwards take in my opinion. Scrum was invented and aggressively pushed by management to legitimize the chaotic change in requirements, which is only partly inevitable in the industry and partly their fault to begin with. Initially, Scrum and Agile in general had a veneer of "gamification" attached to it and it fooled some devs into liking it, but ultimately both devs and the management found it disappointing (for different reasons). That's why these days when the leverage is firmly in the management's hands due to the the state of the job market and the constant threat of outsourcing that every shop has its own dysfunctional process that's only "agile" in name.
bwainfweeze@reddit
I was around and in the Agile space before the XP book was published. While the proponents were managers as you say, they were line or second level managers. The tools of Agile are meant to provide fast feedback for the worst predations of poor organizational management - middle to upper management practices. That is why developers bought into it. It was a bargain we made to stop wasting years working on things we knew would never ship as designed.
The bargain part was that we tried to do better too. But part of why it worked is that it felt so counter-intuitive that they had to trust in us and the outcomes instead of their 'gut' instincts, which were objectively bad.
Scrum comes along and makes all of the counterintuitive parts seem intuitive, and fit their world view. Now instead of a pact for improvement it became yet another set of thumb screws, keeping devs off-balance so they never have time to consider big picture architectural problems or whether we are working on software that is actively and provably unraveling human civilization (Meta, Google).
That's what I mean by all the old problems are back.
gurudennis@reddit
Thank you, that's an interesting perspective. I won't deny that the various products of the Agile ideology have their value, and I'm certainly no proponent of waterfall in its purest forms. Yet I think we both agree that today we have arrived at a state where Scrum is used as a means of perpetuating short-term interests over balanced software development. If anything, our collective inability to come up with a better system has fueled the explosive growth of tech debt that I've been observing in the past decade.
CheithS@reddit
We haven't improved at the process of developing software.
Too many people still want to take shortcuts.
Too many people don't test properly (except in very simple cases unit tests are not enough).
Requirements -> technology mapping is still not great and too many software dev still have tech 'religion'.
Ability to estimate in the 'Agile' world has dropped through the floor leading to atrocious planning and dependency mapping for complex projects.
Etc, etc, etc.
We haven't improved because we actually haven't got any better at what we do. Tooling doesn't help.
_ezaquarii_@reddit
What I'm observing is that all modern advantages are enabling more incompetence to be tolerated elsewhere.
Distributed atrophy.
Stubbby@reddit
Definition problems, Scope problems, Management problems, Talent problems, Support problems, Culture problems.
Out of the 6 main groups of issues contributing to failures they listed, offshoring makes ALL of them worse.
TheElusiveFox@reddit
So I'd say a couple of things...
Software projects are always going to have a high failure rate compared to other industries... Outside of a handful of key sectors, software tends to be developed with the mindset of move fast and break things, because unlike other industries, we can patch it later, and the consequences for a bug are usually relatively small...
That is why agile like project management methodologies are so popular. It lets software teams work with a very compressed design/planning phase, with the understanding that you are coding somewhat in parallel to those phases and might need to pivot as you gather more information. That is a super power in software, unlike say construction where, you can't poor concrete until you have the exact engineering specs, and those specs cant change whether you want or need them to or not.
But that flexibility is also a problem - it means software projects can get bogged down in endless scope creep as different people want different features, and it means your vague requirements might mean you are coding a solution to a problem that doesn't actually exist and once some one with the right domain knowledge understands what you are doing you can get shut down even if lots of people thought it was a good idea...
And I'd say as projects are moving faster and faster, that is the time consuming and important work getting cut out that leads to failure... sure A.I. coding saves time but that just means the sooner the right manager sees a working prototype the sooner they see that the project isn't actually what they want/though...
Final thoughts
I think with how fast tech moves - there is a lot of risks for devs who want to be individual contributors in their career in being the domain expert in a shiny new tech, only for that tech not to be relevant a year later, that leads to a lot of devs with very surface level knowledge on a lot of things, especially at companies led by execs who pivot from shiny object to shiny object...
gingimli@reddit
I think employee retention plays a big part. There’s really no incentive for an individual to stay at a company more than 4 years, in most cases it works out to the individual’s advantage to frequently job hop. This results in constant brain drain and developers that aren’t super invested in long term solutions because they won’t be around for the fallout anyways.
writebadcode@reddit
Yeah I stayed at my last job for 6 years and then I got laid off. Around the 3 year mark, someone I knew who worked at NVIDIA tried to get me to join his team, but I turned him down.
That was in 2020… the stock price was like $9/share. It’s 20x higher now.
I kinda regret that one.
gingimli@reddit
If it makes you feel better I probably would have done the same. I don’t think there’s anyone that expected the video game chip company with E3 “booth babes” to become a $4 Trillion company.
diablo1128@reddit
Stayed at my first job for 15 years, private non-tech company in non-tech city, and it was a complete mistake. Even if you don't consider the money aspect, I basically learned and experienced a lot less than people who have changed jobs over the same 15 years.
I more or less have experienced 1 way of doing things in a professional setting. That's just not impressive to recruiters and interviewers. It's not that I am not familiar with other ways, either process or code level things, but I just haven't had the opportunity to do it in a professional setting.
robby_arctor@reddit
Exactly. I'm 5 years in and have had 4 jobs because I'm trying to get decent raises. I like where I work now, but I will probably have to quit again next year if I want a decent raise.
I'm basically never working on a product/stack that I am deeply familiar with.
Empanatacion@reddit
We are failing at the rate that we have all collectively decided is the right balance between time, cost, and quality.
We are biting off bigger chunks and failing at the same rate, so we're getting more done.
As for quality... I'm holding a mobile device that has instant access to the entirety of human knowledge.
PracticallyPerfcet@reddit
Don’t underestimate the incompetence of leadership.
What is happening now, especially with AI, is more software than ever before will be created that is never used.
SnugglyCoderGuy@reddit
What is success predicated upon? Typically it is being on time, on budget, and on scopr. Those are predicated upon what? Estimates. What is classically done very poorly? Estimates.
The failure is with project management theory and how they are applied, largely without change, to software products.
There is a subtle, but monumental, difference in words above.
lambda_legion_2026@reddit
Management doesn't care about quality. Devs have to fight their leaders to ensure quality improves on any substantial project. Most of us just want a paycheck and don't enjoy that kind of confrontation. That means we give leaders the fast results they want even if it's shit and then we move on to the next thing.
dedservice@reddit
Basically the Peter Principle of software development projects: There is some risk level or expected failure rate that is acceptable for software projects, and the difficulty/size/scope/deadlines are pushed harder until a project inevitably reaches that risk profile.
If we were building the same systems that people were in 2005, we might finish in a week where they take three months. But so would all our competitors - pushing us all to try to take bigger risks with our systems.
pydry@reddit
Management has gotten more authoritarian and the outcomes of this are worse.
flukeytukey@reddit
Every time we introduce a new tool or framework or way to deploy or whatever thing that's supposed to make life better, it never does.
Take React. I spent 9 years writing vanilla js with bootstrap and jquery. I made performant web audio players, graphs, dashboards, live tables, you name it. Not only was it easy, but it was well written, because I write my code well. No frameworks.
Now we have react, in which its my 6th year. Not only does it take me the same amount of time to do anything, it's gotta be transpiled, it's got endless quirks and gotchas, it doesn't provide any structure (i know its not supposed to) and it has to run on basically another system altogether (its virtual dom).
All we seem to do is step sideways, over and over, and rarely move forward. Why do new frameworks come out all the time? Why do big frameworks like MUI et al reinvent their own wheels every couple of major releases?
Maybe we just all actually suck, or there is just no way to perfectly and easily translate human desires into code.
neilk@reddit
Nope, it's that tools are rarely the cause of project failure.
Look at this diagram. I don't even see one failure factor that could be addressed by type safety.
https://spectrum.ieee.org/media-library/diagram-showing-causes-of-technology-project-failures-definition-scope-management-culture-etc.png?id=62207045&width=900&quality=85
The article does not get into the details of why this particular project failed, although it does not bode well that they thought they could skimp on pilot testing (which I assume is alpha or beta testing).
I find it more interesting that practices like CI/CD, and all the abilities we now have to monitor and observe, all the way up to the frontend, haven't had any impact. But all that data has to eventually be used by a decision maker, and if decisions are hampered by other factors, all the tooling in the world isn't going to help you.
Nekadim@reddit
Because doing right things is not the same as doing things right. Tooling helps with the latter but not helping with former in any meaningful way
cachemonet0x0cf6619@reddit
Article says it’s unmet lofty goals. it’s never your fellow devs.
Tired__Dev@reddit
Bureaucratic bullshit. I got halfway through the article and it started signaling alignment with me, so that’s my bias. There’s two things I see, as someone that’s failed a project with my own money, and ran a successful agency with many projects.
One, artisan bullshit from the software team. That means when the craft of software development is more important than the customers being served. This is resume driven development, dumb architecture decisions like greenfielding with a microservice, or pedantic subjective code practices in code reviews.
The other one is management. Non technical management needs to be removed from this. Weak product managers lead to the above problems. Project managers that are technically incompetent don’t actually have the context desperately needed to measure progress. Performative people suck all of the energy out of people. Then when they start fighting with each other they will actively sabotage projects for their egos or because they don’t like someone. There’s thousands of levels of developers that exist these days.
mxldevs@reddit
Tools to reduce and manage failure don't matter when they're not being used or enforced.
Even if previous teams set up a system, new people taking over might decide it's a waste of time and money and just bypass them
Wooden-Contract-2760@reddit
Management hasn't improved. They take on ~40% more work for development teams than what they could deliver properly.
The amount of actual implementation this covers may grow, but players will always play and from the time whence whips increased performance, there was never a moment when managers hesitated to leverage those tools in exchange for a slightly higher defect rate as long as it yields.
Hence, defect/value is a valid ratio to balance and likely won't change for the better as the whole industry leans towards gambling and racing.
JollyJoker3@reddit
How do we have more type safety? We used Java 20 years ago, now most code is Javascript/Typescript or Python. Java forces you to get the types right.
ashultz@reddit
A project that was going to be killed by failure to face reality in 12 months will not miraculously do better in 10. Projects mostly fail because people refuse to face the obvious, things like "you have no plan or priorities" or "did you talk to any customers" or "your engineers told you LLMs don't work that way" and none of those are actually technical problems or time-based problems.
Lucifernistic@reddit
You also need to consider the fact that the scope of projects is vastly more complex now. You can't compare scaling a SaaS solution that serves four million users and has a complex series of interconnected parts and machinery, to correctly implementing a LAMP stack 20 years go.
The internet is bigger. The products are vastly, vastly, vastly more complex. The scale of things dwarves what came before. So it doesn't seem crazy to me that failure rate is relatively consistent, since the introduction of more resources and better tooling needs to balance against increase in scope and complexity.
In a word, we are failing the same %, but we are doing way more while keeping that failure rate the same.
bwainfweeze@reddit
And the number of projects attempted has also changed. You can’t directly compare percentages when the denominator is not the same.
Maybe 10% is the native comfort level for project success and we just keep dialing up the pressure until we achieve that rate again.
behusbwj@reddit
As someone who works primarily in large companies, it’s astounding that the failure rate doesn’t balloon.
To keep a product operating, scaling and growing among the exponentially increasing complexity and revenue is not an achievement to take lightly.
thecodingart@reddit
Most developers suck - and there’s more developers today then there was in the past
NGTTwo@reddit
Because the fundamental problems haven't changed. We're able to tackle bigger projects in the same timescales, but the problem was always the surrounding organization, not the technology. Unclear and changing requirements, poor management, and corporate politics have always been the primary reason for project failure, not anything inherent to technology or timeline.