Productivity frameworks like DORA, SPACE, and DX Core 4 can't answer the important questions
Posted by iamalnewkirk@reddit | ExperiencedDevs | View on Reddit | 12 comments
Productivity and performance are different things. A team can have great deployment frequency, high developer satisfaction, smooth CI/CD pipelines, and still deliver the wrong thing late. These developer productivity frameworks can't catch this because it's not what they're designed to do.
The bigger issue is these frameworks explicitly refuses to measure at the individual level. I get the reasoning (although the examples used are shallow and easily refuted).
"The key is not to measure at an individual level. Doing so can cause inauthentic results since some might inflate metrics, like maximizing individual pull requests, to skew results for personal gain" — Bill Doerrfeld
Organizations are fundamentally individual-centered, e.g., compensation, promotions, PIPs, performance reviews, etc, all map to a specific individual. So you end up with a system and dashboards that can only gesture directionally, i.e., they tell you a team is slow but can't tell you whether that's a tooling problem or one person not delivering while everyone else carries the weight.
Every problem gets treated as systemic. Slow team? Must be the build times. Must be the process. Sometimes it is. But sometimes it's a person, and the framework literally cannot surface that.
Does anyone else have strong opinions on this, and/or the broader DevEx movement?
I'm all in favor of improving the development environment, including tools and workflows. I just don't like conflating that with performance evaluation and business outcomes.
TomOwens@reddit
Two Deming quotes come to mind: 94% of troubles and opportunities for improvement belong to the system, and "a bad system will beat a good person every time".
The idea that organizations are centered on the individual is part of the problem. Results come from teams and the systems in which people work. Focusing on identifying issues with the system will help you resolve the vast majority of problems. Centering on the team and optimizing for team performance will be more effective. Using metrics that reveal problems in the system and identifying opportunities to align compensation with team and organizational performance is the point of these approaches.
RelevantJackWhite@reddit
"Organizations are fundamentally individual-centered, e.g., compensation, promotions, PIPs, performance reviews, etc"
Disagree hard. Organizations are oriented on what they create, which is not done by one individual, it is done by the org. You see the parts that are individualized because they directly affect you, but the execs are not focused on individuals.
iamalnewkirk@reddit (OP)
Hard disagree with your disagreement.
Organizations, teams, groups, departments, etc are all abstractions. Each one ultimately represents collections of individuals. Corporations have agents and stakeholders. Agents are called into court, not corporations. It's all individual-centered. That makes it a distinction without a real difference.
Executives are absolutely focused on rewarding and retaining, or dismissing, individuals. The entire enterprise depends on it.
These frameworks exist to improve the working conditions of individuals, but they just don't want to hold individuals accountable. That part is quite interesting.
RelevantJackWhite@reddit
"Executives are absolutely focused on rewarding and retaining, or dismissing, individuals. The entire enterprise depends on it."
this is only true because individuals are needed to create the product an org is interested in creating. That's exactly why engineers worry about the threat of AI.
These frameworks do not exist to improve working conditions of individuals. They exist to more effectively create products.
iamalnewkirk@reddit (OP)
AI speeds up building, not deciding. It will dramatically impact the numbers of software engineers, but AI doesn't decide autonomously (yet).
It lowers the cost of writing code but doesn't ensure you're building the right thing. Most failures come from bad product decisions, not slow execution. The real bottleneck is judgment, not productivity.
Being maximally efficient with exceptional working conditions doesn't mean you're building the right thing (i.e.,"effectively create products").
RelevantJackWhite@reddit
Yeah, I agree - I'm talking hypothetically here. If AI were that good (or if there were any other way to accomplish the business goals), the individuals would be dropped
iamalnewkirk@reddit (OP)
True.
Goducks91@reddit
Yep. If they could get rid of every engineer and keep the same quality/output they would.
IPv6forDogecoin@reddit
Maybe write your own posts instead of copying the output of chatgpt
iamalnewkirk@reddit (OP)
Why?
vansterdam_city@reddit
I think the problem is that measuring data and evaluating developer effectiveness are mutually exclusive because software development / developers are not fungible.
We can easily compare apples-to-apples between two people who make widgets in a factory. But almost by definition, if there are repetitive software tasks then we have failed our jobs by not automating them already.
Each software task is unique enough that it's hard to measure. I think it's a fundamentally unsolvable problem to attempt to distill our work into measurable metrics.
iamalnewkirk@reddit (OP)
1000% agree. But disagree on the measurement piece. We can (and do) effectively measure software engineering outcomes, just not in the way most people think and the way that developer productivity frameworks suggest.