Management wants numbers, what KPIs do we give them?
Posted by MartinSch64@reddit | ExperiencedDevs | View on Reddit | 111 comments
To sum it up: I work in a small developer team of about 10 people, consisting mostly of software engineers. We all feel like our team is working pretty efficiently and working in harmony. That said, we connect to quite a lot of external apis, where there are regular changes on the other ends we need to adapt to constantly. That work does not show really well as nothing visible changes if this job is done right. Also there is a lot of legacy code and technical debt. Because of that new features tent to take longer.
Now the leadership of the company wants some more reporting from us including some KPIs.
We are well aware that SWE KPIs usually don´t work well and usually fall to Goodhart's law resulting in a worse output.
Like you want me to write more lines of code? Sure can I do that, does not say these lines need to do something useful.
What numbers should we give them, which:
- Can improve over time
- Do not incentivize us to do something hurting the quality of your code base, so number go up
- Are easy to collect/measure
The goal is not really to find something that measures our output/efficiency (IMO that is not really possible), but just to satisfy management and not make us look bad or will be hard to improve on in the long term?
Interesting-Frame190@reddit
Point number 7 of the agile manifesto.
"Working software is the primary measure of success"
Could be used to argue the points you've outlined. Its obviously hard to measure whats working, so measuring whats not working could be of more use. Some of my favorites are mttr (shows overall code health relative to system complexity and adaptability). Quantity of open bugs and rate of bug creation can be useful, but is a slippery slope to drive bad decisions with priority. If you measure the quantity of bugs, you'll want to pair that with overall ratio of bugs worked to stories worked to even out incentives.
RandyHoward@reddit
I once had a CTO and CEO go to battle about this. The CEO wanted KPIs for the dev team, left it to the CTO to determine what those metrics were. For weeks the CTO met with us devs and we all basically agreed that there are no great metrics. You can’t really judge a devs performance based on story points or tasks completed, because these things vary so much depending on the task. So the CTO refused to provide any KPIs.
Things got real heated after that. After the CTO refused, the CEO put him on a PIP. The very next day the CTO resigned, effective immediately with no notice. It was quite glorious to watch the chaos.
roodammy44@reddit
He put the CTO on a PIP? What kind of idiot was that guy? I suppose he was looking for the quickest way to make him resign, which he got.
aruisdante@reddit
I mean, in fairness he told his CTO “figure out a way to measure the value of your department” and the CTO’s response was “It’s not possible to measure the value of our department,” which is basically refusing to do his job.
Like, it’s stupid, but I also get it.
stubbornKratos@reddit
I mean I’m sure there’s more to the conversation but I don’t think it’s stupid. It’s their job to figure this stuff out
ephemeral_resource@reddit
The best I've seen, in my opinion, is managers would get together and discuss/rank employees as part of regular evaluations (quarterly?). It gave lots of opportunities to bring up things that are hard to capture as consistent metrics across a department.
Such as:
Supporting your peers more which has huge business value (one person can unblock several a day) but is less practical to capture.
Adaptability to new projects which may only matter every so often.
etc
Elmepo@reddit
It sounds like you're talking about stacked ranking, and it's generally agreed to be the worst way to manage unless you're actively attempting to reduce headcount. Once people know they're being ranked, they tend to act according to what's best for their survival, which is often not what's best for the team
mirageofstars@reddit
I mean yeah. “CTO can you provide me with any info on whether the dev department is doing well or poorly with respect to business goals, budgets, and industry standards?” and bro just said “nah.” I agree with the CEO. CTO is a C suite exec not a dev manager.
naverag@reddit
Was that that the CEO was asking for? Or was he asking "CTO can you provide me with numbers to say who the best and worst performing members of the dev team are so I can bully or fire them?"
RandyHoward@reddit
Yeah, it was kind of a both sides were at fault situation. The CTO did suck to begin with, even from my perspective as a senior dev on his team. He shouldn't have basically said, "No fuck off," when his boss asked for a way to measure performance of his team. He did propose some things that we all agreed weren't real measures of performance, but the CEO shot it down. Then came the PIP. Putting a CTO on a PIP is kind of unheard of, and even if it happens you don't make it known to his subordinates. We all knew.
CEO was a real idiot though. When the CTO quit they gave me access to his email so I could get things the team needed. Then they realized I had access to a lot of confidential information and revoked my access, but not before I read his PIP, his resignation, and the salaries of everyone on the team lol.
That company is dead now. I heard that some time after I left the investors ousted the owners and took over the company. Everybody got laid off and I don't think the investors continued it for long.
robobub@reddit
Sounds like an olden day version of OpenClaw
RandyHoward@reddit
It was a health company selling exercise and diet programs, and supplements. It had 3-5 devs on staff plus the CTO while I was there. One of the bigger shitshows I've seen. But it led me to meet a guy who I'd move on later to partner with, and we successfully sold our company 2 years ago, so that was good.
Unhappy-Ladder-4594@reddit
Yep. CTO should have known his job was to play the game the CEO wanted to have played. Give some easily-gameable metrics, that way everyone is happy. Even though devs and dev managers will know the metrics don't really mean anything in the real world.
EasyMode556@reddit
Asking someone to measure something that can’t be measured, and then after looking in to it having that person come back to you with the honest truth that it cannot in fact be measured, is not “not refusing to do his job”, quite the opposite, getting honest and accurate feedback is exactly his job.
Not doing his job would be if he made up bullshit KPIs that were meaningless just to make the CEO happy, causing the CEO to make key business decisions based off of useless nonsense data that they were told was reliable and useful.
Sensitive-Ear-3896@reddit
I would have returned with the metric: “Can the business run without us: 0” Before resigning.
roodammy44@reddit
He told him to find a way to measure it *in numbers* and that’s what the CTO pushed back on.
I mean, if I was the CTO I would have said something like “revenue” and “number of customers” which is probably not what the CEO wanted.
Measuring people’s work by numbers is called management by metrics and a cursory google will tell you how stupid that is.
zeroconflicthere@reddit
Absolutely could have used story points as a metric given the sizing variability. That way when it dips he could have pointed out it was because of vacation time for example.
Over time a team (without churn) can get quite consistent with estimation. Again, the fact that you can be cautious while relying on fibonacci helps to give a forward view on what can be delivered.
of course things change and stuff needs to be resized. But that's just another way to present the KPIs to explain why vehicle is changing.
Story points is a great way of getting stuff like technical debt included that wouldn't get prioritised otherwise also.
I don't know if any better metrics but from my own experience you could demonstrate a team working at a consistent velocity. And the best thing for the team is a story point is just so arbitrary.
46516481168158431985@reddit
If team is consistent you can just go with tasks done per period of time making points redundant. Points are great at sizing up stories whether they fit sprints and what not and forcing discussions about implementation but most people are so allergic to it.
johnpeters42@reddit
I would've been inclined to just make something up that sounded plausible but had one or more fudge factors.
aruisdante@reddit
Which is what almost everyone does in practice, and ends with developers wasting a bunch of time doing meaningless paperwork chasing making those numbers going up and to the right. Hence Goodhart’s Law.
Beginning_Feedback65@reddit
We recently had a reorg where our product owner, who had too much on his plate anyway, got more responsibilities higher up in the business, and so we got another PM to help represent us or some nonsense. The new PM requested daily updates written on all tickets about whatever we had achieved that day, just incase his bosses asked what we were up to. We already have a daily standup. The team pushed back on the redundancy, and we settled on "the pm can either attend the standup whenever he wants, or he can just reach out to us directly when he needs information." I've neither seen, nor heard from this guy since.
aruisdante@reddit
You would be amazed how many PMs don’t realize developers don’t have time to do their meaningless paperwork for them.
johnpeters42@reddit
Emphasis on "make something up". If it requires additional input beyond what the devs were doing anyway, then it's not made-up enough.
Spimflagon@reddit
Goddamn. Raise a glass to that CTO.
juxtaposz@reddit
Sometimes it's far easier to quantify the value of a team in their absence than in their presence. Imagine if they were Thanos-snapped out of existence. What would we miss?
scientz@reddit
I would do the same - the CTO was incompetent.
You might not want to hear it, but answering management with "there are no good metrics to measure" is not an acceptable answer. Lines of code, number of tickets etc. are far from ideal or good metrics, but you need _some_ metrics to know how your teams are doing. Most imperfect metrics will still help you baseline, track trends and spot deviations. A good manager uses these as one of the tools in their toolbox to keep track of whats going on. A bad manager will over-index on the metric and solely focus on improving it.
At a high level you want your metrics to align with the business goals. Development velocity (from ticket to production), business value generated (even better if you can tie work to revenue), KTLO vs roadmap work etc.
upsidedownshaggy@reddit
I think the issue is, at this point, anyone whose worked in a tech related role for a few years knows that their role and department are viewed as cost centers for a business. These KPIs will eventually be gamed so engineers can continue to justify their role's existence to MBA brained CEOs who only see value in terms of arbitrary number on chart keeps going up every quarter.
Izacus@reddit
Having good metrics is how you fight against being seen as a cost center - it'll tell C-suite about the value and effectiveness of your teams.
Being able to choose the metrics that make you look good is the ideal scenario for you - saying "I refuse" will just make you look like a cost center that can't defend the reason why it's eating millions per year.
upsidedownshaggy@reddit
I guess that still doesn't solve the overarching issue of whatever KPI you choose, even if it's making you look good right now, will eventually move away from being a measure to monitor team health and instead will turn into an ever shifting goal post that engineers will start to game.
I mean just look at all the articles coming out right now of major tech companies having to dial back their AI agent usage because they made some dumb ass metric to see who could use the most tokens, engineer figured out how to automate agents to do pointless tasks overnight that didn't do anything super useful while burning a bunch of tokens just to meet the now gamed KPI.
scientz@reddit
But the counterpoint is how do you manage something that you can't measure? How is software engineering so unique that it's impossible to measure, according to engineers?
upsidedownshaggy@reddit
I think you misunderstood, I'm not claiming that what we're doing isn't measurable, I'm claiming that what eventually happens is any KPI that gets chosen will shift away from being a measure and will end up as a goal that engineers will end up gaming just to keep their jobs and progress in their careers. It's entirely a management philosophy/culture thing than an engineer philosophy/culture thing. Like you said a good manager will use KPIs to pinpoint deviations and establish baselines, bad managers will use them as goals to be continually pushed to ensure they get their bonus.
scientz@reddit
Oh I'm with you. My counterpoint was more in the general theme of the post and thread, not directly at you and your post. Sorry if it came across like that.
Sinidir@reddit
Lol. Saying lines of code and number of tickets are far from ideal metrics is like saying a dog turd is a far from ideal lunch meal. They are counterproductive garbage. There are no good metrics is the correct answer. Quantifying a garbage metric literally drives you into the opposite direction of good software development.
MartinSch64@reddit (OP)
People already quit because of a lot of reporting upwards.
I think its now going towards micromanagement and more pressure.
Our head of development fortunately does a really good job shielding us from that and allowing us to do our job. Hope he doesn´t break some day as I really like working with that dev team and do care about our product.
oditogre@reddit
If you don't have any good ideas off the top of your head, I'd suggest leaning on soft skills - ask around, especially above yourself. Find out what exactly these mounting pressures are. Why is your team being asked for this, now?
If you can get any info along those lines, that could very helpfully inform the kinds of metrics you devise and produce, because you can target them specifically at those pressures.
Is money tight? Show how your team has been supporting revenue-generating teams. Is there some key deliverable that is at risk of missing its target delivery date? Show your team's turnaround time on tasks prompted by external teams, or show how your team has reduced downtime for other teams. Etc.
RandyHoward@reddit
I'm fortunate that my current team does none of this measuring performance stuff either. We don't even assign points or deadlines to tickets, but we do define the pool of tickets for each sprint. They get a bit annoyed when tickets carry over to the next sprint, but we don't have a ton of pressure performance wise, for now.
k3liutZu@reddit
I mean, measure what is good for the business:
- number of outages per month (should be 0, and remain 0)
- number of unintended regressions introduced per release cycle
- time to market (how long does it take to deliver a change into customers hands)
etc
Izacus@reddit
It's funny how everyone agrees that there's no good metrics... until we talk about AI. Then suddenly lines of code generated, issues created and PRs merged are amazing metrics for engineers to pull out 😛
danny4kk@reddit
I agree that many times the measure becomes the goal unfortunately. However, you can still use KPI to help improve areas, it's not just a check box exercise.
Some examples but like everything you need to put a bit of thought into how you approach these and depends on your situation:
Code test coverage
Service uptimes
Spending on tools and servers
Bug tickets raised by QA if you have them, per feature; as in fewer raised the better dev has done
Number customer bug tickets that are valid
Time to recover metrics when you have outrages or such
drakir89@reddit
As a QA dev I do not like the tickets raised KPI one bit. It has basically the same accuracy as "features" or "story points", that is there is some value if you measure the same team over time (so in this case the same QA team testing different dev team's products) and the teams are unaware this is tracked.
BaNyaaNyaa@reddit
"BTW, could you group multiple bugs into the same ticket? Thanks!"
46516481168158431985@reddit
Please don't talk to devs, raise a ticket if you have anything to report.
BackFromExile@reddit
All of these are bad metrics imho
Higher coverage = more time spent
Really depends on the company, can easily be impacted by things outside of the control of the team
Higher = better or lower = better?
Easy, just don't test anymore = obviously higher quality. Also this causes devs to test less thoroughly because they hope less things will be found.
Customers report all kinds of shit that is valid
That's basically the same and the service uptimes
roodammy44@reddit
These are some of the only things I’ve seen that makes sense to be measured.
It makes me wonder if OP is hiding a subtext that management wants more productivity and thinks it can “management by metrics” its way there.
It’s ironic that if management actually Googles that phrase, or even asked an LLM they would find out quickly why it’s such a bad idea.
Confident-Stage-8681@reddit
I’m in a very similar situation. Management doesn’t really care about the work being done, they only want KPIs so they can look good to their management. Ultimately, KPIs are being made up, because if anything falls outside the acceptable range, you’re doing a “return to green plan” which is just additional work on top of what you already have to do. I’ve never found any useful metric in my situation, so I just provide system uptime. Easy to calculate and something we care about. So far, it’s worked. Unfortunately it doesn’t solve your issue of needing the number to go up.
ugathanki@reddit
How about every time an API change breaks your code or some other external event occurs, you document that it happened and sort it by urgency and importance? Then, you can say "we spent 1234 hours last month on very urgent problems of critical importance, and 4321 hours last month on very urgent but not very important problems. We estimate that we've added about 1324 hours worth of very important, non-urgent tech debt, and about 4231 hours of non-important, non-urgent tech debt." or whatever it ends up being.
keepingtechnosafe@reddit
DORA metrics. They have been proven to be the most important even in times of AI.
Lacutis@reddit
The only KPIs that make sense for development are at a team level. Number of deploys this sprint. Percentage of deploys with a change failure. Number of total points completed during the sprint. Number of logged exceptions in a domain.
The best use of KPIs for.development is to track overarching trends across an entire team or domain.
Can these be gamed? Absolutely. Do they need to be at the team level? No. Its actually useful to see things like an uptick in exceptions in a domain, or how refactors over time lower defect rates or change failure percentages.
Individual developer metrics are often useless, can only be compared historically to the same person, and even then it doesnt account for things like different types of work being done over time, etc.
Spimflagon@reddit
I mean there's only one metric that matters. One sheet, A4, ten point font, left aligned. Just says:
Put it in a monospace font, so they know you're not dumbing down the technicalities for them.
andrei9669@reddit
I would start by defining SLOs
nomoreplsthx@reddit
What's the actual business thing that you deliver?
Engineering metrics should just be product metrics.
Shazvox@reddit
Fuck if I know. These are the things I hate about dev work.
You hire me to be a dev, then you ask me to defend the work you asked me to do? WTH?
Gusatron@reddit
Staff turnover
Icy-Roll-4044@reddit
It's funny cus this is one of the few metrics tht's actually hard to game
ContemplativeLemur@reddit
That's rarely a main leadership concern
Gusatron@reddit
That’s why I said it
sol_in_vic_tus@reddit
Vacation time used
Gunny2862@reddit
Mandatory mention of Goodheart's Law!!!!!
bbw_slayer@reddit
My company uses pensero to fire people, You can try that. I was on PIP a while back due to that but I learnt to game it
Obsidian743@reddit
DORA + SPACE
goblinspot@reddit
30+ years in sw development. 25 in management.
There are no KPIs that work that measure output. Legacy code simple tickets can be a walk in the park or a nightmare. Greenfield work can be easy or blow up once you start coding.
Things you can show: Standing bug count. New bug count. Production down time.
Biggest KPI. Are you meeting deliverable date set by your business partners? Are they delivering requirements on time?
Wonderful_Device312@reddit
Op mentioned they have a lot of churn due to external API's changing etc. I think they should probably try to capture that as well.
Probably also include vulnerability remediations.
Both of those can be big time sinks that management needs to know about as their own things rather than letting them be silently absorbed as general internally driven tech debt work.
In general though I think with any kpi its important to ask what decision it'll support, what kpi will measure the outcome of that decision (they're not always the same kpi), and if it's a continuous decision or a one time decision and therefore how much effort you put into the data collection.
Eg. If they're struggling with a lot of external API churn, quantify that and once you have the numbers decide if your dependencies need a review.
No_Flan4401@reddit
Work done, but not as some bullshit number of story points, but described with one sentence.
'avoided using 5 hours on optimization of endpoint, due to a clarification meeting with external Dave, that said they don't use our endpoint anymore, therefore reducing load by 89 percent' (ok that was a long sentence).
I truly believe and perhaps I'm stupid, that the idiot guys doing 'leadership' is trying to understand what's going on, but have zero competencies, so they resort to KPI. To battle this, increase communication and transparency so they can see, understand and be taught that software development is 100 other things than just to write code or add a new feature. Let me know if it works or else just go for PR amount and game the fuck out of it. One unit test is one PR.
Mother-Influence-815@reddit
Just show them a graph going up and to the right 📈
african_or_european@reddit
Whatever ones will annoy you least when you have to maximize them.
QuitTypical3210@reddit
Estimate tasks by days as points. Start as normal estimates. Track completion time. Estimate - completion time = amount of time saved.
Slowly begin overestimating tasks. Amount of time saved increases over time
Taco_Enjoyer3000@reddit
>Slowly begin overestimating tasks. Amount of time saved increases over time
The secret sauce
beefz0r@reddit
That's some malicious compliance that would induce a management boner for sure
fakeclown@reddit
Then they'll begin to notice estimates are inflated then demand budgeting less time for a feature. It oscillates like that at my company. At least, it's cyclic so there is breathing room.
gjionergqwebrlkbjg@reddit
Or they mark the orgs which deliver less in the same time as low performers and decrease headcount. That tends to do the job.
WhiteXHysteria@reddit
Ideally you should be able to explain why a task will take some rough amount of time.
Usually it'll be some amount of unknowns. Whether it's a legacy system that could hold surprises or it's dependent on some other thing from another team.
I generally tell my leaders a best case, realistic case, and worst case.
We hit the best case probably 25% of the time. The realistic case like 65%. Somewhere between realistic and the two edges about 10% and basically never hit the worst case. Maybe one time in my decade here.
The biggest issue we have are being owned by one person who will have an idea come in a dream and want a proof of concept the next say totally derailing things. And they don't care they are wasting money so long as their ideas are being tested. 99.9% of them are terrible but the paycheck hits the same so it's whatever lol
Canadianingermany@reddit
I mean each apobvhange has a gitlab task right?
Are you adding points estimates during grooming?
The tasks and the point estimate as well as delivered points are the basic KPIs that will make most management happy b
roodammy44@reddit
Points are arbitrary and vary between teams and even people. A 2 pointer for someone might be an 8 pointer for someone else.
gjionergqwebrlkbjg@reddit
Nobody at the level asking for those metrics will accept story points, they will ask for specific deliverables at specific times.
Canadianingermany@reddit
you and I know that story points are relatively arbitrary. But does the CEO know that?
Moreover, just because they are arbitrary, does not mean that there is no value in tracking it over time.
Gubru@reddit
That only matters if you’re comparing the output of two teams. OP is not doing that.
geeeffwhy@reddit
there was a study (that has even since been replicated!) showing a positive and significant correlation between developers happiness and public stock price.
in general, i think the best thing is to identify the business’s actual key results and figure out a proxy or chain of proxies backwards from there.
but this is fundamentally an extremely hard problem that has plagued the very best in the field forever.
LonkPNW@reddit
The numbers that I've seen that work best are those that focus on mitigating surprises or distractions from the core development work. Demonstrating that you can generate code that is more stable, easier to maintain, and gets ahead of possible security issues should give you a few good numbers to pull from.
I'll say too that an effective PM can be super helpful here, enabling you to focus on outcomes of the code produced. For example, when working on technical debt, if you can show that you're only doing the things necessary to drive the specific outcomes you're looking for, that may help. You're not fixing everything, just the issues that are causing slowdowns / dropping NPS scores / ticket volume.
Void-kun@reddit
Depends on what you're delivering but if you can break it up into epics/features/pbis/tasks etc and then give effort estimates to each, you can track the effort delivered per sprint.
If they can see what features are being delivered and the effort assigned to each they should be able to track your team velocity.
Then the only thing that matters is the work being delivered.
Illustrious_Pea_3470@reddit
Sounds like reliability metrics are reasonable in this case. How often do breakages occur? What’s the distribution of times to recovery? What’s the customer impact (in revenue) if they did occur?
doyouevencompile@reddit
Individual dev metrics are hard. All metrics are pretty easy to game. I think those metrics provide some insight, but cannot be hard metrics. If one engineer in your team is committing once a week, it's worth looking into, but maybe they were writing a design doc or doing some research or been in and out of meetings to get stakeholder / cross-team alignment.
However, I think measuring team / product efficiency is possible to some extent. The metrics you should use depends on your org structure too. I think you can answer some of them by just asking yourself what are the thing you can do in your systems to make your life better.
That's it. Never do LOC, # of commits etc. I'm personally against setting goals against high test coverage. Usually a waste of time and in my experience the worst codebases I've seen had really high coverage goals. You fix a bug by removing some code and somehow the coverage goes down. Just unit test business logic and run a lot of integration tests.
Thethubbedone@reddit
"A metric ceases to be useful when it turns into a target"
sweetnsourgrapes@reddit
Admittedly I haven't read allll the responses, but this just came to mind, sorry if already mentioned..
Get hold of some "outside your sphere" metrics and include those. Depending on what your projects are for, that could be things like
customer satisfaction (ie with the software)
increased user base (if relevant)
lowered infra costs due to increases in code efficiency (sql improvements, better caching or whatever)
further to above, calculate lowered infra costs not as absolute $$ but relative to increased usage/users
more productivity - from your users, not from your own team. E.g. if it's B2B software, search your db for signs that your customers are doing more business over time = a kpi of success for your company purely because the software is working well / better.
Stuff like that - use your imagination - that are measurable increases that reflect back on continual improvements you implement.
In other words, the message is "your KPIs are also our KPIs" to a certain extent. A company is a team working together, not silos. Your KPIs are not ONLY measurable by units of X within your immediate sphere, but also by how your contribution boosts KPIs across the business.
It's selling a different way to look at things, sure, and might sound like a copout, but I also think it's common sense.
Nofanta@reddit
That’s their job.
Upbeat-Drive-7790@reddit
Obviously they are incompetent
Upbeat-Drive-7790@reddit
KPI only really makes sense when you have 150+ engineers.
Obviously, in small teams management doesn’t need to overcomplicate things with dashboards and metrics that don’t reflect reality.
For a team of \~10 people, the only KPI that actually matters is whether the company is making revenue. If there is no revenue yet, then the closest equivalent is company valuation or overall capitalisation.
That’s the only metric that reliably reflects whether the company is going in the right direction and helps keep everyone aligned
bzBetty@reddit
dora metrics
t-tekin@reddit
Improvement over “lines of metrics”, but most of the time they are too rigid as well. They miss so many nuances.
There is the SPACE and DevEx, which are introduced as improvements on DORA. They mostly add developer sentiment and developer experience metrics. And yes they are better but still not great.
calvintiger@reddit
Tell them your primary KPI is the KPIs themselves. As in last quarter we tracked 8 KPIs, this quarter it’s 10 so 25% improvement.
PsychologicalCell928@reddit
The problem often is that the CIO doesn’t take a broad enough view of the business to define the KPIs.
Often the technology is so essential to the business that an IT issue needs to be immediately reflected in business metrics.
For example, the metric downtime is expressed at some level ( < 10 hours / week/month/quarter). However it should be expressed as costs of downtime per time period.
the system crashed during the overnight batch on a holiday weekend. Cost to the business - zero.
The system crashed repeatedly on Monday mornings - cost to the business significant.
So don’t make the KPI “number of system crashes” but rather “crashes during operational hours”.
——————-
Another key thing to do is track the cause of issues back to the source. Too often tech takes the hit for operational errors and/or poor specifications.
KPI - system downtime. Cause: business entered incorrect holiday calendar.
KPI - system failed to meet performance objectives. Cause: system specs stated a maximum of x customers and y transactions per hour. Volumes were 10x and 100y due to “holiday special” or “prices entered were entered in EUR rather than USD which caused a run”
KPI - system failed to meet customer response specifications.
Cause: Business has failed to delete obsolete /defunct customers despite metrics showing the upward growth and its effect on performance.
Sensitive-Ear-3896@reddit
Well if you’re a software company Dollars earned from the software we wrote is a pretty good one
Individual-Praline20@reddit
No matter what they are, most of them can be gamed. So use the most stupid ones. 🤷
Murky_Citron_1799@reddit
Sounds like you would benefit from categorizing your tickets as "keeping the lights on (updating existing api changes, etc)" and "new feature". That would give management insight into what proportion of your work is maintenance and you could justify investing in refactoring or other non feature initiatives to reduce your maintenance burden. It will also provide you some evidence that you aren't just being slow to release new features.
It would also help to get in the habit of trying to make each ticket roughly the same size of effort, the smaller the better. Break down big tickets into smaller ones. Then you can measure cycle time and get insight into how fast your team is moving. That allows you justify improvements to your systems to allow you to release new features faster by reducing cycle time.
MoreHuman_ThanHuman@reddit
"I don't have a good MBA, so I don't know what metrics would help you determine my value. Isn't that your job?"
SpaceMonkeyOnABike@reddit
For estimates, use a triple point analysis. Best case, expected case, and, worst case. That way you can explain most results.
Entire_Jacket_3457@reddit
What I find useful is to make a list of the top 3 to 5 business problems that you are currently working on solving. Then you put relevant KPI:s for those business problems.
For instance: Problem: Churn rate is high. Currently at 45% KPI/goal: 40% churn How to reach: - simplify onboarding flow - improve user dashboard Etc.
I would say this is how you create actual value and transparency. It also makes the team focus on the right thing (as long as your analysis/problem identification is correct of course)
Perfect-Campaign9551@reddit
Honestly I don't buy the argument that people will game the metrics. Maybe if you're young you have the energy for that? It kind of takes just as much energy to game the metrics as it does to do the work
joshocar@reddit
High SEV events and time to resolution. You need both because you will always have high SEV events, but time to resolution is important. This does a few things: it directly affect the customer, so it is important, it helps people focus on high quality code, and it drives CI/CD pipeline quality. This is also important with LLMs and how it can affect code quality.
soundman32@reddit
The problem you have is visibility, which is probably why management want KPIs. All your manager wants is to plot a pretty graph against work available and work done, to present to the higher ups.
My suggestion would be something like this (which is not agile/sprint related as you don't mention that in the post).
Make sure all tickets are fully specified. This is non-dev work but there should also be a recurring ticket to book the time against.
Make sure each ticket is roughly equivalent in terms of work (this can be hard to break down, especially to begin with)
Report how many tickets are created vs completed each period.
Then you can report X tickets created, Y tickets completed in a period.
ThlintoRatscar@reddit
Second this.
My President wants metrics when the Board asks, "What did we get for the money we invested in R&D?" and "When will we get the payout?".
They do that when I haven't done a great job making the value visible, or when other departments are struggling and everyone gets metric-ified.
DORA is the framework that has evidence correlated with profitability. SPACE is the more general purpose framework correlayed more holistically with Engineering performance in mature organisation.
Tickets are the fundamental unit of work done in both frameworks.
ragged_mirth@reddit
System uptime of your integrations and how quickly you respond to external API changes would actually show management you're on top of things without gaming the numbers.
aaaaargZombies@reddit
It would probably be a hard sell but it sounds like you want to measure to the product rather than the thing producing the product. Things like uptime, number of runtime errors, performance, etc.
SamMakesCode@reddit
The only metric I’ve had any success with is cycle time.
Measure the time it takes for tickets to go from “to do” to “complete”. Measure it over time and try to gradually increase it. On a small number of tickets, it’s worthless but over a longer time period it’s pretty good
danny4kk@reddit
This is norm a pretty good one and worth tracking regardless as the metrics are useful, I would just advise be careful adding it to official 'performance' of individuals, and keep an eye on teams a bit to ensure it's not being gamed.
I found on one team it was discovered they wouldn't pick up tickets on a Friday, they wouldn't start ticket officially until they had made some progress which can lead to overlap or someone else picking up the ticket despite someone else has technically started it. They were frequently skipping over the final bits of our DoD. Tickets that looked hard they would avoid.
That team were previously under a microscope earlier in the year for bad performance reviews and both their EM and PM replaced for bad decisions on projects. Which likely added to this worry causing this kinda of behaviour.
cooking-chef-2000@reddit
What does the cycle time realistically read? and how is it usually aggregated since tickets can come in the form of big and smaller story points?
I remember seeing a popular metric by the number of storypoints a team can complete in a sprint on average
tevs__@reddit
Dora for productivity, but I find the best metrics are related to the quality of the software, which depends on what you're building.
For example, my team does customer journeys - onboarding/exiting/renewing subscriptions - so our primary metrics are journey success (how many journeys do not require intervention) and journey duration (how long does the customer need to spend).
Neither of these directly relates to code, but it's how the business sees our success. We refactored order management to make it simpler, which halved the number of failed journeys or we'd like to spend 4 weeks fixing tech debt to reduce the numbers of failed journeys
Tieing your metrics to the business makes your team more relatable to non engineering, and makes it much easier to justify development.
MartinSch64@reddit (OP)
This is a good point, I could think of a couple of "percentage of items processed without errors" within the app, that kinda measures how well the app is working.
tevs__@reddit
Oh, and if you have a product owner - defining those metrics is really their job..
kolodaer@reddit
Are you SW Engineers ? create a good reporting algorithm that keeps the manager happy and allows them not to fuck up the workflow.
03263@reddit
Dollars of profit from your work
Esseratecades@reddit
"We are well aware that SWE KPIs usually don´t work well and usually fall to Goodhart's law resulting in a worse output."
"The goal is not really to find something that measures our output/efficiency (IMO that is not really possible), but just to satisfy management and not make us look bad or will be hard to improve on in the long term?"
Given that you're aware of the former and pursuing the latter, the actual metrics you choose don't really matter. Which ones look good for your team right now?
Does your team currently have good velocity? Do they currently have a low bug count? Do you have a low number of unplanned outages? Like you said whatever you choose is going to fall apart later anyway so it's better to just use real metrics that make you look good enough right now for management to leave you alone.
The other option(which may yield the same result) is to interrogate the following.
"We all feel like our team is working pretty efficiently and working in harmony."
Why do you feel that way? If you really are working as well as you think you are then you should be able to explain why, and logically somewhere in there will be a number you can offer.