Are there any good KPIs for individual developers on small teams?
Posted by tyler_church@reddit | ExperiencedDevs | View on Reddit | 108 comments
I am the lead developer at a small (\~10 people) consulting company. We provide niche software for large companies as well as other services. The development team consists of myself, 1 other senior dev, and 1 junior.
My manager is pushing for individual KPIs to use as goals for the development team, but I’m at a loss for coming up with anything meaningful.
I’ve already explained why “lines of code written” is a bad metric.
“% of items delivered within estimated hours” seems less bad, but still not right: our estimates aren’t meant to be that precise. I crunched the numbers and big surprise: the junior is slower than the seniors. I don’t think shoving this metric in our faces will lead to improved performance.
Are there any metrics that serve as good goals for individual developers?
kenflingnor@reddit
Your manager, and whatever other leadership are involved in this conversation, should be prepared for any individual metric to be gamed. I'm being measured by delivering against my estimates? My estimates will now grow by 3x. I'm being measured by commits? Every small change I make is now going to be a single commit. So on and so forth.
darkstar3333@reddit
I always find this hilarious because its very true.
We as outsiders want to impose a systemized metric on people who are typically incredibly good at critically dissecting and understanding complex systems.
eltee27@reddit
Not that I disagree with any of this, but come compensation time how does a manager figure out if someone is deserving of a raise and by how much?
tyler_church@reddit (OP)
I think it's weird that you're downvoted because this is a valid question. How does a non-technical person decide whether a programmer is doing a good job? It's a question a manager has to answer, one way or another.
mjm65@reddit
I thought that was the whole concept behind SMART goals. You and your manager work throughout the year to set expectations based on the role and what needs to be delivered.
eltee27@reddit
The M on smart stands for measurable which people seem to be vehemently against 😅
Stealth528@reddit
Programmers shouldn't be reporting to a non technical person. Of course it's not going to be possible for a person who fundamentally does not understand my job to accurately assess my performance.
Vector-Zero@reddit
A good manager should be able to understand the impact of his employees. Employees can write a paragraph or two to summarize their biggest contributions to remind management, and compensation can be allocated to favor the most impactful engineers, possibly skewed in favor of good junior engineers for repention's sake.
Izacus@reddit
The reality a lot of folks won't admit on this sub (because they weren't managers) is... managers and promo commitees do look at those metrics. And use them for promos and bonuses.
superdurszlak@reddit
I've actually seen the latter in action.
Developers at one company literally stopped doing squash & merge in favor of merging all the "fix final fix" kind of commit madness as-is, because the CTO decided to measure individual performance by number of commits they made.
Big_Function_N1@reddit
Part of our KPI is how many bugs are found in production. A lot of incidents turn into new tasks instead
TheOnceAndFutureDoug@reddit
Remember when they used to do it as lines of code committed? Every function gets a detailed comment describing it, nothing is chained, line separate every function and variable...
SeniorIdiot@reddit
It's amazing how so many "leaders" have never heard about Goodhart's law.
Sparkly-Sparrow-6893@reddit
I worked for a consulting firm developing relatively niche software for a large industry group, either alone or with a couple other developers. The best metric was always whether people are complaining - i.e., whether the clients were happy. This was a much harder metric to satisfy than any software engineering metric, because making the clients happy meant many late nights solving their problems.
nomoreplsthx@reddit
Individual KPIs is kind of a contradiction in terms. KPIs are organizational by definition.
Regular_Tailor@reddit
Don't let them bill per feature. Time and materials, never worry about this again.
KPIs: test percentage above x% Points per sprint+/-10% of your own average after 3 sprints Pushes work twice a day.
Just make them binary and achievable, but not rankable so everyone on the team does best practices and gets a gold star.
rwparris2@reddit
Everyone keeps saying team metrics are better.
Outside of DORA, what team metrics do you all find useful?
UniForceMusic@reddit
Code that stood the test of time metric? The less refactors a certain part gets, the higher the metrics for that developer?
iamgrzegorz@reddit
The question is: do you want you developers to work as a team towards a common goal or work individually on their metrics? Because if you set in individual KPIs they will stop caring about the team goals.
konm123@reddit
This. If you set up individual metrics, foremost, you'll get that developers work for their metrics. Soon, they'll figure out what it takes for their metrics to show up good.
darkstar3333@reddit
This. Development is a team sport, if you were to compensate a sports team based on individual KPIs the team would cease to operate effectively.
You need both contributors and multipliers that build upon a shared goal and outcome.
If you give the team true autonomy with true responsibility (and repercussions) they'll typically self support, reinforce and grow together.
Izacus@reddit
Whats funny about post is the fact that every team sport will track individual metrics for each player and rank them separately as well.
So... maybe not the analogy that's useful here? ;)
fallingfruit@reddit
It's not but it's because stats in sports are very hard to pad or game in a significant way, and because stats are actually directly correlated with a simple measurable goal to win games by score
GrizzRich@reddit
That's not necessarily true for all sports btw. Baseball was easily quantified. Games with more variable movement are less easily quantified.
Izacus@reddit
They're equally easy to pad just like the developer metrics are. The difference is that the managers/coaches won't stand for that crap :)
darkstar3333@reddit
Do stats exist to win games or fuel sports betting industries?
If we could bet on Joan's velocity this sprint, would it mean something?
mmbepis@reddit
Not to mention many professional sports leagues have monetary bonuses tied to individual stats. It's definitely easier to correlate goals or points to team success than lines of code or really any other software development metric so the analogy isn't 1 to 1
bluetrust@reddit
It's an interesting comment, which I agree with.
But I've been following the Kraken NHL team for the past couple years. It really seems like during contract negotiations, the offensive players generally get about $100k a point that they scored the last season. So 60 points (goals + assists) = $6M/year contract. I'd expect given what I know of incentives then for that to result in a lot of selfish behavior, but I really don't see much of that. Maybe making money isn't their primary motivation?
Hairy_Garbage_6941@reddit
Because the selfish guys don’t get ice time to get the points.
bluetrust@reddit
Oooh, good point. The kraken "healthy scratched" (I.e., took out of the line up) a number of players last year.
Izacus@reddit
In reality there's plenty of selfish "top players" in sports which hog accolades and even make their teams lose. It's like... one of the most common sports stories.
Even Ted Lasso had a character like that :D
darkstar3333@reddit
Software development is filled with humans? Who knew?
Behaviour is behaviour, nothing here is new
EvilCodeQueen@reddit
Yes, but a goal is worth the same as an assist.
TheOnceAndFutureDoug@reddit
Remember when Microsoft had a bonus pool that wasn't enough for everyone to get their full bonuses so they told people that only the top performers would get their full bonus, thinking it would incentivize everyone to compete to be better and instead people started sabotaging other teams and their own teammates to be a "better performer" by comparison?
Seriously, can we stop letting business degrees run shit? They clearly have no idea what they're doing.
snorktacular@reddit
People who are pretty much only motivated by competition straight up don't understand how to incentivize people when success heavily depends on collaboration. It's just alien to them, unless it looks like nepotism.
msamprz@reddit
How do you do it?
I'm not trying to be a smartass, I genuinely want the thread to continue, because you've articulated your complaint well! So I'd like to hear your articulated solution as well!
noonemustknowmysecre@reddit
How do we fix that? We don't let business people run the software team.
How do we run the software team? We get marching orders from the business people, we tell them when it'll get done, we break it down into tasks, and assign the tasks to the devs.
How do we "incentivize" software developers? Largely I don't, we are professional engineers. You need to really accept everything in this video as fact. I tell people to go do things and they go do it. They're not children that need a little cake at the end of the day. If they're grumpy about pay and ask, I give them all the advice I can about how to go get a higher paycheck. I am FIRMLY on the worker's side of the labor dispute here because I am a worker. I don't own the company. Fuck you, pay me. But I get paid plenty, I'm fine.
Really, go understand that RSA video. Self-direction, autonomny, enriching their skills, authority to go fix shit, purpose, and a plan about how we're going to make stuff better.
TheOnceAndFutureDoug@reddit
The frustrating thing is it's all the things you'd expect:
Basically, respect people and treat them fairly and people tend to work well. Not everyone, not all the time, but most people. And the ones who don't? There's a huge pool of great talent in the world. Find those people.
MacroProcessor@reddit
A lot of the replies here are reminding me of Godwin's law (https://en.wikipedia.org/wiki/Goodhart%27s_law). Basically, as soon as a measure becomes a target, it ceases to be a good measure because people game the system. Something I wish more people understood when discussing KPIs.
KPIs aren't necessarily bad -- if your KPI is something like "low number of production failures", then that's kind of a good thing, even if people "game" to it. Though it's still complicated.
No-Economics-8239@reddit
1) Delivers quality code in a reasonable time
2) Decent enough person to work with
3) Care and concern is commiserate with compensation
Oh, those metrics are hard to measure and quantify? Yes. That is what makes them useful metrics. If it helps, you can tell your manager to use t-shirt sizes as a measurement.
Dearest-Sunflower@reddit
hilarious! thanks for the chuckle lol
Due_Helicopter6084@reddit
MPH, money per hour.
CautiousRice@reddit
Well, is the project shipped on time? Did it bloat out of proportion?
noonemustknowmysecre@reddit
Does it work, can you prove it works, is it maintainable, how long will the next feature take, when something breaks how easy is it to fix, do you know when it breaks, can you train the new guy on it, are there sections new guys aren't allowed to touch, is it possible to replace a library, which libraries, how easy is it to replace any given library or tool or process, does anyone know how to do that, does more than one person know how to do that, is it secure, can you prove it's secure, how could you possibly prove it's secure don't bullshit me, is it documented, how well is it documented, how up to date is that documentation, is there a spec, does it match the spec, is there a buglist, is that list maintained, is that list growing or shrinking, are you hiding bugs, does it take up too much memory, does it take up too much processing, does it take up too much bandwidth. This list is not exhaustive.
I can make any of those look good at the expense of others.
CautiousRice@reddit
yep. The idea is that most objective metrics, like PRs, test coverage, LoC are completely meaningless, especially in the AI era where you can fake them.
But it's difficult to fake it when the project is not shipped at all, or the bug queue is only getting bigger. And, as you point out, it is particularly shitty when the system is hacked.
noonemustknowmysecre@reddit
HAhahahahaha, oooooh sweet summer child...
Also easy. What bugs? Queue? What queue?
CautiousRice@reddit
didn't cross my mind that a small team may not use bug tracking, version control and such luxuries of life.
noonemustknowmysecre@reddit
aww geeze, you're still not getting it. The bug queue doesn't get bigger if you just ignore the bugs. If the metric by which coders are judged (and get raises based on) is the bug queue getting bigger, that'll absolutely affect what gets entered into the bug queue.
You said all that's difficult to fake? c'mon.
CautiousRice@reddit
😵
Mr_Willkins@reddit
KPIs are bullshit, a complete waste of busywork invented by management consultants, I've never seen them make any meaningful difference. If you have to use them, do the minimum you can get away with to maintain the charade.
EffectiveLong@reddit
For? As long as your customers happy and you are making money, stop with these fugazi. Or maybe start with those two metrics first before doing anything else
nazbot@reddit
Committed stories vs delivered stories is only useful one I’ve found.
cachemonet0x0cf6619@reddit
i like dora because it correctly places the metrics on outcomes.
Deployment Frequency—How often an organization successfully releases to production Lead Time for Changes—The amount of time it takes a commit to get into production Change Failure Rate—The percentage of deployments causing a failure in production Time to Restore Service—How long it takes an organization to recover from a failure in production
Arkarant@reddit
Deployment frequency is kinda useless tho, there's no gain from deploying on every single thing over a bigger thing
cachemonet0x0cf6619@reddit
you must be reading that as deploy frequently and this is the wrong way to think about it. frequency is the rate at witch a vibration happens. you decide this as a team according to what make sense.
Embarrassed_Quit_450@reddit
No. Team objectives are the way to go.
Any-Neat5158@reddit
In my 12 years doing this, these systems are almost always done poorly. Even when they are done well it's still basically bullshit.
The first taste of I had was so bad that my "objectives" had nothing to do with my day to day activities. And they tangibly had nothing to do what the company was doing either...
Take this Pluralsight course. Get this cert. Do this lunch and learn session. I could knock my performance review objects clear out of the park and absolutely suck at my day to day work. I point this out to my manager two years in a row before things started to actually change.
The issue is it's hard to objectively measure what we do. Customer had this problem. We fixed it. Because of that, this customer didn't leave and we signed 5 more.
Lines of code, story points, PR's, commits... none of those metrics can be properly assessed without context.
I spent two weeks once tracking down an extremely intermittent issue with a SQL scheduled job that was basically running a few sprocs. They were very large, very complicated sprocs. It took 10 days to find it, 2 days to effectively solution it and about 30 minutes to implement the solution. Less than 100 lines of code when it was all said and done. But the impact was tremendous.
To look at that and say "oh... they only wrote 100 lines of code for 12 days worth of work?"
Sure. But I fixed an ongoing issue that's been causing pain to our users for months now and that other engineers have looked into and were unable to fix.
Upstairs_Passion_345@reddit
Your Boss should not be your boss any more, you should become his boss. Micro managing such a small team (KPIs lead to actions) is awfully wrong.
sol_in_vic_tus@reddit
What I suggested in the past was vacation days taken, with more being better. It's a good indicator of a functioning team if people are actually able to take vacation time. It's also a worthwhile goal to manage individuals toward for long term success.
Strangely, managers never seem to want to use it.
eldreth@reddit
No
LogicRaven_@reddit
Relevant read explaining the no: https://newsletter.pragmaticengineer.com/p/measuring-developer-productivity
Sheldor5@reddit
McKinsey ... destroyer of any meaningful in any company they touch
this company is the most incompetent shit I have ever seen, their ideas are beyond stupid and highly destructive to any functioning team ... I think they are just hired to make the environment so shit that people quit on their own instead of being fired ... and of course only the better colleagues quit because they know they will find a new job without issues
autodialerbroken116@reddit
That's a bold take. And I too, have some qualms about the high end consulting giants.
If you wouldn't mind elaborating? I haven't worked with a large consulting group personally in my time as an engineer. Could you explain what typically comes around as actionable from a typical consulting gig?
I've heard about the horrors of contracting Boston Consulting Group in pharma, as well as "Bioteam", which are supposed to be "big genomics data" powerhouses but in reality seems underwhelming.
Would love to hear an informed take on McKinsey
teslas_love_pigeon@reddit
Read the book "When McKinsey Comes to Town."
https://www.goodreads.com/book/show/60644838-when-mckinsey-comes-to-town
This is a company where the executive board should face the death penalty for crimes against humanity.
Sheldor5@reddit
I don't know what the actual goal was but my then employer (3k employees) hired them and they re-organized everything.
I was working in access management and everybody was doing everything: build, deploy, maintain with root access to everything so the whole application lifecycle and we were responsible for the services we built
they split up the team into development (build and bugfixing) and operations (deploy and report issues) and as you can imagine maintaining became a nightmare, operations had no idea about the internals of the applications and we no longer were able to look into logs/databases to analyse issues
all processes became extremely inefficient, slow and tedious, within 3 months most of the good people quit so I, with 3 yoe and 1,5 years in this company, became the oldest and most experienced developer in the new department and inherited most of the services from the people who quit
and of course zero rais for the increased responsibilities and mentoring 2 additional juniors
so I quit a couple of months later too
autodialerbroken116@reddit
Hmmm...of course you've seen other teams organize into development teams, testing teams, and operations teams online. I'm not saying it was your team or managements fault, but would you say that the advice on implementing the shift and coordinating activities was poorly planned on McKinseys side?
Sheldor5@reddit
there was no communication at all ... one day our team leader (who also had no info until this day) shared the "news" and the effective date which were coming from further up
tyler_church@reddit (OP)
This was a surprisingly good article, thank you.
TheOnceAndFutureDoug@reddit
Every time I've ever seen a team do this one of two things happen:
586WingsFan@reddit
/thread. Seriously, do not discuss this any more
detroitsongbird@reddit
The only thing that really matters:
Is the team delivering quality features that meet the business goals?
Are the processes documented, followed, and with as few roadblocks as possible?
Is the team keeping the bug count down, keeping the code current (dependencies, CVEs, etc)?
Are things automated? IoC, regression tests run as part of the build pipeline?
If not set reasonable goals that the teams, minus management, agrees on.
Then get management buy-in on the goals.
X percent of time will be spent keeping the code and stack current.
Y percent will be spent keeping up with bugs.
So yes, in the end about 1:2 of a dev team’s time is for new stuff, the rest is for the above.
saposapot@reddit
No. The only KPIs that sometimes (rarely) work are team KPIs. Even those are very hard to get right and have them actually mean team performance.
Also, never tie those KPIs to raises or individual performance review. Use them to measure team health and maybe when you do some changes to measure if they improved things
apartment-seeker@reddit
Who is your "manager" here? The CEO?
tyler_church@reddit (OP)
The organizational structure is a little weird (jointly owned partnership), but I think CEO is close enough to accurate. My boss knows enough about programming to be dangerous, but he's not a CTO-type: he never managed any developers until they started working with me a few years ago.
bulbishNYC@reddit
Reporting to a non-technical boss always sucks. He measures you by how fast you push out features. That limits your technical growth because eventually you will be sitting on a pile of undocumented spaghetti code putting out dumpster fires all day. You don't learn proper automated testing, CI/CD, clean modular code practices - it works? - push it out and start on the next thing.
skeletal88@reddit
For consulting companies the most important metric is how happy the customer is with your work. Sounds cheesy but it is true.
Lines of code written, number of bugs fixed, anything is meaningless, for a consultancy it is the most important what the customer thinks of your work, so you need to get some feedback from them and casually or formally present it to your managers.
Another question is why are there only 3 developers in a 10 person consultancy. What are the others doing exactly? I have worked at consultancies and like.. 90% of us were developers.
tyler_church@reddit (OP)
We do a lot of work around helping companies comply with various complex environmental laws/regulations. We've got lawyers, analysts modeling pollution, admin staff, etc. The software side is new-ish, and is starting to grow.
canderson180@reddit
Say Do ratio +/- 10% if you can do it without it feeling like a crushing weight, then work with the individual to gauge how they don’t over or under commit. Unless everyone is a clone, they will have different velocities. We do velocity at the team level and then individual KPIs are tailored to individuals. Have you tried talking to the individuals to see what they think?
Does everyone know their definition of success and how their outputs contribute to the greater good?
tyler_church@reddit (OP)
Main idea was # of items shipped to prod per week, but it seems like this would just encourage a ton of items. Not entirely, some of our tickets are probably too large/coarse-grained right now.
canderson180@reddit
Yeah that’s a real challenge that might also require the rest of the org to be a bit more agile. Can a single item be small enough to be delivered in a reasonable amount of time while still providing value to end users or stakeholders?
It can be a struggle, but try to define KPIs that are specific to individuals to help them grow, while having team KPIs that actually roll up to org strategic KPIs
tyler_church@reddit (OP)
Yes, actually. I'd say our median item size is 3-7 hours of work, and we have CI/CD humming along so we deploy to prod almost every day.
Definitely a struggle, that's why I'm here :) I've been trying to find one magic KPI that works for all developers, but individualizing them sounds like a better approach now that you mention it.
bulbishNYC@reddit
I learned to never accept any 'lead' position in a company with dumb managers. See, they read 'lead' as essentially an engineer manager expecting you to provide reports, micromanage and drive deadlines like Sales department manager.
DogmaSychroniser@reddit
I'd say that you should challenge people on their estimates and give them a KPI to keep within them 75% of the time.
Probably gonna get tarred and feathered for this opinion but it's a good thing to be to be the guy who says shit, gets it done.
No_Illustrator2090@reddit
Sure, unless the whome team starts overestimating - which they will.
forgottenHedgehog@reddit
That's why on a higher level you are comparing the value of what each team delivers and base performance and future growth on that. The game doesn't end at that level, if you barely deliver anything you will end up being cut.
DogmaSychroniser@reddit
Eh I figure it's still a more reasonable KPI than many that get bandied about
tyler_church@reddit (OP)
Yeah the prevailing opinion on estimation in this sub is weird. Accurate estimation is possible. I spent years as a freelancer and if I told every client "it's impossible to say how much this will cost in time or money" I never would've gotten contracts.
Longjumping-Ad8775@reddit
The amount of money brought in by the customer and your company.
Aveheuzed@reddit
I just take the occasion to remind you of Goodhart's law:
Think about it, and consider how your manager wants to use those KPIs. Then maybe suggest some measurements (even basic ones like LoC), but make sure they are interpreted properly and not used raw. A diminution in LoC may be woth looking into, but it's not good or bad of itself.
cballowe@reddit
Most that I can think of tie to team level expectations rather than individual level expectations. How well are they lifting up others and sharing the team load.
On a team you may want something like "resolves their fair share of customer escalations" - if 5 come in and everybody takes 1 or 2, great ... If one person does all 5 because everybody else avoids them, less great for the team. If one person jumps on all 5, how is the rest of their work going? Are they acting like a hero/risking burn out/etc?
When someone is asked for a code review, how long do they take to get to that and get a first round of comments out? (Measure in business days, not finer granularity - some people jump on interrupt tasks like that, others aim for a much more structured "review at the start and end of the day" and neither approach is necessarily better).
If you have an on-call rotation "responded to pages within SLA while on-call".
Vowly122@reddit
Individualität KPI does Not make any sense. KPI usually is a strategic metric and should hold value for any part of the business. No one cares how many pr you push, but how many features a team can handle in a given time. You manager should really read some books, he seems uneducated
a_reply_to_a_post@reddit
individual metrics are gonna get gamed...
DORA metrics were in fashion a few years ago, but that is general team performance, and not meant to measure individuals on the team
Chemical-Plankton420@reddit
Count the bags under their eyes.
BCBenji1@reddit
Goodhart's Law
79215185-1feb-44c6@reddit
I use # of slack messages sent.
Those who write the most in slack are the most willing to communicate with others which is one of the best identifiers I have.
RobertKerans@reddit
👍
79215185-1feb-44c6@reddit
You know the real lazies won't even react right and that you're basically proving my point? Your reactions are actual communication unlike what I with some people who may show up for standup and then become MIA for the rest of the day.
RobertKerans@reddit
I know I know I just thought it was funny. I do agree with you I think: however, very likely to immediately fall into same issue as every other target and immediately becomes useless as a perf measure (it's probably the easiest thing to game out of anything)
RobertKerans@reddit
🎉
RobertKerans@reddit
🤞
Rain-And-Coffee@reddit
At my company developers setup a quarterly career plan that details at a high level what they will work on.
The goals are in SMART format, whip are supposed to be measurable.
Its main point is to drive conversation during review sessions.
noonemustknowmysecre@reddit
1) Is the business making money.
liquidpele@reddit
In short... no.
This is why technical managers are required for a healthy org, the managers have to know enough to see who's doing what, and what level/speed the work is at. If you have non-technical managers, then there is no way to measure productivity in a way that won't give the wrong incentives - so they often just push RTO and hope that you being stuck in the office means you'll get some shit done, or if they're completely unqualified then they'll push for insane things like lines of code .
darkstar3333@reddit
The one question/counter that has always worked for me
"What are a few good metric to determine the quality and accuracy of a user story?"
More often than not, requests are so full of holes that most teams have to piece things together to guess at what is being asked.
People confuse communicated scope with non communicated expectations all of the time.
PredictableChaos@reddit
"When a measure becomes a target, it ceases to be a good measure"
swansandthings@reddit
KPIs are useful for high level business goals. Are we growing market share and revenue? Do we have unique features? Do we have need more feature gaps? Do our customers like us.
Their usefulness at a high level makes mediocre minds assume that they can be used to evaluate individual performance, but they probably can't for autonomous knowledge work (or any work more complex than lower skilled variants assembly line productions.)
dbxp@reddit
I don't think there's ones which can cover everything but there are indicators you can track ie if one dev has twice as many bugs raised on their stories as everyone else or number of blockers resolved for other team members. I think they have to be used in combination with good management though, they should be alerts that something needs looking at not targets.
0vl223@reddit
Take a stupid one where gaming it won't hurt the team. Stay away from anything related to estinates, hours or lines of code.