Measuring Productivity of Engineers

I know a lot of people like to tout Results Only Work Environment (ROWE) as the future, and have all these success stories.  For those of you not in the know, ROWE is where there are no office hours or expectations of a 40-hour work week.  Instead employees are given goals to meet and as long as they are meeting those goals they can work however much they and whatever hours they want.

I…sort of agree with the premise, but there are so many nuances and caveats that any sort of pure ROWE playbook I think wouldn’t work. There are VAST differences between trying it at a small startup vs trying to implement in a large bureaucratic organization.

Maybe ROWE works at some places but at the workplace I was at it where this was tried didn’t work well as a true indicator of the value of an employee. The main reason is that it is hard to give good measurements and estimates on tasks and how that relates to what you contribute.  I was given tasks that, frankly, took heroic efforts to achieve and ended up working 70-hour weeks just to meet my baseline goals.

Let’s expand on this. Let’s say I have two devs, Joe and Bob. Let’s say Joe commits twice as much source code, his story point velocity is twice as high, and he produces half the defects (bugs) that Bob does. What does that mean?  At first glance you might plan on firing Bob, but hang on, we’ll get back to that.  From a productivity perspective does this mean I should pay Joe twice as much as Bob? Should there be a direct correlation between salary and productivity?

On one contract I worked on, there were 4 people on my team. Three of them got laid off so it was just me. I took over everyone else’s tasks, got them done without adding tons of overtime, and the customer said my work was even better than before. So….do I deserve those other people’s salaries?

What’s interesting is how ROWE coincides with the progressive management style used at the startup and/or superstar crowd.  Just lay out strategic goals, ensure a shared corporate culture and vision, and let people go.  If you’ve ever seen the Star Trek: Deep Space 9 episode “Starship Down”, there’s an interesting conversation between two characters on how to lead engineers:

O'BRIEN: Can I have a word with you, sir?
WORF: Of course.
O'BRIEN: With all due respect, I think you're riding the men a bit hard. You have to understand, they're out of their element. They're not bridge officers, they haven't been to Starfleet Academy. They're engineers. They're used to being given a problem to solve, then going out and figuring out how to do it.
WORF: What are you suggesting?
O'BRIEN: Give them a little slack. Ease up on the reins. Let them do what they're good at, and give them a little encouragement now and then.
WORF: I will take it under consideration.

This approach relies upon the premise that people honestly want to do their best, are passionate about their work, and have a lot of professional pride.  If you give these people resources and just let them go, they will go off and produce greatness.  What’s critically important though is you have to make sure you’ve clearly communicated the goals and vision, and you have to make sure everyone understands them and is honestly working in that direction.  There’s nothing like having someone work hard for six months only to discover it was some capability that no one wanted or needed.

Now here’s the issue: this only tends to work in a smaller, startup-type organization where you have a lot of control over the corporate culture and you have the ability to get rid of people who don’t fit.  Otherwise the danger is you see the effect of Jack Welch’s 20-70-10 rule.  According to Mr. Welch’s principle only the top 20% of your workforce has the talent and attitude to thrive in such an environment.  There also exists a middle chunk of 70% of people that are just there to punch a clock.  Lastly, there is a bottom 10% that is actively holding your organization back.  The trick is, you have to find a way to manage expectations with that middle 70%.

All this is contrasted with the more “traditional” style of Project Management (I tend to think of this as the PMP Way, which is not fair because I’m not a PMP), where workload and performance measurement is usually centered around tasks.  Projects are planned from the top down, by decomposing into tasks, performing time estimates, tracking percent-complete, and using Gantt charts to roll up status reporting, completion percentage, and schedule tracking.  This method is very tempting to use, especially by A-type personalities with an emphasis on quantitative process and analysis.  This style of project management can easily feed into employee performance analysis.  You can easily look at the numbers produced per employee and use that to measure “productivity”.

At first this seems much more simple, maybe even deceptively so.  Using the example from above with the two developers, Joe is “twice” as productive as Bob and thus should earn twice the incentives. But what if Bob is the guy that does all the build scripts, the puppet/chef maintenance, keeps Jira and Gitlab running, makes sure the nightly CI runs are working, etc. None of that is tracked in workflow but he’s making things better.  Or let’s say we are tracking all Bob’s tasks but Bob is really, really good at helping everyone else whiteboard their problems and is a great “rubber duck” and just in general makes everyone around him better?

The only real, true solution I’ve found to ANY of this is simple: Management by Walking Around (MBWA).  I first heard of this in The HP Way. Just keep in tune with what’s going on, are people working hard, is there too much goofing off (some goofing off is good), are we as a team moving in a positive direction?  If people are top 20%ers then great!  If they are 70%ers then try to be compassionate, positive, and supporting, and try your best to encourage, coach, and move them in the right direction.  If they’ve slipped to the bottom 10%, then start the process to get rid of them (and you might have to use the quantitative approach above for that to happen).

Bottom line: Take the long-term view.  Are you meeting your strategic objectives and milestones on a quarterly and yearly basis?  As a higher-level manager that should be more important to you than just who got what tasks done during this last sprint.  And if you actually walk around, talk to people, and see what’s going on, you’ll know which people are contributing and which ones aren’t.

2 thoughts on “Measuring Productivity of Engineers

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s