The Simple Logic

Thoughts On Measuring Managers

A few friends and I get together via Skype every other week or so to talk frankly about how we manage people. My friends on these calls are all in roughly the same position as I am – five to ten years of management experience in the software industry, with various amounts of direct and indirect reports. We’re all very technical people and have no problems managing lots of computers, but managing people is a very different problem. It’s rare that more than a week goes by that I don’t end up in a management situation that I’m not sure how to deal with, and having these calls allows me to discuss my doubts, shortcomings, and successes with an empathetic peer group. We’ve been calling these conference calls The Breakfast Club, after the original meaning of the term and the influential John Hughes movie (nobody has said anything, but I think I’m Brian).

Last week I brought up the topic of measuring managers. I had just gone through review season at work and (as is typically for my group at work), we’d spent a lot of time revising our review process for individual contributors and for team leads. We spend a lot of time on reviews because reviews are important–reviews are when you not only tell someone how they are performing at work, but reward or punish them with raises, bonuses and promotions–and I think at work the group I’m in has come up with a system that sucks less than most systems I’ve been part of. The problem is that my group has now grown to the point where our system needs to measure and review people managers.

I brought up the topic of measuring managers during the last Breakfast Club call so I could hear other my friends’ thoughts on the matter. Below are some of the ideas from this phone call.

I need to define what skills makes a good manager.

At work, we track eleven different skills that we think are required for individual contributors and team leads, and we give each of these skills a weighting depending on the role. My friends had similar systems but with manager-specific skills and I think that’s what I need to do as well. Which skills are important for a manager is different for every company and for every level of management.

I haven’t defined the skills that are important for management at my work in my group yet (I’ve set aside time for this in the next few weeks), and I’m going to this with input from many of the people in my group. Any suggestions for skills that make up a good manager are appreciated.

Manager reviews, just like individual contributor reviews, need evidence.

I try to include evidence for my comments in any review, and I try to have that evidence show that some behavior was observed over time. For example, I’ll include ticket, bug, email or commit information as evidence, and I’ll include at least two data points, one from the first half of the review period and the other from the second half of the review period. You’d be surprised at how much you’ve forgotten someone has done (or not done). You’ll quickly realize that your intuition about how a review for someone should go is biased towards that person’s performance in the last month or so.

For managers, this evidence is harder to extract, as the artifacts of management are often decisions that aren’t in a formal tracking system. You’ll need to dig deeper, often in emails or meeting notes, to find the evidence that shows a manager making a decision, and you’ll need to keep on digging to find the reasons behind that decision and the outcome of that decision. Furthermore, by reviewing email threads and notes, you get an (incomplete) picture of the manager’s treatment of other people.

360 degree reviews are a good way to gather evidence, but be aware that the responses in a 360 have the same time bias I warned about before, and you should consider the motivations behind any response given in a 360.

I would go far as to say that a review that doesn’t have evidence for a majority of the comments is a poorly done review and the reviewer has failed to do his or her job.

Managers need to be evaluated in the context of their team, but that does not mean a manager’s review is based only the team’s deliverables.

Team-based deliverables should not be a manager’s only deliverables. I’ve worked in companies where everyone but the manager’s manager knows that the team is full of superstars doing great work in spite of their manager. The same team could be managed by my cat and that team would still do amazing work (there’s a point in here about knowing what kind of management is needed and when, but that’s for another day). On the other hand, a manager can make an entire team superstars by removing the barriers to the team’s effectiveness, or getting the team the resources it needs, or by clearly setting direction, or acting as a “shit umbrella” to help the team avoid distractions. On the third hand, a manager could do all those wonderful things for his team and still miss a team deliverable for a hundred other reasons.

I think the important thing about team-based deliverables is that you still need to balance the delivery of those objectives with the evaluation of the manager’s skills, and then you need to evaluate all of the evidence found during the review process.

Final Thoughts.

I’m going to spend some time reading up on manager evaluation on the Internet, and also reading many books with bad titles from the business section of the book store. What do you think about reviewing managers? How is it done where you work? If you’re an individual contributor, do you think your manager is being well-reviewed by his or her boss?