On Surveys, Data, and Other Common Misconceptions

Every so often, I feel the need to talk about things that are wrong. Well, perhaps I do that more than every so often, but I do it to varying degrees. Here I am going to focus mostly on surveys and their data. I'm not going to cite anything in particular, though, so you'll just have to trust me that  I am not making it up.

I have seen the data from lots of surveys that purport to tell us what are the most important elements to making a job desirable. I have read all sorts of things about mentoring, how green the workplace is, whether the health benefits are good, and the amount of focus put on learning and development among other things. These are all very important, but ...

When someone leaves a job voluntarily, ask them why they left. I have very rarely heard someone say they are leaving to go elsewhere because the new company has a great mentoring program, they are a green company, the health benefits are better, or the focus is on learning and development. Far more often, I have heard one of these:

• I got a big pay raise
• I hated my boss and I had to get away
• I hated my job and or the company I was working for
• I was in a dead end job
• I don't have to travel as much
• I get to travel more

What this tells me is that something about the surveys is just wrong. And, I don't think it is the inability of the people who report the results to accurately compile or report the data. I think it falls under GIGO.

GIGO, you say? What is that?

Garbage in, garbage out!

Surveys as a group, are horribly constructed. Correction, it would be an improvement on most surveys if they were horribly constructed. Most are worse than that.

If we consider job/pay/benefits related surveys, what would you think of a question like this: "How important to you are your health benefits?" Well, of course, they are very important.

So, since everyone says they are important, this gets construed as being one of the keys to hiring and keeping good employees. But, you rarely hear about employees choosing a particular employer or failing to because of the particular health benefits. The question has always been whether they offer health benefits. Very few people ask specifically what they are, how much they will pay for them, or what is included or excluded.

We could say similar things about retirement benefits. You need to have a 401(k) plan to compete. But, if it's a bad plan, nobody ever leaves specifically because of that. Yet, if you were to read survey data and reports, you would think that the quality of a 401(k) plan was a top 3 attraction and retention tool. I will say, however, that employees will think twice before leaving a company with a great retirement program, especially if they have a defined benefit pension plan. It is a retention tool and it is a differentiator.

Okay, rant over for the day.