Posts
This is all good. Or, at least, this could all be good. These surveys, though, have their problems.
- Questions are often poorly worded
- Possible answers either cover too much territory or not enough territory
- Press releases summarizing survey results seem to disassociate cause and effect
- Survey populations may not be unbiased
- Surveys inevitably are constructed to produce the findings that the surveyors think should be produced
Questions are often poorly worded
This is a no-brainer, but the world at-large doesn't seem to mind. I saw a survey question recently (the group had not been bifurcated yet into people who liked versus those who disliked their consumer-driven health plan (CDHP)) that asked "What do you like best about your consumer-driven health plan?" The possible answers were something like:
a. my quality of care is higher
b. it costs me less
c. I can choose my own physician
d. it promotes a culture of wellness
My immediate reaction is to ask where is e: none of the above? Let's look at the possible answers. Anybody who says that their quality of care is higher under a CDHP must be hallucinating. What would make it higher? If you can choose your own physician, why would they provide better care when you are in a CDHP than they would under a traditional health plan?
If you say that it costs you less, I would ask you less than what. Yes, the premiums are lower than they would be in an HMO, for example. On the other hand, they are higher than they would be if you had no insurance at all. Isn't this like having a deductible on an automobile insurance policy? If you choose a higher deductible, your policy costs less. But, in the health care policy, you usually don't get to choose your deductible. And, in the case of high-deductible health plans (HDHP) which are typically the cornerstone of CDHPs, the deductible is typically higher than most people can effectively budget for.
If you say that you can choose your own physician under a CDHP, that is true, but can't you choose your own physician under any health plan? It's true that your care may not be covered by the plan, but for many people, if that is really the reason they are in a CDHP, I would say that they are quite misguided.
Do you really think that CDHPs promote a culture of wellness? If I were texting, I would reply "lol." According to a recent Aon Hewitt survey (oops, now I am citing a survey), 35% of participants in CDHPs are sacrificing medical care because they cannot afford their part of the cost under these plans and 28% are postponing it for financial reasons. FACT: that is not indicative of a culture of wellness.
Suppose I think the CDHP that I have been forced into just plain sucks. How do I answer this question?
Answers cover too much or not enough territory
In the last section, I managed to deal with answers that don't cover enough territory. Sometimes, they go the other way and cover too much.
I took a survey recently about automobiles. The survey asked me a series of questions. For each question, I was supposed to answer on a scale of 1-13, with 1 meaning I strongly disagreed and 13 meaning I strongly agreed. Come on, people, 1-13? Do they really think that ten minutes into a survey, I can rate things on that fine a scale. They asked me if I would consider buying a Lexus when I next purchase a vehicle. So, perhaps I went through a train of thought like this. Lexus makes a very good car. They are stylish, safe, high-performing, and dependable. They are also expensive. Would I consider buying one? Yes, I would probably consider it, but I really don't want to spend that much money on a car, so how strongly would I consider it? Hmm, is that a 5 or a 6 or a 7 or an 8 or a 9 or a 10? I don't know. If it was on a 1-5 scale, I could probably happily fill in the little button for a 3. But on a 1-13 scale, that would be equivalent to a 7 and I just don't know if I'm a 7 or not.
Press releases ignore cause and effect
I saw another survey (if I could find the actual survey again, I would cite it here) recently that said that fewer companies were funding (informally) their nonqualified deferred compensation (NQDC) plans. The headline said something about recent guidance on corporate-owned life insurance (COLI) being the reason for this. Hmm, the survey had no questions in it about why fewer companies were funding their NQDCs. And, further, I'm not sure what recent is, but I can't find any recent COLI guidance that would affect funding of NQDC plans. Perhaps the authors of the surevy had a bias?
Survey populations may not be unbiased
Suppose a large consulting firm does a survey. In my experience, they send the survey to a nice cross-section of large companies. Perhaps it looks something like the Fortune 200 plus all of that firms clients not in the Fortune 200 that generate at least $1 million in annual revenue for the firm. Of the companies surveyed, who do you think are the most likely to answer the survey? Could it be the consulting firm's large clients? Aren't they the ones most likely to actually open the survey? Who are least likely to answer the survey? Could it be the companies that have recently fired that large consulting firm?
Do you think that the results of this survey might be a little bit skewed? Do you care? I do.
Surveys inevitably are constructed to produce the findings that the surveyors think should be produced
Suppose you ran the health care consulting practice at a large consulting firm. Further suppose that you are a big proponent of consumerism. In fact, you have built your consulting firms health care consulting practice around CDHPs. You ask your survey group to do a survey around health care plans. You want to be able to make a bold statement in a press release that shows how wonderful CDHPs are and for all the reasons that you have been touting.
Do you think you will make sure that the questions have at least a small bias that will lead to your desired result? If the findings come back differently than you had hoped, do you think you will publish the results as is, or will you find a way to tweak the results? Will you tout the portion of the results that support your practice or will you be unbiased in how you release the survey results?
I don't need to answer those questions for you. You don't need to answer them either. We all know the reality.
These surveys ... they do have their problems.
a. my quality of care is higher
b. it costs me less
c. I can choose my own physician
d. it promotes a culture of wellness
My immediate reaction is to ask where is e: none of the above? Let's look at the possible answers. Anybody who says that their quality of care is higher under a CDHP must be hallucinating. What would make it higher? If you can choose your own physician, why would they provide better care when you are in a CDHP than they would under a traditional health plan?
If you say that it costs you less, I would ask you less than what. Yes, the premiums are lower than they would be in an HMO, for example. On the other hand, they are higher than they would be if you had no insurance at all. Isn't this like having a deductible on an automobile insurance policy? If you choose a higher deductible, your policy costs less. But, in the health care policy, you usually don't get to choose your deductible. And, in the case of high-deductible health plans (HDHP) which are typically the cornerstone of CDHPs, the deductible is typically higher than most people can effectively budget for.
If you say that you can choose your own physician under a CDHP, that is true, but can't you choose your own physician under any health plan? It's true that your care may not be covered by the plan, but for many people, if that is really the reason they are in a CDHP, I would say that they are quite misguided.
Do you really think that CDHPs promote a culture of wellness? If I were texting, I would reply "lol." According to a recent Aon Hewitt survey (oops, now I am citing a survey), 35% of participants in CDHPs are sacrificing medical care because they cannot afford their part of the cost under these plans and 28% are postponing it for financial reasons. FACT: that is not indicative of a culture of wellness.
Suppose I think the CDHP that I have been forced into just plain sucks. How do I answer this question?
Answers cover too much or not enough territory
In the last section, I managed to deal with answers that don't cover enough territory. Sometimes, they go the other way and cover too much.
I took a survey recently about automobiles. The survey asked me a series of questions. For each question, I was supposed to answer on a scale of 1-13, with 1 meaning I strongly disagreed and 13 meaning I strongly agreed. Come on, people, 1-13? Do they really think that ten minutes into a survey, I can rate things on that fine a scale. They asked me if I would consider buying a Lexus when I next purchase a vehicle. So, perhaps I went through a train of thought like this. Lexus makes a very good car. They are stylish, safe, high-performing, and dependable. They are also expensive. Would I consider buying one? Yes, I would probably consider it, but I really don't want to spend that much money on a car, so how strongly would I consider it? Hmm, is that a 5 or a 6 or a 7 or an 8 or a 9 or a 10? I don't know. If it was on a 1-5 scale, I could probably happily fill in the little button for a 3. But on a 1-13 scale, that would be equivalent to a 7 and I just don't know if I'm a 7 or not.
Press releases ignore cause and effect
I saw another survey (if I could find the actual survey again, I would cite it here) recently that said that fewer companies were funding (informally) their nonqualified deferred compensation (NQDC) plans. The headline said something about recent guidance on corporate-owned life insurance (COLI) being the reason for this. Hmm, the survey had no questions in it about why fewer companies were funding their NQDCs. And, further, I'm not sure what recent is, but I can't find any recent COLI guidance that would affect funding of NQDC plans. Perhaps the authors of the surevy had a bias?
Survey populations may not be unbiased
Suppose a large consulting firm does a survey. In my experience, they send the survey to a nice cross-section of large companies. Perhaps it looks something like the Fortune 200 plus all of that firms clients not in the Fortune 200 that generate at least $1 million in annual revenue for the firm. Of the companies surveyed, who do you think are the most likely to answer the survey? Could it be the consulting firm's large clients? Aren't they the ones most likely to actually open the survey? Who are least likely to answer the survey? Could it be the companies that have recently fired that large consulting firm?
Do you think that the results of this survey might be a little bit skewed? Do you care? I do.
Surveys inevitably are constructed to produce the findings that the surveyors think should be produced
Suppose you ran the health care consulting practice at a large consulting firm. Further suppose that you are a big proponent of consumerism. In fact, you have built your consulting firms health care consulting practice around CDHPs. You ask your survey group to do a survey around health care plans. You want to be able to make a bold statement in a press release that shows how wonderful CDHPs are and for all the reasons that you have been touting.
Do you think you will make sure that the questions have at least a small bias that will lead to your desired result? If the findings come back differently than you had hoped, do you think you will publish the results as is, or will you find a way to tweak the results? Will you tout the portion of the results that support your practice or will you be unbiased in how you release the survey results?
I don't need to answer those questions for you. You don't need to answer them either. We all know the reality.
These surveys ... they do have their problems.
No comments:
Post a Comment