I just realized this morning that I hate surveys. They tend to be as impersonal and non-conversational as a TV signal — even when a human being is conducting the survey in person. They always see me as part of a group rather than as an individual (which is how each of us feels our needs). They always make assumptions (about me, about what I might want, about what I belong to) that range from slightly-off to outright-wrong. And they always lead to conclusions that represent neither me nor the population in which I am being grouped.
I don’t doubt or deny that surveys do a lot of good. But only in the context of a marketplace where vendors alone bear the full responsibility for relating to customers. Once we, as customers, get tools that let us educate vendors personally, many surveys will become unnecessary. One way we can gauge the success of VRM is by watching the number of surveys decline.
Thought: Some of the best survey questions are the ones that never get asked because sales and marketing impulses override knowledge that the customer would certainly say “no”.
One example is a global customer preference not to hear a sales pitch, “you can go to our website” or “our menu has changed”.
The ability to express global preferences is high on my VRM wish list.
I got a customer satisfaction survey call from the company I was already in the middle of a service issue with this week. Had they known that, perhaps they might not have called, or better yet, called not with a survey, but an apology or an explanation as to why I had to wait all day Saturday for their service tech to not show up, or for that matter, call to say he wasn’t going to.
The thing about surveys like the one this poor person was conducting was that they don’t reflect any kind of actual interest in how you are I personally feel about their product or services, but are just hoping instead to get some kind of effectiveness measure of whatever management program they may recently have initiated, and against which various performance metrics are being assessed internally.
My menu has already changed.
Good suggestion for a “global customer preference.” If customers could do unto companies as they have done to them, perhaps a call from one to our home phone would be met with, “Please listen to the following options as my menu has changed. To pitch a new service, press 1. To inquire about billing, press 2. To request a survey, press 3. And don’t forget to visit my web site, where you can get a better idea of my preferences and needs, and can therefore offer a truly valuable product.”
The problem with a typical survey is that it is insular unto itself — it has no relationship to anything outside of itself. Thus survey questions tend to be dumb questions, like when you are sent a survey after you have complained to customer service that you are unhappy. Invariably the first question on the survey is “Are you happy with us?” Duh! I just told you that I’m unhappy.
Interesting that this comes up now. I just reworked MyMindshare, throwing out the whole “select interests” system because I hate selecting interests and it is entirely inadeqate as a means of expressing interests.
I have replaced it with a simple question/answer system that is like a survey, but the difference is that we learn from the answers and then ask smarter questions. Also, it values answers just like it values mindshare.
If I wanted to make MyMindshare VRM-compliant, what would I need to do? Are there guidelines anywhere?
Would surveys still suck if it were mostly vendors that got surveyed instead of customers?
Nope. That’s the idea. 🙂
Sorry to be the boring, disagreeing guy on the thread, but in addition to a supporter of the current trend of increasingly symetric relations, I happen to be a charted statistician, and I can’t agree with you. I’ve save much hassle and efforts with properly designed surveys, and I beleive that:
– what you are attacking are poorly designed surveys; on of our professor left us with an easy-to-say all-encompassing advice: “The surveyees need to feel listened to; otherwise, not only he won’t be happy, but he won’t answer properly.”
– wide, quantitative approaches are never enough, but usefully compliment open-surveys; actual data can be great (you woudln’t beleive how much hidden truths lie in the length of a phone conversation).
Surveys have helped people make decision—they now need to help people understand decisions.
I received tons of marketing calls (I hate those; hopefully, they are illegal here) and even more surveys. Each and everytime, I explain that my job is to design those, and to explain each problem to the poor clueless student on the other side of the line. They tell me they can record the call, and that the managers will listen to my tape–can’t tell if it is true, but that’s my call to educate more people.
Big issues number one and two so far:
– “mapping” questions, i.e. asking the same five pointless “Do you like that aspect . . . ? A lot, not so much, etc.” for several cases: it is mind-numbing, and the grade-sticking have you miss the problems;
– Not trying to figure out the consumer point of view first: how many times the questions are filled with marketing Mumbo-jumbo! How many times the product manager felt like forcing his distinctions into infite threads of questions!
I’m a big supporter of the semi-directive question:
– ask anything open “What do you think of . . . ?”;
– if the surveyee doesn’t have an idea, offer some leads;
– listen, ask for short clarification, and try to have those match predefined “boxes”;
– take note of what the surveyee has to say about such boxes, especially their relevance;
– add new boxes if necessary; change them rapidly if they are off-the-mark .
It takes more time, and demands clever surveyors; but why pay less to have something that is worth zilch?
That’s excellent cricicism, and very informative. Thanks.
Do you think poorly designed and performed surveys are themselves symptoms of asymmetric relations frameworks?
I understand your reticence about surveys in general – and specifically in the context of the SAP Web 2.0 survey to which you responded.
Though I respect everyone involved it does seem that many are running to the far walls of a big room in this discussion. Surveys – in any direction – are not inherently good or bad. They are tools. Even among surveying techniques there are many variations from discussion guides to focus groups to quantitative methods.
There are definitely asymmetric relations in any question and answer framework. Usually its as simple as the fact that power accretes to the one asking the questions! As Bertil points out above the quality of the exchange is as much about the quality of the questions and the purpose of the surveyor as about the success or failure of unequal power relationships.
But I would have to agree that inexperienced surveyors protected by a large power imbalance tend to produce poorly designed surveys more often. There should be a license to survey.
Dear Doc,
Many market researchers like myself have read the Cluetrain Manifesto and really value your opinion. While I agree that there are too many poorly designed surveys out there, I also believe that quantitative surveys might be the only scalable way to listen “correctly” to customers. I say scalable, because it would be hypocritical to pretend having authentic conversations with millions of customers. Of course it is important to listen to the blogosphere and other social media, but spontaneous insights from volunteers (bloggers) do not always represent the majority of what customers think, hence the need to use statistical samples of customers in quantitative surveys in addition.
So, here is my question: what could we do to reduce our “survey suckage”?
Respectfully,
Olivier
First of all, suckage is not a word. Omitting combinations of letters that are not words might be one start to improving your surveys, to say nothing of your sentences. Quantitiative surveys are the only “correct” way to listen to customers? What about focus groups? Non-qualitative and not measured by some collection of “scaled” variables. Certainly a valid way to get information from consumers you would never get from a droll quantitative survey.
Too often marketing people throw in a lot of unrelated survey questions they are interested in, but which have little relevance to the issue being studied at hand. This is especially true in claims work, and it can be very damaging to the claim.
“First of all, suckage is not a word.”
What is that trash? “Astronaut” was not a word until it was a word. We all understand what Olivier is saying…why get nasty about the syntax; it dilutes anything else you have to say when you do so.
Agreed, Kurt.
One grace of English is that it’s an open, inclusive, adaptive language. It grows like a snowball. Thanks to that grace, suckage is now a word.
Surveys are a nightmare for me everywhere you go someone is wanting you to complete this or that survey. The company I work for gages your work on these stupied surveys. You go into this (HUGE Company ) not knowing if you are going to have a job when you leave based on a survey that some mono tone person has called no doubt agravated to be grilled about how their service was today. God forbid they give you a 8 on a scale of 1-10 . 8’s are unacceptable which puts a target on your head like a bulls eye. Your threathen with all kinds of stuff . My job is based on me getting a 9 or 10 on a survey not on my ability to perform my job. How fair is this I tell you When I get surved I give the highest score so some hard working doesnt get a penalty or loss their job because they have some 10 bosses on her back . People if we hate the surveys and the companys who demand them just give the poor reps 10’s or the highest score you can to keep the Amercians employed . I think we should protest these stupied surveys and start basing a employee on the job they do not if a customer has a bad day and hates your company. I cant help half the USA hates your company its not your employees its your GREED>