A biased survey question is one whose wording, answer options or position in the survey pushes people toward a particular answer, so what you collect is your own opinion echoed back. Leading questions are the most common kind. Biased: “How much did you enjoy our new checkout?” (it assumes they enjoyed it). Neutral: “How did the new checkout work for you?” The fix is usually one adjective out and one follow-up in. Below: ten types of biased questions, each with a bad example and a neutral rewrite, a fix-it table of 12, 24 neutral questions with their follow-ups, and a checklist for reviewing your own survey.
The polite yes would have shipped the bug. The neutral question got “fine”; the probe found the field.
People agree with a leading question to be polite, and the survey records the agreement as a fact. Each example below names the leading question and the yes it earned, then shows the neutral question, the short first answer it gets, and the follow-up that finds the real one.
The polite yes would have shipped the bug. The neutral question got “fine”; the probe found the field.
Agreement isn't evidence. The neutral question got “a bit”; the probe showed the change moved a meeting instead of removing one.
The statement inside the leading question set the answer. Behavior says alerts, not reports, is the page to invest in.
Each card has the bias, why it skews the answer, a bad example and the neutral rewrite. The first six are wording problems; the last four hide in the answer options, the order and the person answering.
The wording hints at the answer you're hoping for, usually through an adjective (“easy,” “new,” “excellent”), a tag (“right?”, “don't you think?”) or a statement placed before the question (“Most users tell us…”). Cut the adjective and ask what happened.
The question assumes something about the person that may not be true, so any answer confirms the premise. Check the premise first, or add “if at all.”
Two questions joined by “and” or “or.” Someone happy with the price and unhappy with the quality has no honest answer. Split it.
The question takes for granted that an event happened or a feature was used. People who never had the experience guess or skip.
“Always,” “never” and “every” force a no, because almost nothing is always. Ask for a count over a set period instead.
Internal names and technical terms mean nothing to the person answering, so they answer something else or skip. Describe the thing, not the feature.
More positive options than negative, or no midpoint, and the average drifts up on its own. Use as many negative points as positive, a midpoint, and “not applicable” when it might not apply.
The question before this one colors the answer. Three questions about an outage, then “How satisfied are you overall?” measures the outage. Put general questions before specific ones and sensitive ones later.
A statement with agree/disagree invites agreement, because agreeing is easier than objecting. Turn the statement into a question, or make it open.
People report the behavior they think they should have. Ask about the last specific time, not the habit, and make it safe to say the unflattering thing.
Both are biased, and the fix is different. A leading question is fixed by its wording; a loaded question is fixed by checking its premise.
The third column is the part other lists leave out. A neutral question on its own often gets a short answer; the follow-up is what makes it worth asking. Copy the middle and right columns.
| Biased question | Neutral rewrite | The follow-up to ask instead |
|---|---|---|
| “How much did you enjoy the event?” | How did the event go for you? | Which hour would you skip next time? |
| “How satisfied are you with our fast, friendly support?” | How satisfied are you with support? | Think of your last request. What took longest? |
| “Wouldn't you agree the app is easy to use?” | How easy or hard is the app to use? | What's the last thing that took longer than you expected? |
| “How satisfied are you with the price and the quality?” | How satisfied are you with the price? | What would you expect to pay for what you get? |
| “What do you like most about our reports?” | How do you use the reports, if at all? | What were you trying to find the last time you opened one? |
| “How likely are you to recommend our award-winning service?” | How likely are you to recommend us, 0 to 10? | If you did recommend us, what would you say? |
| “Do you read every product update we send?” | How many of the last five update emails did you open? | What made you open the ones you opened? |
| “Most of your colleagues say the new office layout helps. Do you agree?” | How, if at all, has the new layout changed your day? | What's one thing you do differently now? |
| “I'm proud to work here. Agree / Disagree” | What, if anything, makes you proud to work here? | When did you last feel that? |
| “How often do you exercise? Daily / Several times a week / Weekly” | How often do you exercise? Never / Less than weekly / Weekly / Several times a week / Daily | What tends to get in the way on the weeks you don't? |
| “How well does our API's rate-limit strategy fit your integration architecture?” | What was it like connecting [product] to your other tools? | Which connection took longest to set up? |
| “Now that you've read the outage report, how satisfied are you overall?” | Overall, how satisfied are you with [product]? (asked before the outage questions) | What's the first thing that comes to mind about the last month with us? |
Every example is invented for this page. Each rewrite keeps the question's goal, or replaces it with the behavior that answers it.
Twenty-four neutral questions, numbered so you can copy a set. Under each is the follow-up to ask when the first answer is short. Each set links to its full use case. For 66 more open questions see open-ended survey questions; for all types by situation see the survey questions hub.
For customers and clients. None of these tells the person what you hope to hear.
How would you describe our support to a colleague?
What's the last thing support did that you'd mention?
What, if anything, did you expect from us that hasn't happened yet?
When did you first expect it?
Which part of what you get feels most worth the price?
What would you drop first if the price went up?
What was the last thing you looked for on our website?
How long did it take to find, if you found it?
How often do you use [product] in a normal week?
What's usually the reason you open it?
Which of our emails, if any, do you open?
What made you open the last one?
How did the delivery go?
What, if anything, would you change about it next time?
What's one thing about working with us you've mentioned to someone else?
How did you put it to them?
For staff. Statements with agree/disagree are rewritten as questions so agreeing isn't the easy way out.
How would you describe your workload right now?
Which week in the last month was heaviest?
How does a good week here differ from a hard one?
What made the last hard week hard?
How useful was your last performance review to you?
What did you do differently afterward, if anything?
How clear are the priorities for your work this quarter?
Which priority, if any, is the least clear?
How, if at all, has your job changed in the last six months?
Which change has mattered most to you?
How do decisions that affect your work reach you?
Think of the last big one. How did you hear about it?
How would you describe the feedback you get from your manager?
When did you last get some?
How much say do you have over how you do your work?
When did you last want more say than you had?
For people using a product or a feature. Ask about the last time, not the habit, and describe features instead of naming them.
How did setting up [product] go?
Which step took longest?
Which job do you turn to [product] for most?
What did you do that job with before?
How does [product] fit with the other tools you use?
Where does information get copied between them by hand?
Which part of [product], if any, do you avoid?
What happened the last time you used it?
How would you describe [product]'s speed?
Which screen do you wait on most?
How does [product] compare with what you used before it?
What's one thing the old way did better?
How often do you read our release notes?
What made you read the last one you read?
How useful are the notifications [product] sends you?
Which one did you last act on?
Read every question against these seven checks before you send it.
A leading question gets agreement. A neutral question gets a short, honest answer, and the follow-up is what turns it into something you can act on: “What's the last thing that took longer than you expected?” after “How easy or hard is the app to use?” The probing questions guide has fifty follow-ups by type, and feedback questions has the five best neutral questions for each situation.
A form can't ask the follow-up. An agent survey can: Spoke asks your neutral question, reads the answer, and asks a follow-up when it's vague. You set the goal, the topics, how deep to probe and what to stay away from, and every conversation is kept as a transcript.
A question whose wording, answer options or position in the survey pushes people toward a particular answer. “How much did you enjoy the workshop?” assumes enjoyment; “How did the workshop go for you?” doesn't.
“Wouldn't you agree the app is easy to use?” The tag “wouldn't you agree” and the word “easy” both point at yes. The neutral version is “How easy or hard is the app to use?”
A leading question pushes toward an answer through its wording (“How much did you enjoy the workshop?”). A loaded question assumes something about the person that may not be true (“What frustrates you about doing reports by hand?”). Fix the first by rewording; fix the second by checking the premise.
Cut the adjectives, split any “and,” check that the premise fits everyone, balance the scale, and move sensitive questions later. Then add the follow-up you'll ask when the neutral answer is short.
One that asks two things at once, such as “How satisfied are you with the price and the quality?” Someone happy with one and unhappy with the other has no honest answer. Split it into two questions.
The tendency to agree with a statement because agreeing is easier than objecting. It shows up in agree/disagree items like “I feel valued at work.” Turn the statement into a question, or make it open: “What, if anything, makes you feel valued at work?”
No. In a sales conversation or a coaching session, where you want to give someone a push, a leading question can be the right tool. In a survey whose answers you'll count, never: the push ends up in the numbers.
Ask one thing in plain words, without an adjective that hints at the answer. Ask about the last specific time rather than the habit. Use a balanced scale with a midpoint. Test the survey on someone who doesn't know what you hope to hear.
Spoke asks the follow-ups for you, with every person. Free to start, no card. Your first ten agent surveys are on us.
