The Survey trap - How to understand, your users, ask the right questions and get to product market fit at 10x speed.
The survey results are in! and utterly useless...now what?
Stack rank your goals
Product teams rely on user insights to build stuff people want. But the recurring problem I have seen teams run into is using surveys to understand and make product decisions. If you're using Google Forms or SurveyMonkey, these tools barely scratch the surface. The magic and discovery of deeply understanding what problem to solve and the core job your user is trying to get done can't be discovered with multiple choice and fill-in-the-blank questions. These types of questions don't dig into why users think how they do and don't allow you to easily follow up to probe deeper on their perspectives. Now I do believe when you have a really constrained set of questions you want to find out, like the difference in preferences between a blue colored button and a black button, surveys are great at forcing the user to make a decision between a set of constrained fixed choices. But largely, when we are building products, things are not as well defined. We have a range of factors to consider, from willingness to pay to the severity and intensity of the problem. It turns out, if you give users a fixed set of questions with no opportunity to follow up or ask probing questions, you will miss out on clearly understanding what your users want.
The art of the awkward silence
One of the hardest things to do when thinking about what to build and prioritize is speaking to your users, the dreaded startup advice you have probably heard a million times. Yet most teams still find it very difficult to do - difficult to find the right users to speak to, difficult to schedule time when both parties are available to talk, and difficult to understand what the right questions to ask are. Then, once we do have the conversations, it takes a considerable amount of effort and time to understand what the key insights are. The biggest trap I have personally fallen into is asking biased questions to validate a set of assumptions I have. If you ask biased questions, you'll get skewed data. And that leads down a nasty path of building features no one wants. Say you ask "Would you use our new recommendation feature?" Most will say sure, why not? That doesn't reveal anything about how they would actually use it or if it solves a pressing need. I feel your pain. User interviews are tough but so crucial. Thankfully, there are now AI tools that can help. Imagine an intelligient assistant pwered by AI that can chat with your users - in your app, on your website, wherever you need insights. It asks thoughtful questions designed to spark open-ended convos. The key is the questions are unbiased. They don't try to validate predetermined ideas. Just understand perspectives. This research assistant can have natural dialogues at scale to uncover sticky needs. You get tons of rich qualitative data without the manual work. And it can contextualize conversations based on each user's behaviors. Personalized engagement and insights. The AI does the heavy lifting to understand motivations and pain points. No more scrambling to run flawed surveys. Users feel heard and you get actionable findings to build what they want. teams can have continuous conversations that reveal the unexpected. The real gold that takes products to the next level.
The sooner the analysis the better
We are all busy and so inevitably after the user interview, we have other tasks to attend to, it is important to take some time before the analysis or debrief but my advice is to carry out analysis not more than a few days after your user interviews for two reasons, firstly the closer you are to the interview the less you have to rely on the power of your memory which likely means you misremember key important aspects of the interview.
Do not take notes
I feel very strongly about this, except if you are a Jedi level multitasker, you probably should not try to take notes, listen and ask questions at the same time, it is an impossible mission, if you have a colleague joining you asking them to help take some notes is ideal, if that is not possible, it is best to watch the recorded conversation afterwards and take notes, trying to do both at the same time means inevitably not truly listening to the participant.
Ask "why is that important to you and what is the consequence of that problem"
Like every painter has a favorite toolbrush, I have also developed my favorite questions to ask. Often when identifying a user problem, understanding the intensity of the pain or the core job the user is trying to solve for is extremely important. I find asking participants what the consequence of a problem is a key way to both understand the core pain as well as the motivation behind it. By digging into why a particular issue matters to the user and how it impacts their life or work, I gain crucial perspective. This helps me determine which problems should take priority in the design work. It also uncovers the emotional context and broader significance tied to certain pain points.
