# User Interview Review ## Instructions Analyze this as a user interview, customer discovery call, or focus group. "You" is the researcher or moderator; the other speakers are participants. Score the call against the six criteria in the playbook below, giving each a 1 to 5 and quoting the line from the transcript that earned that score. Never score a criterion you have no evidence for; mark it n/a and say what was missing. Judge only what was said out loud, never intent, never enthusiasm you cannot hear in the words. Flag every signal moment separately: mark each compliment, each hypothetical ("I would," "I'd probably," "I'd definitely pay"), each generic ("usually," "generally," "always"), and each feature request. For each, say whether You converted it into a past-tense fact, a number, or a real commitment. Where a score is 3 or below, write the exact question You should have asked instead, in Your own register. ## What this lens is for A conversation meant to learn whether a problem is real and worth solving: a discovery interview, a user interview, a focus group. A good session ends with the participant's real past on the record, the problem proven by effort they already spent, and a next step that costs them something. It does not end with a warm "this was so helpful" and a notebook full of compliments. ## How to read the call - Read once end to end before scoring anything. First-pass scoring rewards the friendly moments. - Mark every compliment, every "I would / I'd probably / usually / generally," and every feature request. - For each of those, check the next minute: did You turn it into a past-tense fact, a number, or a commitment. - Note every number anyone said: frequency, hours, money, dates, tools they already pay for. - Check the last two minutes separately. Most sessions lose the commitment there, not in the middle. - In a focus group, attribute each line to a speaker and watch for one voice the room starts agreeing with. ## The criteria ### 1. Grounded in their real past, not your idea **Listen for:** In the first questions, You ask how they actually handle this today, in specifics from the recent past, before You describe what You are building. | Score | Sounds like | |---|---| | **1-2** | Straight to the pitch. "We're building a tool that summarizes research calls. Would that be useful?" | | **3** | A real question, then a pitch that answers it for them. "How do you handle notes now? Because we're building something for exactly that." | | **4-5** | Their world first. "Walk me through the last research project you ran. What did you do with the recordings after?" | **Quote:** The first question You asked after the warm-up. ### 2. Compliments deflected to a fact, not banked **Listen for:** When a participant praises the idea, You dig past the praise to a behavior or a number instead of recording "loved it." | Score | Sounds like | |---|---| | **1-2** | The compliment is banked as validation. "That's amazing, thank you!" and the note reads "positive." | | **3** | A light thanks that lets the praise stand. "Glad you like it. Anything you'd change?" | | **4-5** | The compliment is turned into evidence. "Appreciate that. When did this last cost you enough that you went looking for a fix, and what did you try?" | **Quote:** Every line where a participant complimented the idea, and what You asked next. ### 3. Specifics and numbers, not generic opinions **Listen for:** You pull concrete facts (last instance, frequency, time or money spent, current tools) rather than accepting "usually" or "I'd probably." | Score | Sounds like | |---|---| | **1-2** | Generic and hypothetical. "Do you generally struggle with analysis?" answered with "Yeah, sometimes." | | **3** | One specific, then back to generalities. A single real detail surrounded by opinions. | | **4-5** | Anchored in a real event. "How many interviews did you run last quarter, and how long did the write-up take on the last one?" | **Quote:** The first concrete number or dated event You got, or note that none came. ### 4. You talked less and did not pitch **Listen for:** You let silence do the work, keep Your idea out of the question, and the participant speaks more than You do. | Score | Sounds like | |---|---| | **1-2** | You talk more than they do; every pause becomes a feature explanation. | | **3** | Mostly listening, but You pitch at the first flicker of interest and steer from there. | | **4-5** | You ask, go quiet, and follow with "tell me more about that" instead of "so what we do is." | **Quote:** The longest stretch a participant spoke uninterrupted, or the moment You started pitching. ### 5. The problem proven by prior effort, not agreed in theory **Listen for:** You establish that they have already spent time, money, or effort trying to solve this, which is what makes the problem real. | Score | Sounds like | |---|---| | **1-2** | Agreed in the abstract. "Would this be useful?" answered "Definitely," and never tested. | | **3** | They call it a problem, but You never check what they have done about it. | | **4-5** | The effort is on the record. "What have you already tried to fix this? What did that cost, and why did you stop?" | **Quote:** The first evidence of money, time, or a workaround already spent, or note that none exists. ### 6. A real commitment, not a warm goodbye **Listen for:** The session ends with a next step that costs the participant something: time, an introduction, access, money, or sending real data. | Score | Sounds like | |---|---| | **1-2** | A warm sign-off and nothing else. "Super helpful, thanks so much." | | **3** | A soft yes to stay in touch, with no specific commitment attached. | | **4-5** | A commitment with a cost. "Would you introduce me to the two people who own this budget, and try the beta on a real project next week?" answered yes. | **Quote:** The last commitment made before the goodbye, or note that none was. ## Scoring Score each criterion 1 to 5. Mark n/a where the transcript gives you nothing, and say what was missing rather than guessing. Report the average across the criteria you could score, to one decimal place, and report the signal tally separately: how many facts and commitments You pulled, against how many compliments and hypotheticals got recorded as if they were data. - **4.0 and up, with criteria 2 and 5 both strong.** The session produced signal. Write the findings and the follow-ups. - **3.0 to 3.9.** Warmth with leaks. Name the two lowest criteria; those are the next session's discipline. - **2.0 to 2.9.** A pleasant conversation that taught you little. Compliments came, facts did not. - **Below 2.0.** You interviewed your own idea, not their life. Rewrite the discussion guide before the next session. A high average with a 1 on criterion 2 or 5 is not a good session. It is a session where you were flattered. Say so plainly. ## Red flags - **The banked compliment.** Praise for the idea gets written down as evidence, with no fact behind it. - **The hypothetical as a commitment.** "I would definitely pay for that" is treated as a sale instead of an optimistic guess. - **The early pitch.** You are explaining the product before the third question, and every answer after that is being polite. - **The theoretical problem.** Everyone agrees it is a problem; nobody has ever spent an hour or a dollar trying to solve it. - **The warm goodbye.** The session ends on "this was so helpful" with no next step that costs anything. ## How to report - Open with the average score, the signal tally, and the single lowest criterion. One sentence each. - Under signal: list the facts and commitments actually extracted, and the compliments or hypotheticals that never got converted. - Under what went well: each criterion scoring 4 or 5, with its number and the quote that earned it. - Under what to improve: each criterion scoring 3 or below, with its number, the quote that cost it, and the exact question to ask next time. - Under evidence: only what the participant actually did or spent, in their own words, never what they said they would do. - Under next step: the commitment secured, its owner, and its date, or "none secured." ## Coaching notes - Work one criterion at a time. Someone asked to fix six things fixes none. - The order that compounds fastest: 2, then 5, then 1. Deflecting a compliment is a reflex you can build, proving the problem is a skill, and opening on their past is a script change. - If criteria 2 and 5 score low across four sessions, it is not an interviewer problem, it is a discussion-guide problem. Rewrite the guide to open on the last time it happened, not on the idea. - In a focus group, score the moderator hardest on criterion 4. The dominant voice is theirs to manage, and a consensus one loud participant built is not signal.