How do I get feedback on my early concepts to understand if my potential solution will be viable?

Part 2: Concept Testing

In Part 1 we reviewed how to validate whether or not the problem you’re trying to solve is one that your prospective customers care about. By now, you should know:

1. What their needs are and how they describe them in their own words
2. The challenges they face in trying to meet those needs
3. The outcomes that are most important for them to achieve.

Now that you understand the problem deeply, now it’s time to start ideating on potential solutions. As you brainstorm and prototype your ideas, consider testing out these concepts as early as possible.

Concept testing is a simple way to get feedback on your initial high-level, low-fidelity representations of your solution. This could be a mockup of a product, a presentation with slides describing a product or even some rough drawings and wireframes.

Your set of concepts should represent distinct approaches to solving the problem, each with their own clear hypothesis on why it will meet your customers’ needs.

In Part 1 we used the example of a startup that wanted to create a savings and investment calendar for recent college graduates. Even if you’re incredibly passionate about your initial concept, you should come up with alternative concepts that are a valid ways of solving the needs so that you can compare and contrast what feedback you get from each.

If you’re in a rush and don’t have time to create multiple new concepts, you can use a product that already exists by describing its features and functionalities and removing all identifying branding. In the case of our example, this might be another financial planning app that takes a different approach then the one they are considering.

How to do Concept Testing with User Interviews‍

Goals

• Share a range of potential solutions to get high level feedback from potential customers.
• Understand which aspects of different solutions work for users, and which don’t.
• Learn how customers imagine using your product within their existing workflow.

Tactics

• Ideal World: 8-12 interviews across 3-5 concepts
• Quick and Dirty: 3-5 interviews across 2 concepts

‍

Interview Discussion Guide

• Mini-needs validation: Do a short version of the needs validation you did in Part 1 by asking them a few questions about who they are, their relationship to the problem space, and how they solve it today.
• Present the first concept. Describe the how the first concept works in a very matter of fact way, not in a sales-y way — you’re not trying to influence their perception or bias their feedback. Allow them to ask any clarifying questions and ask why this information is important for them to know.
• For each concept, do the following:
  • Gauge concept comprehension: What do you believe [concept] does?
  • Audience fit: What type of person would use this? Why?
  • Use-case fit: Think about the last time you experienced this problem. Could you have used [concept] to solve it? Why or why not?
  • Product-Language fit: How would you describe what [concept] does does to a friend?
  • Sources of value: Which aspects of [concept] are most valuable to you? What are least valuable?
• Repeat the same thing for each concept. At the end, ask customer to compare the concepts and say which one meets their needs better and why.

‍

What signals should I look for in my interviews?

• 75% of customers can identify the specific features that would help solve the problem, and can articulate why it would be valuable. It’s not enough for people to say “It seems cool!” You want to hear feedback that shows they not only understand how the product works, but that they can identify how it would precisely help them solve the problem.
• 50% of customers can identify how they could have used the to solve the problem when it recently occurred. It’s not enough for people to say “yeah I would use this!” because they’re probably just telling you what you want to hear. Instead, you want to hear them describe exactly how it would have fit in their workflow the last time they experienced the problem.
• Customers can describe what additional features or capabilities this concept might need in order to be effective. When customers really understand a concept and why it can be valuable, they will often can start “co-creating” with you by suggesting how to make it even better. This can signal that they’re already starting to imagine how it fits within their life and workflow, which is incredibly promising at this stage.

‍

Examples of interviews with good signal and bad signal

Good Signal

Using our example of a team interested in creating a savings and investing calendar for recent college grads, a good interview might sound like this:

You: [Presents concept of savings and investing calendar]. What do you think this product does?

Customer: Seems like it’s some sort of calendar or scheduling tool that helps you figure out how much money you can save and spend at different times in the month.

You: What type of person do you think would use this and why?

Customer: Well to me the most useful part of this is that it helps you figure out the right timing of what to save or spend. If you’re like me and you don’t have a ton of money left over after a paycheck, sometime it’s hard to know whether or not you have extra funds to use because maybe there’s a charge coming up and you have just enough money to cover it. I have a question: would this look at what you typically spend in a month try to estimate what is safe to use, or just the charges in your bank account?

You: That’s a good question. Why is that important for you to know?

Customer: Well because even if it seems like I have money in my bank account, I know that maybe I have a loan payment or an electric bill coming up next week. And since it has access to my bank account I would expect it to be able to know that so that it doesn’t tell me I can move money somewhere and then I overdraft.

You: I see, that makes sense. You mentioned you use a spreadsheet to figure out what you can save each month. Do you think that you could use this to replace how you used the spreadsheet last month?

Customer: In theory, yes because it has access to my bank account and it can know my paycheck and expenses. That’s basically what I do with my spreadsheet so this would be way better. But last month it would have been tricky because I went to a friend’s wedding and spent a lot more than I usually do. I don’t think it would have been able to predict that.

You: It sounds like you’re saying month to month you might have some unpredictable expenses that you need to account for?

Customer: Exactly. You know what would be great though? If it just let me put that in here as a future expense. Oh and you know what would be AMAZING? If it connected to my Google Calendar and could see events and plans I had in the future and then estimated how much it might cost and built that into the budget!

You: Thanks for that idea! So if you had to describe this to a friend, how would you describe it?

Customer: I think it’s a calendar that basically monitors your current spending and spending for the next few weeks and can tell you every single day what to do with your money — how much you have to spend, save or invest.

From one interview we know:

1. The customer understands the overall concept of a  “calendar” to help them manage their cash flow throughout the month. They believe it could be useful for people like them.
2. They can describe why the features are useful, and how it would help them do what they do today even better. It could fit into their workflow but needs to accommodate for one-off or non-recurring expenses.
3. They expect the tool to understand their general spending habits, not just what spending looks at one point in time. They’re open to connecting more sources of information like their Google Calendar so that the tool can give better guidance.
4. There’s product language fit — they describe it using the same terms of needs and values that the team does.

You can also show the second concept in this same interview, and ask customers to compare and contrast how they would use it.

Poor Signal

On the flipside, an interview with poor signal might sound like this:

You: [Presents concept of savings and investing calendar]. What do you think this product does?

Customer: Seems like it’s some sort of thing that connects to your bank account and tells you your balance on each day?

You: What type of person do you think would use this and why?

Customer: I’m not sure really because it seems like what my checking account mobile app does. It doesn’t seem like it would show me any new information. But I can see that it is trying to tell me that I should save or invest money on certain days. That’s kind of interesting.

You: Think about how you managed your finances over the last few months. How could you have used this, if at all?

Customer: Hmm well I don’t really do too much, but I think this could be interesting because I do hope to save some money this year for a vacation I want to take. But I just try to put away a few hundred dollars a month in my savings account so…I don’t know. It seems really interesting though.

You: If you had to describe this to a friend, how would you describe it?

Customer: I think it’s a place to connect your bank account and remind you of what your balance is.

From this interview, we know that:

1. The participant often used a questioning tone when describing what the concept does, which means they don’t feel confident in their understanding.
2. They couldn’t identify what made it useful. In fact, they said they’re existing bank app performed the same functions as the concept.
3. They use vague language to affirm it like “It’s interesting” without being able to say exactly why.

If you’re getting low signal, you might need to interview more people to get more feedback. Be wary of this — don’t shop around for affirmation because you like your idea. It’s easier to accept that something isn’t resonating and spend time investigating why than it is to move forward blindly. You also might need to refine the concept if there seems to be a lot of confusion, clarifying questions or poor product-language fit. If you find that people get it but can’t describe how it fits in their workflow that’s a red flag — you might not have the right features or there might be too much friction to drive adoption.

Assuming you see another 1-2 interviews that give as good of signal as this one did, you can move forward to the next step: Usability Testing!

‍

How to use HEARD to validate user needs

HEARD is a great way to test out early concepts, especially when you’re short on time or want to get data from more than a handful customers.

Using HEARD, you can conduct open-ended, qualitative 1:1 interviews with your customers that are moderated by AI. In each interview, you can attach an image or text description of your concept. The AI will interview your customers for you by asking questions that adapt in real-time based on what the customer says — just like a human interviewer would. Afterwards it will synthesize the data and transform it into insights without any manual analysis required.

HEARD has templates that you can use to get started, including one called “Concept Testing” Start off by entering your research goals and your hypotheses around user needs.

‍

HEARD will suggest a set of questions you can ask based on your goals and potential product. Use these questions or add your own. You can add images to any discussion question to show concepts of your product to your users.

‍

After your customize the interview, you’ll receive a link you can send to your target customers. When people click the link, they’ll land on a web experience where they can start the interview on-demand — no back and forth over scheduling required!

‍

After they click start, they’ll be taken through the series of discussion topics and questions you selected.

‍

For every topic, the AI will ask adapt and ask follow-up questions to collect deep insight from participants.

‍

After your customers complete the interview, HEARD will analyze the raw data and pull out the top themes from across the conversations so that your team can start learning immediately without any manual analysis required. HEARD uses AI to cluster the raw conversational data into unique themes, each of which is quantified so that you're able to understand the size and impact of the theme. Now, you have qualitative data with real quantitative measurement rather than relying solely on feedback from a handful on in-person interviews to make important decisions.

‍

Dig deeper on every theme by reading through user quotes to see how they describe their experience using their own words. Quotes help you build connection with your users, and can help your team and stakeholders build empathy with customer needs.

Next Up: Validating a Customer Experience

If you passed the threshold and got good signal from this customers at this stage, now it’s time to move to the next phase: Validating a customer experience through Usability Testing.

‍