Written by Kene Anoliefo
|
May 5, 2024
A guide on how to test out product concepts when you're on a tight product development timeline.
In Part 1 we reviewed how to validate whether or not the problem you’re trying to solve is one that your prospective customers care about. By now, you should know:
Now that you understand the problem deeply, now it’s time to start ideating on potential solutions. As you brainstorm and prototype your ideas, consider testing out these concepts as early as possible.
Concept testing is a simple way to get feedback on your initial high-level, low-fidelity representations of your solution. This could be a mockup of a product, a presentation with slides describing a product or even some rough drawings and wireframes.
Your set of concepts should represent distinct approaches to solving the problem, each with their own clear hypothesis on why it will meet your customers’ needs.
In Part 1 we used the example of a startup that wanted to create a savings and investment calendar for recent college graduates. Even if you’re incredibly passionate about your initial concept, you should come up with alternative concepts that are a valid ways of solving the needs so that you can compare and contrast what feedback you get from each.
If you’re in a rush and don’t have time to create multiple new concepts, you can use a product that already exists by describing its features and functionalities and removing all identifying branding. In the case of our example, this might be another financial planning app that takes a different approach then the one they are considering.
Using our example of a team interested in creating a savings and investing calendar for recent college grads, a good interview might sound like this:
You: [Presents concept of savings and investing calendar]. What do you think this product does?
Customer: Seems like it’s some sort of calendar or scheduling tool that helps you figure out how much money you can save and spend at different times in the month.
You: What type of person do you think would use this and why?
Customer: Well to me the most useful part of this is that it helps you figure out the right timing of what to save or spend. If you’re like me and you don’t have a ton of money left over after a paycheck, sometime it’s hard to know whether or not you have extra funds to use because maybe there’s a charge coming up and you have just enough money to cover it. I have a question: would this look at what you typically spend in a month try to estimate what is safe to use, or just the charges in your bank account?
You: That’s a good question. Why is that important for you to know?
Customer: Well because even if it seems like I have money in my bank account, I know that maybe I have a loan payment or an electric bill coming up next week. And since it has access to my bank account I would expect it to be able to know that so that it doesn’t tell me I can move money somewhere and then I overdraft.
You: I see, that makes sense. You mentioned you use a spreadsheet to figure out what you can save each month. Do you think that you could use this to replace how you used the spreadsheet last month?
Customer: In theory, yes because it has access to my bank account and it can know my paycheck and expenses. That’s basically what I do with my spreadsheet so this would be way better. But last month it would have been tricky because I went to a friend’s wedding and spent a lot more than I usually do. I don’t think it would have been able to predict that.
You: It sounds like you’re saying month to month you might have some unpredictable expenses that you need to account for?
Customer: Exactly. You know what would be great though? If it just let me put that in here as a future expense. Oh and you know what would be AMAZING? If it connected to my Google Calendar and could see events and plans I had in the future and then estimated how much it might cost and built that into the budget!
You: Thanks for that idea! So if you had to describe this to a friend, how would you describe it?
Customer: I think it’s a calendar that basically monitors your current spending and spending for the next few weeks and can tell you every single day what to do with your money — how much you have to spend, save or invest.
From one interview we know:
You can also show the second concept in this same interview, and ask customers to compare and contrast how they would use it.
On the flipside, an interview with poor signal might sound like this:
You: [Presents concept of savings and investing calendar]. What do you think this product does?
Customer: Seems like it’s some sort of thing that connects to your bank account and tells you your balance on each day?
You: What type of person do you think would use this and why?
Customer: I’m not sure really because it seems like what my checking account mobile app does. It doesn’t seem like it would show me any new information. But I can see that it is trying to tell me that I should save or invest money on certain days. That’s kind of interesting.
You: Think about how you managed your finances over the last few months. How could you have used this, if at all?
Customer: Hmm well I don’t really do too much, but I think this could be interesting because I do hope to save some money this year for a vacation I want to take. But I just try to put away a few hundred dollars a month in my savings account so…I don’t know. It seems really interesting though.
You: If you had to describe this to a friend, how would you describe it?
Customer: I think it’s a place to connect your bank account and remind you of what your balance is.
From this interview, we know that:
If you’re getting low signal, you might need to interview more people to get more feedback. Be wary of this — don’t shop around for affirmation because you like your idea. It’s easier to accept that something isn’t resonating and spend time investigating why than it is to move forward blindly. You also might need to refine the concept if there seems to be a lot of confusion, clarifying questions or poor product-language fit. If you find that people get it but can’t describe how it fits in their workflow that’s a red flag — you might not have the right features or there might be too much friction to drive adoption.
Assuming you see another 1-2 interviews that give as good of signal as this one did, you can move forward to the next step: Usability Testing!
HEARD is a great way to test out early concepts, especially when you’re short on time or want to get data from more than a handful customers.
Using HEARD, you can conduct open-ended, qualitative 1:1 interviews with your customers that are moderated by AI. In each interview, you can attach an image or text description of your concept. The AI will interview your customers for you by asking questions that adapt in real-time based on what the customer says — just like a human interviewer would. Afterwards it will synthesize the data and transform it into insights without any manual analysis required.
HEARD has templates that you can use to get started, including one called “Concept Testing” Start off by entering your research goals and your hypotheses around user needs.
HEARD will suggest a set of questions you can ask based on your goals and potential product. Use these questions or add your own. You can add images to any discussion question to show concepts of your product to your users.
After your customize the interview, you’ll receive a link you can send to your target customers. When people click the link, they’ll land on a web experience where they can start the interview on-demand — no back and forth over scheduling required!
After they click start, they’ll be taken through the series of discussion topics and questions you selected.
For every topic, the AI will ask adapt and ask follow-up questions to collect deep insight from participants.
After your customers complete the interview, HEARD will analyze the raw data and pull out the top themes from across the conversations so that your team can start learning immediately without any manual analysis required. HEARD uses AI to cluster the raw conversational data into unique themes, each of which is quantified so that you're able to understand the size and impact of the theme. Now, you have qualitative data with real quantitative measurement rather than relying solely on feedback from a handful on in-person interviews to make important decisions.
Dig deeper on every theme by reading through user quotes to see how they describe their experience using their own words. Quotes help you build connection with your users, and can help your team and stakeholders build empathy with customer needs.
If you passed the threshold and got good signal from this customers at this stage, now it’s time to move to the next phase: Validating a customer experience through Usability Testing.
Get the best content on user research, and product building delivered to your inbox.
Get notified when we publish the next installment of Off The Record