A Lean Approach To Product Validation
Email Newsletter
Weekly tips on front-end & UX.
Trusted by 200,000+ folks.
Flexible CMS. Headless & API 1st
Register!
One of the biggest risks of building a product is to build the wrong thing. You’ll pour months (even years) into building it, only to realize that you just can’t make it a success.
At Hanno, we see this happening time and time again. That’s why we’ve put together a Lean Validation Playbook.
Why We’re Going Lean
“Lean” in this case means that you’re moving swiftly to figure out what you’re going to build and how you’re going to build it with as few resources as possible. These resources might include time, money and effort. The lean startup methodology is advocated by Eric Reis, who has massively influenced the way we work through his book The Lean Startup.
In this short guide, you’ll learn how to validate a product-market fit, talk to users and hack together prototypes. The goal is to make sure your product idea is a viable one, which will save you time and money.
By the end of the process, you’ll be able to focus on building your product, confident that you have a higher chance of success!
Ready? Let’s get started.
The Process
Four distinct stages make up the lean validation process. Only once you’ve passed all four can you be confident that your product idea is worth developing.
- Validate the problem.
Is this a problem worth solving? If users don’t think this is a major problem, your solution won’t be appealing. - Validate the market.
Some users might agree that this is a problem worth solving. But are there enough of them to make up a market for your product? - Validate the product.
The problem might exist, but does your product actually solve it? - Validate willingness to pay.
There might be market demand and a great product. But will people actually be willing to reach into their wallets and pay for it?
What If An Idea Meets All Four Criteria?
If an idea meets all four validation criteria, then you’re free to move directly into product development if you choose! But you might instead choose to take a step back.
By working through the lean validation process, you’re sure to receive heaps of feedback from users. With this deeper understanding, you might realize that there’s a bigger opportunity to tackle. In this case, you should feel free to pivot!
If you do, you can repeat the lean validation process with your pivoted and improved product idea.
Either way, you can be confident that you’re taking a strong idea into product development.
Validate The Problem
No matter how confident you are in the product idea, you first need to figure out whether the problem is a real one that needs solving. To do this, your best option is to talk to potential users directly. The focus here is on getting qualitative validation of the product idea.
First, we start with a small number of properly sampled, representative users and verify that the problem exists for them. Then, we optimize and verify this later on with more users and at greater scale.
The learning you take from these interviews will give you the confidence you need to move the idea forward. Or it’ll show you that you need to pivot the idea as you learn more about the real underlying problems.
The three techniques described below are meant to be used individually, but they form a suite of methods that can provide a better picture of the problem you’re trying to validate.
Technique 1: Find Five People Who Are In
Finding at least five people who say they would be keen to use your hypothetical product is a sensible indication that you have a problem worth solving. While it doesn’t necessarily guarantee that the problem actually exists or that your proposed solution is good or valuable, it’s the simplest way to begin validating the problem you’ve identified.
Here’s an example from Rob Walling, creator of an email-marketing tool named Drip:
“I wanted to find 10 people who would be willing to pay a specific amount for the product once it was complete. This forced me not to think about features, but to distil the idea down to its core value: a single reason someone would be willing to pay me for the product. I took that and emailed 17 people I knew, or had at least heard of, who may have shared the same pain. This way, I not only had my initial customers who could provide me feedback on the details of how Drip should work, I had the start of an early base of revenue I could use to start growing the product.”
If you can build a good rapport with these users and keep hold of their contact information, they might even be willing to be your first five customers.
Technique 2: Interview Users
By sitting down with people and interviewing them, you will likely learn a lot about both your problem and your users. The focus here is to understand the motivations and needs of each potential user you talk to and to use that feedback to improve the product.
Ideally, you’ll want to have both an interviewer and observer present during the interview. While the interviewer communicates with the user, the observer would take notes.
Some good practices for a fruitful interview include encouraging participants to share their past experiences as well as their current needs and challenges. Ask them how they’ve tried to solve this particular problem in the past and what was the outcome. Talking about this from a personal point of view allows participants to reveal their feelings, which should give you better insight into their motivations and help you to empathize with their needs.
Also, avoid asking people what they want. It’s often difficult for people to share exactly what they want. It’s easier for them to tell you what they’re trying to achieve and for you to ask about their motivations behind this. Uncovering this information enables you to gauge whether your product idea addresses that particular need or whether you need to tweak it slightly to solve a more pressing problem.
Another damper on user interviews is leading or suggestive questions. These are questions peppered with the interviewer’s assumptions, which could lead to false results. Keep the questions unbiased and open-ended — so, “What’s your impression of using feature X?” instead of “How easy was it to use feature X to navigate?”. We’ve compiled more helpful pointers on best practices for user interviews.
Technique 3: Ethnographic Research
Ethnography is often described as the process of discovering the unknown. In ethnographic research, you take on the role of the intrepid explorer and travel to the natural working or living environment of your users. This form of research is great for observing and inquiring about behavior (What do you do?), motivation (Why do you do it?) and cognition (How do you think about what you need and what you do?).
It gives a lot of insight into context, which can be hard to get from other, more formal testing techniques. Learning more about this context helps us to understand how it affects the user experience, especially outside of lab conditions, controlled interviews and tests.
The key here is to spot and capture the “A-ha!” moments while discovering user motivations.
Imagine that you’re building a product to improve the ergonomic posture of office workers. Ethnographic research would be a valuable tool, perhaps even more so than user interviews. You would set yourself a goal of visiting offices to observe users “in the wild” and to see whether the problem exists. You might also visit a range of different offices, too: startup offices, coworking spaces, as well as the typical big corporate offices.
Conducting ethnographic research helps you to see whether the problem is real and perhaps even to uncover new problems so that you can pivot the idea. Ethnographic research is a complex field that goes beyond what we’ve touched on so far. Take a deep dive into the practice by reading more about it.
Why Are These Methods Important?
By focusing on user research, you’re able to avoid common pitfalls, such as assuming the problem you’re dealing with is a pain point for others, too. Too often, we come across scenarios in which a designer says, “I’m pretty much like the end user, so it’d be safe to design something based on my own needs. Whatever I come up with will probably fit the mould for other users.” Keep in mind that you are not your user. Because you’re too close to the problem, what seems like the perfect solution to you might be a terrible solution for the average user. Tackling the problem you’ve identified could also mean that you’ve spent significant time researching the topic and probably understand it more deeply than the average person. Your view of the problem at hand is now biased, all the more reason why you need input from other users, to ensure you’re solving a real problem and not just one in your head.
Validate The Market
Once you’ve spoken with users and validated that the problem exists, you need to make sure the market is large enough to justify taking the idea further. Where will your users come from, and how much potential revenue lies in the market opportunity?
By gathering as much information as you can about the potential market, you’ll be able to make an educated guess as to the size of your target audience and the number of customers you can acquire. It will also help you to figure out a possible pricing structure for the product, which will be valuable as you start to make other financial projections.
How Large Should The Market Be?
If you want the product to support you, then a micro-market, offering a return of just a few thousand dollars a month, might be enough to meet your needs. But if you have big ambitions and want to make billions from the product and become the next unicorn startup, then you’ll need to make sure a big market exists.
“But I’m Going To Create My Own Market!”
True, sometimes a truly innovative product creates an entirely new market. Before Whole Foods existed, the organic grocery market was significantly smaller. Uber created an entirely new on-demand transportation service by connecting technology and people with cars.
While this is possible, be extremely careful about assuming that your own idea will experience a similar fate. Look deep to understand and forecast the market. After all, for your product to succeed, you’re going to have to find a market for it eventually. Proceed with caution!
Here are a couple of tools you might use to validate the market for your product:
- Google Trends allows you to compare the relative volume of search terms. It shows you how demand for each term has changed over the last 12 months.
- Google AdWords’ Keyword Planner reveals the average monthly searches for a given keyword. It also gives you an estimate for competitors and suggested bids. Using this data, you can estimate how much traffic you can drive to your website using Google AdWords paid ads or by reaching the top of search engine results.
- Deep competitor research enables you to understand how your future competitors are trying to solve the same problem. If you’re focusing on a small market, finding just one competitor might be enough. But if you’re focusing on a bigger market, search for more, at least three. Besides searching for direct competitors, look up those that offer similar products and solutions. Then, find out as much as possible about them. Crawl their profiles, read their blogs, look for media coverage. Learn about the size of their team, pricing structure, finances and product features.
- Moz can run content audits on competitor websites to show you how they rank for keywords and which of their content is most widely shared. All of this will be valuable information down the line.
“What If I Can’t Find Any Competitors?”
If you can’t find any competitors, then most likely you’ve either missed something in your research or you’re working on a problem that doesn’t really exist. There’s an exception, however. If you’re starting up a very small niche business, there’s a chance you don’t have any competition… yet.
Another area to look at when conducting competitor analysis is companies that might attempt to move quickly into a new product or service area when competition arises in their market. While this isn’t of immediate concern, it pays to roughly identify fast followers who might step up to offer a similar product to yours to win the market. This will help you to prepare and come up with a plan to handle the situation should it arise.
Once you’ve validated the market, you’re ready to move on to the fun part: validating the product!
Validate The Product
There’s only one way to make sure your product solves the problem you’re focusing on: to get your hands dirty and build a prototype.
It doesn’t matter if you don’t have much technical experience, you don’t need an engineer to build a great prototype. Instead of choosing expensive and time-consuming prototyping techniques, keep things lean.
Once you’ve built the prototype, you’ll begin to run tests with users to gather feedback and learn from them as much as possible.
1. Build A Prototype
Even if your ultimate goal is to build a fancy website or app, you don’t actually need to write any code to validate the idea.
When you’re prototyping, get creative! Identify exactly what the core of your product is, and think of ways to test whether it works in the simplest possible way.
Suppose your product idea is a new kind of hardware and software device to help visually impaired users navigate their home. Geolocation and proximity sensors would identify the user’s location, and a vibrating belt would tell them the direction they need to move in. Building a real version of this product would be very expensive and time-consuming.
Think simple! You can plan and prototype your idea in 30 minutes, without actually needing visually impaired users to test with. During a prototyping exercise, we experimented with a very simple prototype of this product: We blindfolded one of our team members and faked the product by tapping on his waist to simulate the vibrations of the belt. It was an incredibly simple setup that let us carry out basic initial tests of a product idea and get immediate feedback.
If you’re not entirely familiar with prototyping and want to learn more before jumping in, check out these resources:
- “Content-First Prototyping,” Andy Fitzgerald
- “The Skeptic’s Guide to Low-Fidelity Prototyping,” Laura Busche
- “Prototyping for Better Products, Stronger Teams and Happier Clients,” Scott Hurff
2. Test The Prototype(s) With Users
This can be intimidating, especially if you haven’t done it before. Figuring out where to find users and how to run tests with them forces you to go “public” with your product idea and thinking. That’s much harder than working away at your desk on a hypothetical idea.
Once you’ve run your first couple of tests, you’ll start to see how the testing process is not just a source of valuable insight but actually a lot of fun.
Ninja User-Testing Methods
Finding people around you to be test subjects is a great starting point. Ideally, look for users in your target audience — people who actually are visually impaired. However, in the very earliest stages of testing, you can get useful feedback just by grabbing people around you and running simple tests, as we did with our visual impairment test.
Once you learn more from these ninja tests and potentially iterate on your product, taking into account the feedback you gather, you can move on to more structured tests with your target audience.
Remote User Tests
If you can’t find the perfect test subjects from your local community, there are many great ways to test with users remotely. Here are a few tools to help you run remote tests:
- UserTesting An automated, “unmoderated” alternative to interviewing users by getting them to record their on-screen experience of using your product
- UsabilityHub A quick and fairly cheap way to get insights and reactions from a number of users about your prototype
- PingPong A tool for scheduling Skype interviews with remote users
The goal of product validation, whether remote or in person, is to make sure that your product is solving the right problem in the most effective way. It’s highly unlikely that you’ll manage to do this perfectly the first time around — but that’s perfectly fine! Iterations, tweaks and pivots are a natural part of the product validation process and are one of the reasons why prototyping is such a valuable technique.
Once you start to gather strong positive feedback from target users by testing the prototype, it’s time to move on to the final stage of the lean validation process.
Validate Willingness To Pay
There are many ways to validate willingness to pay, but one of the big challenges is that we can’t necessarily trust people’s words. We all know those people who say “Of course, I’d totally do that!” but who then drop out when the time comes to commit. As far as possible, we want to get people to put money behind their words.
How To Build A Validation Website
A single-page website is adequate for validation, especially as a first step. But if you want to gradually iterate and add more pages and content to increase fidelity and learn more, that’s an option, too.
If you’re technical, you could design and code this yourself, but most users are better off using a template website builder.
Here are some good options for building your validation website:
- Squarespace has some great-looking designs.
- Webflow allows for a lot of customization and is good for designing single-page websites.
- QuickMVP is a helpful suite of tools for creating landing pages to validate ideas and products. It simplifies the process of creating the landing page, setting up adverts and gathering statistics.
You’ll definitely want to include a few elements in your simple website:
- Explain what your product does, clearly and concisely.
- Highlight the product’s unique selling points.
- Address any key concerns that users had during testing. If they were worried about security, performance or comfort, explain your solution.
- Add a direct call to action to bring users to the checkout and payment part of the website. This should be clear and unequivocal — something like “Buy now for $99!”
- Make sure the payment page has an email collection form, so that you can gather the addresses of prospective customers. MailChimp is a good option.
- Instead of confirming the user’s order, you should show them a “Sorry” page that explains why you’re conducting the experiment. This way, they won’t have false expectations about receiving the product.
- Run analytics on the website to track how people are interacting with it. Google Analytics is simple and free.
Once the validation website is up and running, you’ll need to drive traffic to it. Be a little cautious about doing this within your network of friends and family. You don’t want to fall into the trap of collecting false positives, people who sign up just because they know you personally.
The best technique for bringing unbiased traffic to a validation website is to spend a little money on Facebook Ads or on a simple Google AdWords campaign.
You’ll probably need to spend between $100 to $500 on ads to bring enough users to the page for validation. This overhead might sound expensive, which is why we leave this validation stage until the very end. Think of it this way: $100 to $500 is far safer and less risky overhead than spending thousands of dollars building the actual product. Plus, if the product idea is strong, this money won’t have been wasted. You’ll be building up a great list of interested users who will likely sign up for the product when it actually launches.
Facebook and Google AdWords have robust targeting possibilities that let you showcase your solution to the right audience, which you would have identified in the “Validate the Problem” phase. You can also further optimize your ad by factoring in data from your initial ads, such as number of clicks, likes and shares received and the location of people engaging with it. Getting started with an ad campaign on Facebook or Google AdWords is pretty straightforward. We won’t cover the steps for setting that up here, but some Googling should get you on your way to quickly creating an ad campaign.
We recommend keeping a close eye on a few metrics:
- Conversion rate How many visitors have attempted to buy the product?
- Total “sales” Learn where (geographically) these sales have come from.
- Cart abandonment rate. In a real online store, this is typically between 60 and 80% (68% is the average). This means that almost 7 of the 10 people who enter the checkout process don’t complete it.
To go deeper, we also suggest the following:
- A/B test Experiment with different headlines and content to see how they affect conversions. However, your findings will be statistically significant only if you’re driving thousands of visitors to the website.
- Speak to customers. Once you’ve gathered someone’s email address, you’ll be able to send messages and newsletters to them and explain how your product is evolving. But you can also reach out to them directly and ask them a few questions to understand their needs and motivations.
Starting As A Concierge Service
The basic principle of a concierge service is that, before actually building any software, you would fake the service and manually fulfill each order.
Consider another example from the real world. Suppose we built PingPong, to help users gather feedback and conduct interviews. The technology behind the product is not incredibly complex, but a decent investment in design and development is required nevertheless in order to launch a strong product. A lot of work goes into a successful product, but at this early stage, we still have a lot of unvalidated assumptions about what users really want.
Instead of going all out and building a service that users can sign up for directly and use, we’ll go the concierge route. Thus, the only way for paying customers to sign up for now is for them to agree that we’ll manually fulfill their order. Eventually, the product will take over and perform these tasks automatically. Of course, there is always the risk that a manual service won’t be able to scale with the demand and you’ll have to turn down customers; while this is a whole other problem for you to tackle, rest assured that you’ve hit upon a crucial problem.
This can feel like a huge time commitment, but in fact it leads to a lot of critical learning. We’ll manually run the service at the beginning and build deep relationships with our first few customers. Because we’re communicating with them a lot, we’ll ask them questions about their needs and expectations and then use their feedback to finetune our business model before we build the full product. This is a great way to validate exactly what users are willing to pay for and to maximize user value.
As a bonus, operating as a concierge means that you might be making money from day one!
Spoof Landing Pages
The best way to validate without getting called out by people who aren’t really prepared to pay for your product is to build a spoof marketing website.
This technique might feel like slightly misleading. But as long as you’re not actually taking people’s money for something you won’t deliver, people will typically understand your motives.
Take Check Maid, an online service for finding and booking home-cleaning services. When the team first created their website, there wasn’t a real business behind it. As founder Alex Brola says:
“We actually validated [the idea] without having any cleaners to do the cleanings. We threw up a site, a booking form, a phone number, and ran some [pay-per-click] ads through Google and Bing, and saw what the conversion rate would be had we actually had cleaners.”
Validating willingness to pay using a spoof landing page is definitely something you need to be careful about. If you do it wrong, you could damage your brand and create a huge amount of user frustration.
The key is to present your marketing content and display a visible “Sign up” or “Buy now” link. When the user clicks the link, you would display a message explaining why the service is not yet running, and offer them the option to enter their email address to receive updates when you do launch. Also, a good idea would be to reward these early users by saying they’re entitled to discounts or other rewards for being early supporters.
We’d recommend not going any further than storing the user’s email address (so that you can contact them later) and possibly their general location if that’s relevant to your product (to gauge regional demand). Make them aware as early as possible in the process that the product is not available. Many laws, especially in Europe, prohibit misleading advertising — be very careful not to take payment or sensitive personal details in a misleading way.
If you’d prefer to avoid a spoof landing page, you could opt for the concierge route, which is less ethically sensitive and often a good alternative.
Validate Several Ideas
Now that you know how to validate a single product idea, let’s look at how you might explore several product ideas.
Let’s consider how our six great ideas shown above fare when we put them through the lean validation process:
- This idea fails at the first hurdle because we can’t validate the problem.
- This idea validates the problem but can’t establish a market.
- This one also fails to validate the problem.
- This validates the problem and market but fails to validate the product.
- This is a pivot of idea 4. Perhaps our first prototype failed, so we decided to build a new one. This time, we can learn from our mistakes and take a different approach. This idea validates the product but fails to validate willingness to pay.
- This final idea validates all four stages. Of this batch of ideas, this is the one we should explore and develop further.
As you’ll see, many of your great ideas will fail early in the process. Finding out that the problem you’re trying to solve is simply not a problem that users are really bothered by is incredibly common. But with perseverance and iteration, you’ll eventually come up with a product idea that validates.
You might notice that the probability of validating each phase is rather low. You might need to finetune your original idea to address a more pressing problem you’ve discovered. This is absolutely fine. In fact, it’s encouraged. Being flexible and keeping an open mind about adapting your idea can pay off handsomely if done right. And that’s why this lean validation process pushes you to fail soon and fail quickly, before you get too attached to the idea or product.
Note that the criteria above, along with their sequencing, are all simply recommended guidelines. The ultimate goal is to be as lean as possible, and you’re the one in the best position to decide what that means.
What’s Next?
Even if you validate, your product isn’t guaranteed to succeed. The process of validation is just the start. But if you’ve worked through this process in full and completed all stages, you’ll be in a far better position.
The goal throughout the process of lean validation is to delay the expensive and time-consuming work of coding as late as possible in the process. It’s the best way to keep yourself focused, to minimize costs and to maximize your chance of a successful launch.
If you’re looking to learn more, check out Eric Reiss’ startup bible, The Lean Startup. As the name suggests, our Lean Validation Playbook is heavily influenced by the lean way of working.
Now that you’ve validated, you’re ready to start thinking about how to make the product a reality. Move fast, but stay cautious! Keep talking to your users, and keep the process lean. We hope you end up working on the next big thing.
Good luck!
If you’d like to read the extended version of this article, check out our Lean Validation Playbook.
Further Reading
- Effectively Planning UX Design Projects
- Lean UX: Getting Out Of The Deliverables Business
- A Guide To Validating Product Ideas
- Giving Your Product A Soul
(al, il, mrn)
