How to measure Product Market Fit?
It relies on many things, including where you are in the go-to-market process and the data you have. Unless you were at the forefront of startup management strategy ten years ago, you didn’t hear much about product-market fit.
At the time, the only genuine definition I had seen was Marc Andreessen‘s, which was more like, “You’ll know it when you have it.” Everything will be flying off the shelves,” said Sean Ellis, the author of Hacking Growth and a growth strategist who created Qualaroo and previously worked at Lookout, Dropbox, and Eventbrite in executive growth roles.
Ellis’ abbreviation of Netscape’s co-definition, the founder’s coined in the blog post “The Only Thing That Matters,” may be an oversimplification. Even so, for Ellis, who resides in Newport Beach and runs the Breakout Growth podcast, Andreessen’s intuitive assumption that “you can always feel product-market fit when it is occurring” holds. Customers are buying the product at the same rate as you can create it — or consumption is increasing at the same rate as you can add more servers,” didn’t match reality.
Ellis understood early in his career as a marketing VP at Uproar and LogMeIn that product-market fit was more of a verb than a term. It wasn’t enough, as Andreessen put it, to be “in a good market with a product that can satisfy that market.” Ellis explained, “We had to iterate and tighten around a product.”
He was successful at his first two companies, which developed swiftly and were sold in IPOs, but he realized his winning streak was going to end when he joined Xobni. To calm his worries, he devised a set of customer satisfaction questionnaires with two goals in mind: For one thing, they might be used to determine how committed Xobni’s clients — mostly managers — were to the product. Two, should his job at Xobni end, they may be used as a litmus test for other potential employers’ growth potential.
The phrase was crucial: He left out the word “satisfaction” on purpose, because “a good manager is never content,” he explained. Rather, he asked, “How would you feel if you couldn’t use [this thing] anymore?”
(a) quite disappointed, (b) somewhat dissatisfied, (c) not disappointed (it isn’t that useful), and (d) not relevant are the four options (I no longer use [product]) It served as the foundation for a quantitative methodology for evaluating product-market fit.
“Over time, I just honed in on that benchmark.” “Says, Ellis.” “Five of the first seven companies I worked for were valued at billions of dollars. As a result, I had a good idea of what success looked like. And what I started to see was that every successful business I worked with had a 40 percent or greater success rate.”
That was the golden number: the percentage of customers who took the poll and indicated they would be very dissatisfied if the product didn’t exist; it’s a metric Ellis claims thousands of businesses have used to establish their market viability.
Of course, his method is just one of many for determining product-market fit, and it isn’t always the most beneficial after the first few months of the product journey, as he concedes.
Sean Ellis survey method:
This survey question, which is best suited for early-stage startups, asks respondents how they would feel if they could no longer utilize a product. The possible responses range from “very disappointed” to “I no longer use the product.”
Cohort retention:
This metric is useful for businesses with a larger customer base since it gauges the percentage of consumers who are still paying for a product after a specified time has passed, usually eight weeks.
Net promoter score:
A measure of how likely people are to tell others about your product.
Lifetime value to customer acquisition ratio (LTV/CAC):
This metric compares the amount of money you spend to acquire a customer to the amount of money you make from them.
The cohort retention curve, according to Danielle Cohen-Shohet, CEO of GlossGenius, a New York-based office management and payment technology supplier for beauty and wellness businesses, is the gold standard at later phases of development. The graph depicts the percentage of users who continue to pay for a product after onboarding for eight weeks.
According to Mixpanel’s 2019 Product Benchmarks Report, the downward-sloping curve should flatten at zero — ideally, at or above 6 to 20% of users — or you’ll be paying more to recruit your consumers than they’re paying you to keep them.
Then there’s the net promoter score, which assesses how likely core consumers are to suggest your product to others, according to Cohen Shohet. Is your product backed by influential communities that can help it move forward?
The idea is that determining market viability is difficult. We simply asked, “Which way is the best?” And we got a straightforward response: it depends. However, what it depends on – the company’s size, stage, cash reserves, and growth goals — is instructive.
Maegan Lujan, director of solutions and services at Toshiba America Business Solutions, Inc., ran a LinkedIn poll in response to an interview request and received responses from technical professionals from 14 different firms. When asked which tool is most useful for assessing product-market fit, they responded as follows:
The retention curve: 43 percent
Net promoter score: 36 percent
LTV/CAC ratio: 21 percent
The Sean Ellis Survey Method:
According to Ellis, his poll is successful because it identifies a company’s primary customers, allowing companies to create realistic user profiles and marketing campaigns based on them.
However, even if a company does not initially exceed the 40% criterion, the survey’s findings are informative. Only 7% of poll respondents stated they would be very disappointed if they could no longer use Lookout when Ellis worked for the company in San Francisco.
According to Ellis, the mobile security company swiftly changed its business strategy in response to the poll. It had given five or six different sets of mobile apps and security software products equal weight at first, but it was the antivirus software that garnered the most attention.
“We were able to set the correct expectations for what the product could genuinely deliver by shining a spotlight on that,” Ellis said. “They were at 60% by the time I left that company. They came out with a billion-dollar valuation in less than three years.”
He warns that there are some guidelines to follow when conducting the poll. It’s critical to weed out individuals that sign up but never use your service. The survey should be sent to 40 people who have used the product at least twice in the previous two weeks (though this varies on the product cycle; dedicated Airbnb users, for example, do not utilize the service every two weeks).
“The simplest way I’ve done it is to make a list of those people’s email addresses and send them an email survey. But I’ve also used it in product-integrated inflow surveys. Email isn’t always well received by mobile users. So, in many cases, you’ll need to trigger it via a mobile app,” Ellis explained.
According to Ellis, if new clients enter the market or a company expands worldwide, it’s a good opportunity to refresh the data, with surveys distributed once a month until product-market fit is achieved. Once you reach a 40% response rate, you can send surveys every six months.
However, because product-market-fit is perishable, it’s critical to be diligent about repeating the study. The widespread availability of voice recording tools unique to the Evernote app, according to Marco Perry, founder of the Brooklyn-based product design consultancy PENSA, is a cautionary tale of how differentiating product features can be eaten by copycats.
“In the most recent versions of iOS or Android, a lot of the fantastic things that are available in Evernote are now available for free,” Perry added. “As a result, the operating system is their competitor.”
One of the benefits of Ellis’ survey is its ease of use; it may be readily conducted using Typeform or other comparable tools. Another advantage of surveys over retention cohorts, according to Ellis, is their breadth: “The value of surveys over retention cohorts is that you can utilize them to understand everything about the users.” ‘Who are they, exactly?’ What did they use previously? ‘What made them want to try the product?’ “Says, Ellis.”
Cohort Retention rate
Industry insiders have long regarded cohort retention rate as a sign of product-market fit. However, the algorithm necessitates data that most firms lack in their early months, according to Ellis. However, if you have the data, as GlossGenius has, the cohort retention curve can be instructive.
“We think of product-market fit as when the product is truly good and we can attract paying customers organically, mostly through word of mouth and referrals,” Cohen-Shohet said. “Having a high cohort retention rate is what generates scalable acquisition channels,” says the author.
The rate is the percentage of active consumers who continue to use a product after a certain amount of time has passed, usually 14 days. A product-market fit is indicated when the curve tapers to a straight line above the X-axis.
The measure at GlossGenius is based on a group of people who downloaded the app within the same 30-day period, according to Cohen-Shohet. They are evaluated as a group because their onboarding, product engagement, and customer service experiences are nearly identical.
Each month, new cohorts are introduced, and the customer experience team’s insights, as well as for analytics technologies like Mixpanel, help to enhance retention over time.
The retention curve, according to Cohen-Shohet, is less biased than survey methods: “You don’t have response bias, right?” she asked. “You’re obtaining data from all users, not just those who have the time and want to complete a survey. She explained, “You’re collecting entire life-cycle data, not data from a single poll at a single point in time.”
It’s unclear how far above the X-axis the retention curve must flatten to establish product-market fit. “Most apps and software have a 6 to 20% eight-week retention rate,” according to a Mixpanel survey. A retention percentage of more than 25% after eight weeks is regarded as elite for products in the media or finance industries. Over 35 percent retention is considered elite in the SaaS and e-commerce industries.”
Andrew Chen, a general partner at Andreessen Horowitz and former head of rider growth at Uber, reports on his blog that the top 10 apps had a 60-day retention rate of 55 percent, the next 50 had a 60-day retention rate of 40 percent, and the next 100 had a 60-day retention rate of 21 percent, based on an analysis of Quetta’s usage statistics from over 125 million Android mobile phones.
The following 5,000 apps had a 60-day retention rate of 11%, whereas the average of all apps analyzed at 60 days was only 7%.
Nonetheless, “you have some indicator of product-market fit” even at that level, according to Ellis. “It’s close to the 7% we found [using the Sean Ellis poll] at Lookout. You simply want to research that [cohort] and figure out who they are.”
According to Perry, analytics tools can help a lot by determining where difficulties in the mobile experience are occurring.
“If someone is onboarding for the first time on your app and dropping out early in the process, you need to figure out why,” he said. “‘Are they becoming perplexed? Do they not consider the time they spend to be valuable? According to Perry, analytics tools can help a lot by determining where difficulties in the mobile experience are occurring.
Are you requesting a credit card when they were expecting a free item? Analytics is excellent in resolving tactical concerns. And, in the end, those factors determine whether some apps succeed or fail. For example, how quickly can you use Instagram compared to its competitors?”
Net Promoter Score
Cohen-Shohet said it’s a good supplement to the cohort retention rate, but it’s a weaker indicator of product-market fit in the end.
“It’s what I’d call people’s emotional reaction to your product,” she says. Is it true that they adore it? Will they tell their friends about your product and recommend it to them? Or are they going to keep it a secret since it’s “meh”?”
According to a study on Survey Monkey’s website, the score can be calculated with just one question: “How likely are you to refer (insert company or product/service) to a friend or colleague?”
Detractors are people who give a product a rating of six or less, according to the research. Respondents who give the product a seven or eight are referred to as “passives,” while those who choose a nine or ten are referred to as “promoters.” These responses are then used to calculate a score using the procedure below.
NPS = percentage of promoters – the percentage of detractors.
According to Survey Monkey global benchmark statistics, a score of zero to 11 places a company in the bottom quarter, the median net promoter score is 40, and the top quarter has a score of 64 or higher.
According to Mike Mason, a product manager at Mercato, a San Diego-based online grocery shopping and delivery startup, the company ranks about 70. Mercato, on the other hand, utilizes the metric to diagnose service shortages rather than to evaluate product-market fit.
Mason explained, “We’ve never actually seen a large rise in the NPS purely based on a function we added.” “It can also be deceiving since you may have excellent customer service but a poor product, or vice versa. It only tells part of the story.”
Another concern is that the folks that fill out the survey may not be the ones who are paying for the service. That’s according to Alex Willen, who spent ten years as a product manager at early-stage corporate SaaS startups like Box, Blue Jeans Network, and Talkdesk before relocating to San Diego to launch Cooper’s Treats, a dog treat company that recently reached product-market fit, according to him.
Willen wrote, “The reality is that 90% of end customers may despise the product, but the actual decision-maker will renew.” “Everyone despises Salesforce, yet no one leaves. You’ll likely obtain a better picture of your product’s value to customers if you measure the actual decision-makers, but it’ll be too late to be useful.”
While the net promoter score isn’t a reliable indicator of product-market fit, it is a valuable indicator of a company’s customer experience and perceived ethos, according to Ellis.
“You can have a great product-market fit but be a terrible firm to do business with,” he explained. “‘The last time I required customer assistance, it took the company five days to respond.’ As a result, your net promoter score plummets. It’s far more a representation of the company’s touchpoints.”
Customer Lifetime Value (LTV/CAC Ratio)
This equation is described in a Klipfolio report as one of the most powerful indicators of product-market fit — the customer lifetime value metric.
(LTV) = Gross Margin % X Avg. Monthly Payment / Churn Rate
/
(CAC) = Sales and Marketing Costs / New Customers Won
This may appear to be a simple measure until you start delving into marketing acquisition costs and defining what it means to be a customer who has been acquired. According to Lamine Zarrad, CEO of Austin-based business Joust, the company calculated its client lifetime value by pulling data from an AI-powered risk assessment tool it employs for underwriting freelancer contracts.
They started by determining the minimal viable product, which in this case was an app that allows self-employed employees to accept credit cards or direct deposits in exchange for rendered services or products, he explained. Payments to freelancers are held in a bank account that is insured by the Federal Deposit Insurance Corporation (FDIC). PayArmour, a follow-up feature, guarantees freelancers will be paid for invoices within 30 days for a 1% no-interest fee.
Joust runs a series of multivariate tests on registered customers using machine learning: “Like any other bank, we ask for your name, your social security number, your address, your phone number, your email address, and so on.” Then we look at each of those attributes separately and run models on how they connect,” Zarrad explained.
To generate customer profiles and analyze fraud risk, the system tracks user activities and over 1,300 credit variables. “And this is where we get a lot of our information,” Zarrad explained. “When you populate your tool with contacts within the first 24 hours, it is an indicator of a good consumer and favorable behaviors,” Zarrad explained.
A back door to computing the LTV/CAC ratio has also been revealed by the algorithm: A user that submits two bills per month, each worth around $2,000, is considered a new customer who is unlikely to churn. They have an LTV: CAC ratio of more than 3:1 using this strategy, which is an industry benchmark for effective growth, according to Profitwell, which indicates “each gained user is worth 3x what you paid to earn their business.”
Go-To-Market
All of these measures, according to certain entrepreneurial-minded doubters like Willen, are lagging signs of product-market fit: a case of the chicken and the egg.
“You’ve probably already signed enough transactions to suggest that you have product-market fit,” he added, “by the time you have enough customers to apply these methods.”
So, if you’re like Willen and trying to start an internet dog treat business on a shoestring budget, what do you do?
He stated, “There’s no better way to determine if people want your product than if they’re prepared to pay for it — or use it, visit it, or do whatever else you’re hoping people will do.”
Willen had already designed the product, and it was inexpensive to make. He was able to get a packaging designed on 99designs for approximately $2,000, set up a Shopify site, do a limited production run, and begin advertising on Google and Instagram for roughly the same amount.
If going to market without knowing you have a product-market fit is too expensive or time-consuming, Willen suggests becoming creative and devising a hack that allows you to judge people’s intent to buy your product for less money.
For a consumer app, this could entail running advertisements and observing how many users click on them. Let’s pretend you have a brilliant concept for on-demand llama delivery. If you want to see if it will work, run some Facebook advertisements, and see how it goes.
“Of course, individuals might just click out of enjoyment in that case,” Willen explained, “so the further you can go to discern true intent, the better.” “If you create a landing page asking for people’s email addresses to learn when llamas will be available in your city, you’ll get a greater signal of intent,” he said.
In other words, there’s a tradeoff between the initial time and money commitment you make and the quality of the product-market fit gauge you’ll receive. You’ll have a nearly irrefutable determination if you fully create and launch your product – the product sells well or it doesn’t — but at a considerable expense.
You won’t get a fantastic signal of product-market fit if you send a simple survey to people in your network asking if they’d be interested in your product, but your cost will be modest.
According to Willen, the goal is to strike the correct balance between cost and signal quality. In certain circumstances, you may need to deploy a scaled-down version of your app, while in others, you may benefit from creating a deck that explains the product and value proposition to your target audience.
“The cost of establishing my dog treat business was so minimal that it made sense to immediately launch,” he said. “However, with more complex corporate software, it’s too hazardous to begin without first determining product-market fit,” he added.