If you’re thinking about starting a business, or if you’re in the early stages of starting one, you probably already know a lot about your product.

You know what you’re going to sell and where you should sell it. You know who’s going to buy it and what they’re willing to pay and how they want to interact with you.

Or do you?

Here are a few of the things we “knew” when we started our various businesses:

• Our customers care more about product features than about intangibles like warranties.
• Customers will pay more for a product if it’s environmentally sustainable. 
• Our product will sell best online and won’t benefit from an in-store retail strategy. 
• There’s a substantial market for a higher-end, more expensive version of this common home product.
• Women who get their hair colored at a salon would do it at home if they had the right products. 
• Buying a mattress is a months-long process and most often occurs around big life events like moving or getting married. 
• Most people who buy pizza ovens are men. 
• Young people will abuse a long-term return policy if we offer one. 

The list could go on and on, but they all have one thing they all have in common: they were all wrong. 

Some of them were just a bit off, and some of them were really, really, potentially business-endingly wrong. Some we ended up tweaking and others we chucked altogether. Nothing we do now is exactly the way we believed it should be when we started.

Being Wrong Isn’t the Problem

It turns out, nothing is obvious. Everything you think you already know about your product, your customer, your market, your cash flow? Most of it is wrong.

And that’s fine. Better than fine, actually: it’s great. Because getting things wrong is the first step to getting them right.

The problem isn’t making assumptions. We all do that. When you first jump into a new market, a new area of the business, or a new stage of growth, it’s impossible not to have assumptions because you don’t know the facts yet. And it’s not that most of your assumptions are wrong. Of course they’re wrong. They’re about things you haven’t tried yet. 

For that matter, if you don’t start with assumptions, sometimes you can’t get started at all.

The problem happens when you get committed to your assumptions—when you believe in them as facts and act on them and then keep acting on them (or making more assumptions) even when the evidence suggests you’re on the wrong track.

That’s why this is the first chapter in the book, because everything else you do will depend on learning this one critical lesson: assuming you’re wrong is the first step toward getting it right.

Later on, we’ll teach you how to specifically test every assumption—and keep testing it until you have real facts to work from. But before that, let’s look at where assumptions show up in every newly-formed business, and how you can start breaking them down.

Six Things You Think You Know

We don’t know you. We’ve never met you or heard about your business. You’re not secretly our accomplice, and we have nothing up our sleeves.

But I bet we can still tell you what assumptions you’re making about your business. 

You want to know what they are?

Here you go. You think you know:

• what product you’re selling, 
• who you’re selling it to,
• what your customers really care about,
• what they’re willing to pay, and
• where they’ll buy your product

And last but certainly not least, you think you know whether you have a viable business.

Does that list look like a comprehensive overview of all the super fundamental, basic stuff you should know? That’s because it is. 

Every founder, in every industry, tends to think they know the answers to these six foundational questions that drive their business. And every single one of them has at least some of them wrong.

Assumption #1: I know what product I’m selling.

Everything else in your business rests on this one question: what am I selling? It can seem so foundational that it doesn’t feel like you could possibly be getting it wrong. After all, many entrepreneurs start their own businesses for the sole purpose of selling a particular product that they love, enjoy, and care about. So asking them to open up that box and think again can be a challenge.

Being committed to your product, even loving your product, can be a huge benefit. Starting and running a business is no joke. It takes a ton of time and effort, and the more you enjoy what you’re doing, the more likely you are to put in the necessary work.

That said, the more willing you are to question even this basic assumption about your business, the more levers are available to you to be successful.

We’re not saying that you should abandon your commitment to vegan ice cream and sell shoes instead. What we are saying is, be willing to think critically about why you want to sell a particular product. That way, if you find that there’s not much of a market, or it’s too crowded for you to get into, or whatever, you still have viable options.

Let’s say you are a committed snowboarder. You’ve moved your entire family to Colorado so you can spend as much time on the mountain as humanly possible. Eventually, you decide the only way to go is to start your own snowboard business. You’ll finally get to eat, sleep, and breathe snowboarding all year round.

When you look into it, though, the success rate of new snowboard companies is dismally low. Snowboards require highly-technical manufacturing processes conforming to very narrow specs. And there are already a ton of small players trying to compete with a few massive powerhouses. 

That’s when it’s time to rethink Assumption #1 by expanding the number of different products you might be willing to sell. Don’t be rigid in your thinking about what your business could be.

Could you be happy selling snowboard accessories? What about goggles or stomp pads or bindings or snowboard-themed t-shirts? Maybe you could even break into the splitboarding niche.

It’s possible that none of these are terrific markets to break into, but the likelihood that you can build a profitable, sustainable business increases exponentially once you unclench your grip on your initial assumption and allow yourself to think, I could be wrong about even the most fundamental “fact” about my business.

🔓 UNLOCK: THE FUNDAMENTAL ASSUMPTION IN YOUR BUSINESS IS THAT YOU’RE SELLING THE RIGHT PRODUCT. BUT ARE YOU? IS THERE A SUSTAINABLE BUSINESS AROUND THIS PRODUCT? HOW MIGHT YOU EXPAND WITHIN THE SAME CATEGORY—OR SELL A SIMILAR PRODUCT TO A DIFFERENT MARKET?

Assumption #2: I know who I’m selling to.

Here’s a question for you: how many nurses are there in the world? You don’t need to know the exact answer. Guessing is fine. Do you have your guess?

Okay, now how many people are there in the world who are not nurses? 

Which of those two numbers do you think is bigger?

Obviously, the number of people in the world who are not nurses is bigger than the number who are. So let’s say you sold really comfortable, easily-washable shoes, and you assumed that nurses were your customers for those shoes. No doubt that’s true. Nurses absolutely need comfortable, washable shoes to stand and walk around in all day.

But what if you suddenly thought to yourself, is it possible that there are literally millions of people who aren’t nurses who also might want comfortable, washable shoes they can walk in all day? Like maybe chefs. And teachers. And veterinarians. 

That’s what happened to Rob Gregg, founder of Gales shoes. His story is inspirational in so many ways. He started his company to meet a critical need for nurses after they saved his friend’s life. He literally closed down his luxury shoe brand to do this instead. That’s incredibly cool. But it was only when he stepped back and asked who else he might be selling to that his new brand took off. That generated the revenue to make his company sustainable and scalable and allowed him to serve a larger community of people with similar shoe needs. Win-win-win.

Whereas some assumptions could cause your business to fail outright, not knowing who you’re selling to is more likely to limit your business’s potential. You’re probably right about at least some of the people who want to buy your product. The question is, are you assuming that those are the only people or the most likely people to buy it? 

Case Study: Pizza Ovens

We recently invested in a business that sells outdoor pizza ovens. Not big, built-in brick things but portable metal devices that retail for a few hundred bucks and can cook a pizza in a few minutes. It’s a nifty, fun little addition to the outdoor cooking space. 

When we thought about marketing content for the ovens, we thought about outdoor cooking and backyard barbecues. When we imagined who was using the product, we imagined middle-aged men. 

And in a sense, we were right. In general, those guys are the ones using outdoor pizza ovens. But they aren’t the ones buying them.

It turns out that the majority of purchasers of outdoor pizza ovens are women. What happens is that Father’s Day, or dad’s birthday, or Christmas rolls around, and the family gets together to figure out what to buy for dad. He’s a big fan of barbecue, but he already has a grill. For that matter, they’ve already bought him every barbecue-related joke apron and every grill accessory on the market. 

That’s when they see an ad for an outdoor pizza oven. Problem solved! Next thing you know, every dad in the neighborhood has one. But almost none of them have bought it for themselves.

We didn’t figure any of this out until we’d already built and launched an entire marketing plan targeted at middle-aged men. Not too surprisingly, it wasn’t working. Our data suggested that there was a big potential market for pizza ovens, but we weren’t getting the sales we expected.

It wasn’t until we stepped back and realized that we’d been working from a broken assumption that we were able to change course and reach the real customer.

Assumption #3: I know what my customers really care about.

Hold up a second. Let’s take a break and talk music. Have you heard any good tunes on Zune lately?

Of course you haven’t, because Zune was a massive failure.

Zune failed for one very important reason. Microsoft, being the enormous force of nature that it is, assumed that customers would want its music player even if it was late to the market. They assumed that, as long as they included cool features, or as long as their technology was better than what was available (which is debatable, but anyway), people would switch to Zune from whatever music player they were using.

Wrong.

By the time Zune came out, everyone already had an iPod. And while Microsoft was busy building music-sharing and streaming into the Zune, YouTube came out. 

Microsoft assumed they knew what customers wanted. They assumed that certain cool features were customers’ most important priority. But it turned out that what customers really wanted was an easy way to listen to music now. And once they had their music on one device or app, they valued the convenience of keeping it there over any specific feature Zune might have offered.

The dust heap of corporate failure is piled high with such assumptions. Blockbuster assumed customers loved the immediacy of same-day rental over the convenience of movies delivered to their homes. Netflix won. And of course, even Netflix had to keep shifting focus as customers switched from wanting home delivery to wanting streaming services to wanting original content.

Case Study: Product Feature or Peace of Mind?

When we started Resident Home, we made what we thought was a pretty safe assumption. We assumed that when someone bought a mattress, what they were most concerned about was the mattress itself. The tangibles. In particular, we assumed they would care a lot about the particular type of foam the mattress was made of.

And they do. People want a mattress that’s made out of high-quality materials. What they really care about, though, is the intangibles. 

A mattress is a pretty big purchase for most people. It’s expensive enough to take some time thinking about, and you keep it long enough to want to make the right choice.

That concept of “wanting to make the right choice” ended up being the deciding factor for a lot more of our customers than we expected. When we were starting the business, we wanted to differentiate ourselves in as many ways as we could. At the time, the industry standard was a 100-day trial period and a five- or ten-year warranty. We saw an opportunity. Right off the bat, we offered a 365-night trial period and a forever warranty.

To be clear, we did not think this would be a key differentiator for us. Our website content focused on foam quality and other tangible features because we still assumed that was what customers cared about. The trial period and warranty were just “added bonus” differentiators that we included because we could.

In fact, we almost didn’t do it. We went back and forth around a bunch of assumptions. For one thing, we thought young people who moved a lot—like college students—would take advantage of the year-long trial period to get a new mattress every time they changed dorms or apartments. We thought there would be an overwhelming number of returns. None of that ever happened.

What did happen was that customers saw our trial period and warranty as a sign that we believed in our product and a way to make sure they were getting this purchase right. Intangibles won the day. If we hadn’t been constantly on the watch for the possibility of being wrong, we wouldn’t have been able to shift so quickly. Fortunately, we have an experimental, data-driven approach that showed us how to capitalize on the new information. We’ll talk more about that in Chapter Two.

The clincher is that now, others have adopted the year-long trial period and the forever warranty. We (kind of accidentally) upped the game for the whole market. 

Believing you know what your customers want without evidence is one of the more dangerous and potentially costly assumptions you can make because it drives so many other decisions. Product features. Shipping and sourcing. Marketing. Even new product rollout timelines.

Any time you make a decision based on what you think customers want, step back and ask yourself, “But do they?”

Assumption #4: I know how much customers are willing to pay.

This one is pretty straightforward, but we’ve been surprised by how many times we’ve seen it cause problems for founders. 

Just like with choosing what product to sell, the real danger here is love.

Founders tend to start businesses around ideas and products they care about. Maybe you love custom high top sneakers. Maybe you have a passion for a particular kind of truffles or an imported Italian flavor syrup.

Or you’re a mountain biker who has finally discovered the only water bottle that really works for you.

It doesn’t matter what the specific product is. You have a deep and passionate feeling about it, and as a result, you are willing to pay a substantial amount to get just that exact right thing.

Unfortunately, your willingness to pay that amount doesn’t in any way reflect whether other people will. It certainly doesn’t tell you how many people will pay that much, which is the key question in whether there’s a real market for your business.

You might even call your love for your product a kind of delusional state. If a lot of your friends have the same passion, you might not even be aware of how far you’ve drifted from how much the average person really wants what you’re selling.

Almost any product on Earth has some potential market. People buy $12,000 watches and $3,000 Louis Vuitton Jenga sets (seriously—look it up). There’s coconut brandy that sells for a million dollars a bottle. 

For most founders, that’s not where to start building a sustainable business. Scalability is based on the size of the overall market, not the existence of a niche. In fact, some founders confuse “small market” for “unique value proposition.” In most cases, if a niche is small, it’s more likely to be maxed out already.

We coached one founder who wanted to sell high-end, imported olive oil. This guy knew his olive oil, and he loved it. He could talk for hours about flavors and textures and all the other reasons why this particular oil was worth the cost. He even knew exactly how many people in the U.S. buy olive oil every month and had a whole spiel for why every one of them should buy his product instead. 

The thing is, the average consumer who purchases olive oil is used to paying about $7 a liter, and he didn’t actually know how many—if any—of those customers would convert to a higher-cost option. 

A much better way to see what people will actually pay is to ask them. There are two main ways to do that. The first is to set up a survey. There are tons of sites online that will send out a survey to a bunch of strangers for very low cost. That will give you some unbiased, objective data about whether people want your product and what they would pay for it.

But there’s still a difference between what people say they would pay, and what they’ll actually put down money for. That’s why the best test is to set up for pre-orders. Get a super minimal ad and website up and see whether anyone actually clicks “buy.” Until they put their credit card information in, everything they tell you is theoretical. The only way to know what they’ll really pay is to see . . . what they really pay.

🔓 UNLOCK: YOUR WILLINGNESS—OR YOUR FRIENDS’ AND FAMILY’S WILLINGNESS—TO SPEND A CERTAIN AMOUNT OF MONEY FOR A CERTAIN SET OF FEATURES IS A BAD GAUGE OF THE ACTUAL MARKET. USE CUSTOMER SURVEYS, OR EVEN BETTER PRE-ORDERS, TO GET REAL DATA.

Assumption #5: I know where customers will buy my product.

We will never know just how much revenue we missed out on because we didn’t get into retail right away.

It’s not that we failed to get retail contracts or turned down good opportunities to sell in retail stores. We just decided, ahead of time, with no evidence at all, that our customers wouldn’t want to buy our product in a store. 

If anything, we were anti-retail. 

We got all excited about the ecommerce revolution. We were going to be direct-to-consumer at a high level. We would cut out the middleman. Customers could buy our mattresses from their bedrooms. They wouldn’t even have to put on pants. Who wouldn’t want that?

Plenty of people, apparently. To start with, not everyone lives in the middle of Silicon Valley, where ecommerce was born and lives. Not everyone exists in the tech bubble or wants every interaction to be touch-free. Because of our own experiences and preferences, we assumed all of our customers would prefer to buy direct online. As a result, we missed a lot of people—and an entire year of retail revenue.

Once we saw our error, we turned in the opposite direction. We went multi-channel. Now we’re anywhere our customers are. Direct through our website? Yes. Through Amazon? Sure. At Target? Absolutely. If people are buying mattresses there, we should be there.

Of course, that doesn’t mean we’re done making assumptions about where and how to sell our product, which also means we’re not done being wrong. 

As we’re writing this, we’re in the middle of rethinking our retail strategy all over again. We started out with the belief that whatever products we have available online, we should have those same products available in retail stores.

But is that true?

Honestly, we don’t know yet. We think it’s possible that different products might sell better in stores than online. Our assumption that we had to sell the same items in person that we sell online could be limiting our ability to get retail contracts or to sell to in-person customers. The point is, we’re still assuming we could be wrong about these fundamental “facts” of our business: even the “fact” that we are an ecommerce business. That might always be our core, but then again, it might not.

Case Study: Dyeing for a Change

Before we started Resident, Eric founded Madison Reed, a company that sold hair dye. His experience running that company (which was and still is successful, by the way) has been super helpful in building Resident because he had the chance to make mistake after mistake after mistake. He learned to assume he was wrong, test his assumptions, make changes, assume those changes were wrong, and start the process all over again. That gave him a ton of insight into the kinds of beliefs founders bring into their businesses.

One assumption he made was that hair dye customers would love to have something new and innovative. As startup founders, we tend to love cool new stuff, so we believe that customers will also be impressed by cool new stuff. That’s what will make them want to buy from us! The cool new stuff will be our differentiators!

So when he started the company, Eric decided that hair dye customers would love a high-tech new color mixer and applicator to improve the way they put color on their hair.

Long story short: they didn’t. They didn’t want to have to learn a whole new way of dyeing their hair. They were happy to have the product packaged better, or higher quality. But having already figured out how to use the dye, they didn’t want that to change.

Fortunately, Eric found this out just before investing a ton of money in a long, expensive R&D process. If he hadn’t done the work to find out what customers cared about, though, he could have gone straight down a rabbit hole with no payout at the other end.

The point is, customers didn’t want to make changes to their hair care routine. They wanted a better, or more affordable, or easier-to-access version of what they already liked. The same is true with location. Customers want to be able to get the stuff they need at places they already are. 

When you think about where to sell your product, don’t think about where you would want to buy it or how you might get customers to buy it differently. 

One of the biggest mistakes you can make is thinking you can convince customers to make a special trip to buy from you, whether it’s a trip to your website or a trip to the local grocery store. Find out where they already are and be there waiting for them.

Assumption #6: I know I have a viable business.

This one is a little different from the first five. Those are all customer-facing. They’re about being realistic and data-driven about who your customers really are and what they want.

Assuming that you have a viable business is about what goes on behind the scenes. And it’s a much more insidious—and much more dangerous—assumption because if you don’t make the business run, it doesn’t matter how great your product is or how much customers want it.

Eric says: “Businesses don’t fail; they just run out of money. Don’t run out of money.”

We will talk much more in the second half of this book about the specifics of managing cash flow. You’ll know so much about cash flow that you won’t ever want to hear the term “cash flow” ever again. (Although we promise you’ll be glad you learned.)

The important thing to remember here is to not assume that your business is viable just because you have revenue or sales. Plenty of businesses make sales and still run out of money. That’s because they assume that money coming in equals profitability, and nothing could be further from the truth.

The ultimate evidence for whether you have a viable business is your P&L, or profit-and-loss statement. The problem is that, when you first start a business, your P&L is just a series of assumptions. 

If you’re like most founders, you are making a ton of guesses when it comes to potential profitability. You’re assuming that your current sales can predict your ongoing sales and that prices in your category won’t suddenly shift wildly. You’re assuming that you understand how seasonality, holidays, and life events will affect your customers’ purchasing choices.

You’re likely also assuming that your shipping rates will remain stable and that your vendor won’t go out of business and that your current customer base in the Pacific Northwest won’t suddenly become a customer base in Florida, increasing your shipping rates exponentially and forcing you to move to a new warehouse in a different state with different taxes.

And have you even thought about returns? Like, what the actual return rate on your product will be, and how much it will cost for customers to ship it back to you?

Like we said, we’re going to get to all of these specifics later in the book and teach you how to prepare for and manage them. 

For now, just ask yourself this: had you considered all of those issues already, or is this the first time you’ve considered that what you “know” about your bottom line might be assumptions?

Plus One

Oh, yeah. There’s one other big assumption that almost every ecommerce founder makes that can tank your business fast. They assume that running an ecommerce startup is a lot like running any other startup. A lot of ecommerce founders come from the SaaS and software development worlds, and they think, “Hey, we’ll get a website up and start selling.”

The assumptions that work in building a SaaS or other digital-only business do not work when you’re talking about selling a physical product.

Now that you don’t have to send floppy disks in a box anymore, selling software is stupidly easy. The customer clicks a button, the software downloads. Voila.

Even the after-sale issues are easier in software. There’s a bug in the code? Just fix it and hit the “send” button to send everybody the new version. A customer isn’t happy with their purchase? Turn off their access and call it a day.

Selling a physical product does not work this way. If you choose the wrong product features, too bad. You’ve already bought 100 or 500 or 10,000 of the things, and now you have to sell them or eat the loss. If a customer isn’t happy with the product, you have to find a way for them to ship it back to you, or dispose of it, or whatever.

That’s why, in this book, we’ve paid special attention to the unique problems of e-commerce: supply and manufacturing, shipping and logistics, packaging, returns, and so on. 

Assumption or Common Knowledge?

In a second, we’re going to get to an exercise where you’ll dig up all the assumptions you don’t realize you have about your business. Before we get there, though, we want to talk about something we get asked about a lot: the difference between assumptions and common knowledge.

Common knowledge is something everyone agrees on because it’s been proven true so many times that it would be a waste of time to prove it again. Assumptions are beliefs that we hold without any (or enough) evidence. 

How can you tell the difference? The key indicator is that businesses can’t succeed without doing the stuff that’s common knowledge. Assumptions, on the other hand, tend to be more unique to your business and your value proposition.

Free shipping is a great example of common knowledge. Thanks to Amazon and other big online sellers, customers expect free shipping. It’s not a differentiator anymore. You may choose to do free shipping over a certain dollar amount, or you might offer free shipping for all purchases, or some other variation depending on your bottom line. But there’s no question anymore about whether customers want it, and it won’t be a unique offering that sets your business apart.

That is the essence of common knowledge. 

Often, you can tell something is common knowledge because it feels like “table stakes,” or the bare minimum. Having a website used to be a differentiator. Now it’s common knowledge that you won’t reach your potential as a business without one. Common knowledge can also relate to known customer behaviors. We know that some customers will return the product. Some customers will cancel their orders. You don’t know up front what the exact return or canel rates will be, but you do know you have to have processes for dealing with these things. They won’t differentiate your business, but you can’t succeed without them.

Assumptions, on the other hand, tend to be specific to you. They might relate to the problem you’re trying to solve or what your particular customers want.

If you make socks and you believe that your new kind of wool will keep people’s feet warmer in colder climates, that might be a powerful differentiator for you. But only if it’s actually true—and only if people really care about it enough to pay for it. 

If you sell a snack product and you believe that customers will pay more if it’s fair trade or environmentally friendly, it could pay big dividends to get those certifications. But again, only if that assumption is true. If customers just want a lower price, those fancy differentiators won’t make any difference. They could even turn customers off.

You get it.

The point is, the way you as a founder should respond to assumptions is very different from how you should act on common knowledge. If something is common knowledge, you should just accept it as a baseline requirement for your business and move on. Look at what everyone else is doing and do that. Don’t reinvent the shopping cart! Pick a plugin or format that customers already know how to use and move on. (We’ll come back to this idea in Chapter Three.)

Assumptions, on the other hand, are both riskier and potentially more valuable. In the early stages of your business in particular, you need to identify any assumptions that are driving your decision making and test, test, test.

Okay, So You’re Wrong. Now What?

Knowing you’re wrong is only useful if you can find a way to be right instead. Now that you know where you’re likely to be making assumptions in your business, it’s time to start testing. Nothing—nothing—beats real data.

It’s not your job to get everything right. It’s not your job to avoid mistakes. Those are impossible goals. 

As a founder, you have one job: testing your hypotheses.

The only achievable goal is to create hypotheses, test them, make changes, and test again. Fortunately, that’s also how to build a successful, sustainable business.

Why is hypothesis testing so important?

Well, for starters, it’s important because everything changes.

Did you wake up this morning and open your letters? Did you dictate your correspondence into a dictaphone? Of course not. Because that’s not how business is done anymore, no matter how insanely futuristic it felt to people in the 1920s. And in the world of ecommerce, things change even faster than in other businesses. The ad that was dominating on TikTok last week could tank this week. Google could change the way it allows you to talk about your company in your business listing. 

Or, you know, a worldwide pandemic could shut down almost all in-person shopping for months or years. Crazy, right?

The point is, even if you somehow managed to get every, single, solitary aspect of your business absolutely right, that would last somewhere between 24 hours and a few weeks before everything changed again. Constant testing is the only way to keep up.

The other reason hypothesis testing is important is that you could be doing almost everything in your business a lot better than you’re doing it now.

Maybe this sounds basic, but we’re constantly amazed by how many new founders overlook it. They set up their business based on what they assume or believe (or want to believe), then they assume the results they get are the best they can expect.

We hear this kind of complaint all the time. My ads aren’t working. People aren’t finding us on Google. People keep abandoning their shopping carts. My warehouse keeps making mistakes on our orders.

Yeah. That’s what happens when you run a business. A lot of what you do is not going to work. The solution is to start hypothesis testing before you even start the business, then keep doing it, all day every day.

How to do Hypothesis Testing

The entire rest of the book will help you understand what to test, how to test it, how to measure or assess the results, and how to change or improve what you’re doing.

But the basics of hypothesis testing are the same, no matter what area of the business, no matter what you’re trying to fix or understand.

Step One: Actually Create a Hypothesis

Strange as it sounds, this is probably the most commonly overlooked step. When something goes wrong, the tendency is to go into panic mode. Change something! Fix it! That’s when you start setting up one-off solutions, quick fixes, band-aids that will just cause more problems than they solve.

Instead, step back and think. Come up with your best guess about what is going wrong. Surveying customers or checking out what people are saying about you on social media or review sites can help you generate better hypotheses about what might be going on.

Again, you might be wrong. But you can’t test something until you know what, exactly, you’re testing. If you go making changes willy-nilly, even if something does work, you won’t know what worked, or why.

Case Study: What’s in a Hypothesis?

Does it really matter whether you come up with a hypothesis? Let’s find out.

Here’s a pretty common scenario: you’ve gotten to the point where your product is selling reasonably well, and things seem to be working. Then, all of a sudden, your return rate spikes. Last month, about 15% of your customers returned their purchases. That’s pretty normal. This month, that rate is suddenly . . . 40%.

Nearly half of your customers returning your products is not good. Not only does it eliminate the revenue you thought you had from those purchases, it also costs you for shipping, generates fees at your warehouse, and causes a cascade of other side-effects. 

The question is, why? Customers were happy last month. What’s going on now? It could be a lot of different things. It could be your warehouse. Their sloppy product-handling processes could be causing lots of orders to be shipped in a damaged condition. It could be the shipping company you’re using, either damaging products or delaying shipments. It could be your manufacturer, changing their specs and putting out a lower-quality product. It could be a social media campaign against your company or even your entire category (plastic straws, anyone?). Or any number of other reasons.

If you don’t know why it’s happening, you can’t know how to fix it. The solution to a warehouse problem is very different from the solution to a PR problem. Jumping in with assumptions about what’s gone wrong can lead you down the road of expensive, time-consuming fixes that either don’t work—or make the problem worse. (Increasing your pro-plastic straw ad spend won’t increase sales if the problem is the perceived environmental impact of your industry.)

That’s why the first step to hypothesis testing is the hypothesis. Come up with every single possible reason why the problem might be happening. Do your best to figure out which one is correct. Then create a way to test that one problem. If that doesn’t fix it, test something else.

Otherwise, it’s not testing. It’s just whack-a-mole.

Step Two: Create a Test

Like we said, the second half of the book will provide you with all kinds of detailed ways of figuring out what’s probably gone wrong and how to test and fix it. The lesson here is to create a test that will let you find out whether your hypothesis was right or wrong as quickly as possible.

If you think your return rate problem is a quality control change at your manufacturer, you can either visit, or you can get some of the product delivered to you to check yourself, or you can get your representative to visit the factory, or what have you. Any of those would be quick ways to test whether quality is the problem.

The key to a good test is that the information is readily or easily available, reflects the real world as accurately as possible, and will give you insights into how to fix the problem.

One example of a common type of test that doesn’t work as well as people think is consumer interest surveys. Let’s say you want to find out whether people will buy your new brand of socks and how much they’re likely to pay. You create some cool info pages about your socks, show them to people, and ask, “Would you buy this?” and “For how much?”

The problem is, everyone wants everything. Sure, they will say that your socks are cool and that they want them. They might even claim that they’ll pay ten or twenty bucks a pair.

But trying putting the “buy now” button there and ask people to input their credit card info. Suddenly the number of super-excited, “definitely would buy” potential customers becomes just a few real “will pay for this” customers. The closer your test is to reality, the better it will be.

Step Three: Measure, Improve, Repeat

Obviously, hypothesis testing isn’t just about running tests. It’s about making things work again. Once you’ve decided on a test, run it and see what information you get back. Then change something and test it again. Do this constantly in every area of your business.

If you wait until there’s a problem to hypothesis-test, you can identify and fix problems. But if you’re always testing and collecting data, you’ll be able to get there a lot faster. 

Are you gathering “reasons for return” from customers so you know what to change to avoid returns in the future? Is someone in your business constantly looking at the reviews of your product on marketplaces like Amazon or on Google so you can get ahead of complaints before they grow? 

If a test doesn’t give you something specific to change, it’s not a good test. If it doesn’t provide you with a way to measure whether the problem improved, it’s not a good test. Making specific changes and measuring their results—and then testing again—is the heart of this process.

And when we say test it again, we don’t just mean this particular problem. We mean you should always be testing everything, as much as possible.