Tuck Knowledge in Practice Podcast: Experimenting as an Entrepreneur

[This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of the Tuck Knowledge in Practice Podcast is the audio record.]

Hart Posen: The world is a world of noise, uncertainty and ambiguity. Yet we still have to act. What we know from psychology is that if left to their own devices, we will make these decisions in inconsistent and potentially inappropriate ways if we don't decide in advance. So we think about this as a design question. If you're going to be an entrepreneur, how do you design the learning process? You know, this image of a certain kind of person becomes an entrepreneur. And I think the evidence for that is pretty darn flimsy.

[Podcast introduction and music]

Kirk Kardashian: Hey, this is Kirk Kardashian, and you're listening to Knowledge and Practice, a podcast from the Tuck School of Business at Dartmouth. In this podcast, we talk with tuck professors about their research and teaching and the story behind their curiosity. This episode is a conversation with Hart Posen, professor of Strategy and entrepreneurship at tuck, and we talk about how entrepreneurs can experiment and pivot to navigate their way to startup success. Hart Posen, welcome to Knowledge and Practice. Thanks for being here.

Hart: Thanks for inviting me.

Kirk: The topic today is something you're quite familiar with entrepreneurship. Um, not only have you been a successful entrepreneur, but it's also one of your primary areas of research and teaching. And as you know, today we're going to be discussing your recently published paper called Programs of Experimentation and Pivoting for Overconfident Entrepreneurs, which was just published in the Academy of Management review. Um, basically, the paper deals with the question of how much an entrepreneur should experiment and when they should pivot to a new approach or idea. Um, now, to me, entrepreneurship probably has a little bit more mystique Surrounding it than many other areas of business. It seems like it's part science, but also has a healthy dose of art in it. Would you agree with that?

Hart: As both a former entrepreneur or a recovering entrepreneur and a scholar who studies entrepreneurship and strategy, I think the mystique is, is is real. And I think the interesting question is what is the source of the mystique? And the source of the mystique is that entrepreneurship is a domain of human action that is sort of shrouded in ambiguity and uncertainty, and it is that ambiguity and uncertainty that makes it so hard to think about it as this very systematic endeavor. How do you make plans? How do you, in a world where you know you're heading into an abyss and in my view, you know, the bigger the upside of the entrepreneurial venture, the more ambiguous and uncertainty uncertain it is. And then the question is, well, how can you think about that in some sort of systematic way? Right. And even entrepreneurship varies, right? If you're opening a corner store, this is not a world of ambiguity and uncertainty, right? If you're starting an AI startup, it's the abyss. Yet you still have to do stuff every week and every day and every hour and have to act. Is it a craft? It's got some elements of a craft to it, but I think we can think about it systematically.

Kirk: Yeah. And that's what you do in a lot of your research.

Hart: It's what I do in my research. It's what I do in the new course I'm developing for tech called Startup Strategizing and a book I've just started to work on to go with the course is all about how we can think about this systematically and in many ways, it's a response to my own experiences as a young entrepreneur in the late 80s through most of the 90s. I came from a family of entrepreneurs, yet it was very hard to see any systematic way to proceed. And even when I was successful, it's not clear that I did anything systematically different than when I wasn't successful when I was an entrepreneur in the 90s. So, so as a as a kid, I studied computer engineering and my first startups were tech startups. And then in retail spaces. You know, the conventional wisdom back then and until fairly recently, was what I'd call one of commitment. Heart doesn't give up. He just keeps plowing ahead. And so, you know, this story of sort of perseverance and commitment, I think, is a story that extends well beyond the narrower milieu of entrepreneurship to sort of broader society and how we talk to our kids about this. And so that was the sort of, um, world in which I grew up as an entrepreneur. Over the last decade or so, that's changed. There was, um, uh, this idea that comes out an idea called The Lean Startup, based on a book by Eric Ries of that name. And then, um, a very somewhat different book by, by Steve Blank. And they made a very different argument. They said, don't commit, fail fast, fail often. And, you know, clearly the world is somewhere in between. But the two stories are completely orthogonal. Right? They're opposite extremes of the world in the scholarly literature. These ideas existed well before. But the work of Ries and Blank really brought it to sort of the Silicon Valley. Foreground, which tends to permeate discussions of entrepreneurship.

Kirk: One question that raises for me is what's the connection between the “Lean startup and pivoting.” Like, how does pivoting, I guess, signify a lean startup? What does a lean what does the lean part of the lean startup.

Hart: If one is going to. Pivot their way to success? The lean idea is that you want to conduct the experiments on which you'll make your pivot decision with the minimal experimental costs. But of course, that's where the problem with the logic comes in. And the problem with they talk about something they call a minimum viable product, an MVP. This is also a term that's sort of become quite widespread, and that's sort of like the minimum prototype or what have you, that you need to develop to get feedback from the world to figure out if it's a good idea or not. And the problem is this the world never gives you the answer. The world gives you a little bit of data, and from that you've got to infer. It's like, you know, we run clinical trials on new drugs. The world never comes out and says this will cure heart or not. The world gives us this really noisy signal that we need to interpret. Now, the minimum viable product idea is do the bare minimum possible to get the answer you need. Yet the answer we need is this is not what we get from any small scale experiment. All we get are noisy signals. And so the fundamental trade off for challenge is not, you know, typically discussed is, is that if you're sufficiently lean and your experiments are sufficiently small, you'll get highly noisy data, yet you still have to decide pivot or not.

And so I think this notion of lean and this notion of pivoting, I think, create a fundamental tension. Now, if you create a prototype of your new technology or your new product and all you get is noisy data back, very noisy data back, and you still have to make a decision, you are still living in the world of tremendous ambiguity and uncertainty, right? Yet at the same time, I think the one thing that this logic does for us is it says, well, we can start to think about this as a pathway through that ambiguity and uncertainty. A way of thinking about how to deal with it. A way that is not in the Lean Startup. But I've been trying to work on over the last number of years through a number of my papers, including the one where we're talking about today. How do we plan? You know, it's a weird thing. How do you plan for uncertainty? And I think this is what I'm trying to get at. How do you design? How do you plan for uncertainty in your startup, and how do you make a set of upfront decisions about how you'll deal with it, as opposed to simply reacting along the way?

Kirk: So basically, you're what you're saying is entrepreneurship is a process where you there's lots of things you don't know, and you're trying to get information On which you're going to make decisions, right?

Hart: So for years people have talked about and it's true in the popular literature and the academic literature, people have talked about entrepreneurship as learning, but they typically mean learning in a half hearted sense in the academic literature. It would be hard did something he got feedback from the world. He learned does next thing. Over the last few years, I've been working on a program of research that takes on that idea and says, can we take learning seriously in in an entrepreneurial setting? And what I did in a paper about five years before, the one we're talking about today was I said, let's take entrepreneurship and thinking about it as two time span, separated by a critical decision, a Pre-entry time span and a Post-entry time span. So Hart comes up with a great idea. I wake up in bed. I've got an idea. You know, this is sort of the caricature of entrepreneurship and of innovation, a falsely caricature. Well, I wake up with that idea the next morning. I don't get out of bed and decide I'm going to invest $1 million or $10 million or $50 million in the venture. In fact, I don't make my entry decision for some time. Let's say, I think it's such a great idea. I'm going to allocate a year to gathering data and learning if it's a good idea, at which point a year from now, I'll make my decision, invest my million dollars and start a business or move on.

And so I've been working on a stream of research that thinks about entrepreneurship like that, that says there's this pre-entry period where you're going to learn, you're then you're going to make an entry decision, pay some fixed cost to enter. That could be an opportunity cost to quit your day job, Or it could be you built a factory and then you're going to enter, and then you're going to be selling. And every day you're going to make this decision, should I stay in or should I close it down and go back to my old job? And, and so I've been playing with that, with that, with that basic model. And so it's a model that says there's learning pre-entry to make the entry decision. After you enter, you earn profits or losses in the world. And every period you decide, should I stay or should I go? It looks like a caricature of what a new venture looks like. And in that earlier paper, this is back in 2018. What we showed is that simple model tended to produce a lot of sort of the stylized facts. We thought about it for entrepreneurship. So for instance, that simple model produced, uh, excess entry, more entrants, more entrepreneurs entering the market.

Then you might think would be economically viable. That simple model produced delayed exit. I tend entrepreneurs tend to stay in the market longer than they should given they entered, and it's not working well. And that simple model produced a correlation between the cost of entry and how long they stayed in the market with a bad idea. We might call it the sunk cost fallacy. And what we show in that paper is that simple two stage learning process produces these stylized facts that are well known in research on entrepreneurship, not because entrepreneurs are biased. In fact, in the model, entrepreneurs are perfectly rational, yet we get excess entry, delayed exit, and sunk cost fallacy because of the nature of the learning process and decision making. And that started me. That work started me on this path of thinking about, okay, if that simple learning process. If that simple two stage learning process produces those stylized facts. How can we design the process of entrepreneurship in a way that overcomes some of those challenges? And those stylized facts emerge in the model precisely because of ambiguity and uncertainty? If, when Hart woke up, he tried something and the feedback he got from the world was the perfect truth, none of those problems would exist. It is because the feedback you get from the world when you go out to do something is heavily noisy.

I used to talk about this, you know. Hart comes up with an idea. He asks Bill Gates. Bill Gates says, “Hart, you're a genius. That's a great idea.” And the next day he runs into Elon Musk and he says, “Elon, what do you think?” And Elon says, “That's the dumbest bloody idea I've ever heard.” And the next day he runs into Mark Zuckerberg and Mark Zuckerberg says that's genius. If you've ever been an entrepreneur, you know it looks exactly like that. Maybe you're not talking to Musk, Zuckerberg and Gates. You're talking to your mother, your friends, your colleagues, potential customers. But that's exactly the kind of feedback you're getting. Very noisy feedback. And so in that two stage learning process, simply noisy feedback is enough to produce excess entry, delayed exit, and sunk cost fallacy. For fully rational actors in the model. And so then the question is how do we design the process of entrepreneurship to counteract those effects? And across a series of work, I've been thinking about how to design this process. In the paper we're talking about today. It's designing the process of pivoting and experimenting. In other work, We looked at how to design entrepreneurial teams. Kirk, if you and I are partners, how should we design decision rights in our partnership to reduce these problems?

Kirk: So let's talk about that program of experimentation. In your paper, you focus on two choices an entrepreneur makes when following this program of experimentation. Um, tell us about those two choices and why you focused on them.

Hart: So most of my work, Kirk, is computational models. So it's a mathematical model that's solved computationally. What you do in those kind of models is you take the real world, which is much, much more complicated than my model, and you strip out everything down to the most central elements. So if we're going to strip the world down to its most central elements, and we're going to think about this notion of, um, uh, commitment and persistence on one hand and, and flexibility and pivoting on the other hand. What are the minimum things we need to do? Well, in the simplest, remember the simple model I told you about a minute ago? There's a pre-entry period. Call it one year at the beginning of that period, you've got your grand idea. You're going to take a year to gather data and decide if you should spend your $1 million and enter. And if you enter every day, you'll decide, should I stay in or should I go back to my old job? Well, that's a story of commitment. Pre-entry right during that year Pre-entry you just keep beavering away at it. You just keep working away at it until you have to make your decision to enter or not. A pivot story and would look different. Let's imagine a world where you're going to do, um, two experiments. So the story with no pivoting, there's sort of one experiment. You do the one experiment, you run it for a long time.

So you get lots of data on that experiment. At the end of the year, you make your decision to enter or not. Well, let's go from one experiment to two experiments. The divide that pre-entry period into two experiments first six months, second six months in between as a pivot decision. So at time zero, you wake up in bed and you've got your grand idea. You'll work on it for six months, you'll get data. Now you're going to have to make sense of your noisy data, your noisy six months of data and decide, should I go with that variant of the idea for another six months of experimenting on it? Or should I pivot to some other related idea and now have six months to experiment on it. So that takes us from the base model of one year on one idea, with no pivoting to one year with one pivot opportunity in the center. If I don't pivot, then I'll have studied that one idea for the entire year. But if I pivot, I'll have had six months on the first version and then another six months on the second version. And at the end of that full year, then I'll make my entry decision. And so then you can take that idea of dividing it into half and then into dividing it into thirds. Et cetera, et cetera. And what this what this produces for you is something very interesting.

It highlights the fact that that doing more experiments and potentially pivoting comes at a cost, and the cost is this how much you learn. If you run one experiment for one year, you'll collect a lot more information and you'll be. Remember that information is still going to be noisy. But all else being equal, you'll make a pretty good decision on it. Now, if I cut that in half and I do two experiments with a pivot opportunity in the middle before making my entry decision at the end of the first year, you're going to have the opportunity to evaluate two different ideas, but you only get half as much information on each. This is the fundamental tension. If experimenting and pivoting more didn't engender any cost, then one should do as much pivoting and experimenting as possible. But there is a cost, and the cost is an informational cost. And then this sets up the trade off. If you've got one year of dollars to spend to make your entry just over which you're going to make your entry decision, you have to make the tradeoff between getting less information on more elements of the idea versus more information on fewer ideas. What we do in the paper is study the nature of this trade off. Right. And to think about how you can design that trade off optimally given the conditions that you face. You see drugs pulled from the market by the FDA after they've been on the market, sometimes for a year, two years, three years, millions of people have taken them.

What this is saying is that the time they went on the market, the signal was still very noisy. The FDA made a decision to put it on the market. Then after many more, millions of people took it and they had more. They said, oops. Entrepreneurship is very much like is very much like that. So you have to decide how many experiments to run and how are you going to decide whether it's been a success or not? If it's a success, stay with the stay with the idea. If it's a failure, switch to a new idea. These two things turn out to be related. The number of experiments to run, and the threshold for deciding on whether it's a success or not have to move in tandem. Because as you run more experiments, given a fixed time window, each experiment gets noisier. Shorter because it's shorter, you got less data. The results are noisier, so you need to change the cutoff point. And what we know from psychology is that if left to their own devices, we will make these decisions in an inconsistent and potentially inappropriate ways if we don't decide in advance. So we think about this as a design question. If you're going to be an entrepreneur, how do you design the learning process?

Kirk: This maybe gets to an important component of your paper, which is the role of behavioral biases in designing a program of experimentation. Right. Tell us tell us about how behavioral biases can influence how you design this program of experimentation.

Hart: So this is a I think for us in this paper a the critical idea, you know, these days in popular television shows and elsewhere, we talk endlessly about behavioral biases, right? We talk about Daniel Kahneman's work and others. This idea that we that humans don't act in rational ways. Now, a long story about what rationality means in that context. It means something very narrow and something very specific. But the big idea is that humans vary in how they respond to the world, and we don't make decisions in necessarily consistent ways. And there's a lot of research that argues that entrepreneurs, like all people, are inherently biased in the way we handle decision making. And so what we wanted to show in this paper is that this aspect of design, designing the program of experimentation, how many experiments you'll run, how you'll decide if they're a success or a failure, can address behavioral biases. So in the paper we look at two kinds of, uh, Confidence, biases, overestimation, and overprecision. I'll just talk about the former. Overestimation is, you know, when I wake up in my bed at night with my great idea. It's true. Probability of success might be 50 over 50, but I overestimate that true probability, I think. My goodness, isn't this a great idea? I think it's got an 80% probability of success. That's estimation bias. Okay, so Hart wakes up. He thinks it's got an 80% probability of success. In truth, it's only 50 over 50. And now I engage in my learning process. And my question was, can we design the program of experimentation to counteract that bias? Because we look at two kinds of biases in all combinations. There are a variety of sort of sets of biases. But the key thing is that we can we can show that you can design this program of experimentation and entrepreneurship in a way that mitigates the negative effects of bias.

And I think this goes back to sort of something I care about a lot as a former entrepreneur, you know, discussion of entrepreneurial bias and overconfidence in what have you is, is, is so prevalent. And it makes it sound as if it's some sort of affliction of humans. And I think this is probably the wrong way to think about bias as an affliction, rather than we are all different in how we make decisions. And when you see bias in decision making as a sort of affliction, a flaw, then you ask, well, can we make heart better? Can we fix him? Can we make him more, you know, more unbiased? I think there's little hope of that. Nor do I even think it's a good idea, because I think there are ecological and social benefits to these kinds of biases. But I do think that that we can design the entrepreneurial learning process to, to, to alter how these biases affect outcomes. And by shaping that process, we, we think about then the quality, you know, the nature of the outcome as effect is a function of both the biases of the humans that are engaged in entrepreneurship and the design of, of the learning process. So, so then we move from a world where saying, well, it's biased. Well, we might want to fix them, but we can't fix them to a world where hearts biased. Everyone is biased in different ways. Can we design our learning process in a way that best takes advantage of what heart brings to bear? And in this paper, we show how you can start to think about that. Okay.

Kirk: So can you give me an example. So say you have the overestimation bias. You wake up and you think your chances are 80% when they're actually 50. How might you design your experimentation to neutralize that bias.

Hart: Yeah. So one thing, one thing you might do is change the evaluation criteria for an experiment. You set the bar higher, but if you set the bar higher, you also probably need to change the number of experiments you run. It's a it's a whole interdependent system. You can't move one lever without moving the others. But it starts to say that, hold it. You know, we can design this process of experimentation to work best with the individuals that are engaged in doing it, that it's not the same design for you and for me because we're different, right?

Kirk: It's sort of customized to the entrepreneur. That's right.

Hart: And so, you know, the big deal in the paper is less the details of how it's customized. Then the idea that we can think about entrepreneurship as a learning process and be we can then design a learning process that works well for you. Really, since the first dotcom boom in the in the 90s, in the late 90s, you know, this image of a certain kind of person becomes an entrepreneur. And I think the evidence for that is pretty darn flimsy. I think lots of people can become entrepreneurs. Lots of people find success and a lot of failure because that's the nature of the game, right? Well, over 90% will fail, but it's not a function of personality Analogy type or the type of biases you have. It doesn't mean those things are irrelevant though. And what I believe is we understand who we are. We can design the nature of the program of experimentation in this paper we're talking about, or we can design the decision-making rights in a founding team in a way that is personalized to, to, to our type. It says that you're not a prisoner of your personality traits or biases, right? Uh, I really don't like this idea that, um, only people with certain traits can or should be entrepreneurs and can be successful as entrepreneurs. Uh, I don't think there's any evidence for it. And in fact, I think by saying there are all kinds of people out there, uh, how can we best leverage the unique things they bring to bear? And the design of the entrepreneurship process? Uh, is, is a much more useful approach to thinking about it.

Kirk: We're going to put a link to your paper in the show notes. Um, but maybe you could just help by summarizing for us. What are your key takeaways from your paper for entrepreneurs or for just firms and managers in general?

Hart: So for years we've talked about many things as a learning process. We talked about entrepreneurship, we talk about innovation, but we tend to do it in this slipshod kind of kind of way. Uh, the key overarching idea is that we can design the learning process. It's a design problem. And if you have a design of a learning process, whether it's the learning process associated with deciding after you have an idea of whether you should enter the market with your idea or not, or you're working on a new product or technology in an established firm, a rather than allowing that learning process to be sort of ad hoc. We can design it. You wouldn't send your kids to college where there was no program of learning and set courses and exams and other things. Yet we're surprisingly, I say this because I have a daughter in college, yet we're surprisingly, surprisingly willing to do that with our entrepreneurial ventures. Wing it. You know, a little bit of winging it is fine. Um, but we can think about it systematically, and I think it's incumbent on us to think about it systematically. You know, Kirk, much of what I do is a response to my own experience. 2 or 3 decades ago as an entrepreneur, I guess three decades ago, I'm getting old. Uh. Uh. I see the possibility of taking a process that is seen as wholly unstructured and, and, and starting to add some structure to the nature of that, of that process. And structure doesn't take away free will, but it gives you a way to function and a way to think about the problem systematically. Um, I really liked your opening remarks, because I think it is exactly trying to get away from this world where everything is foggy and gray and with no well laid out process to start to say, you know, how do you do it? You design a, you design a process. We design processes for learning in other domains. We can and we should do it here as well.

Kirk: Yeah. You mentioned you're working on a on a book. Is that structured learning process going to be part of the book?

Hart: It'll be one part of the book that takes on this a different idea, and it takes the another, another outcome of this work on the Lean Startup has been this idea of product market fit. And your job as an experimenter is to figure out if you're if you've got product market fit. It's a fancy word. All they mean by it is customers like it and want to buy it. And the key premise of the book is, while that's important, there always has to be someone who wants to buy your product. Uh, in many domains of entrepreneurship, the product market fit is not the hard part. Uh Um.

Kirk: It sounds like you're saying you're making.

Speaker3: A big speculative bet. If they're.

Hart: Investing that there are other aspects of the nature of innovation, the nature of value chains that are, in fact, the critical thing, I.

Speaker4: Believe that it's very easy to predict that they're going to be lots of successful companies born of the internet that are going to have very large. So there's.

Hart: This, um.

Speaker4: And so on.

Hart: Great interview. Jeff Bezos gave with CNBC. I think in like the late 90s, just after Amazon went public, maybe it was 98, 99. 97 I don't remember what year the interview was. And this is the height of the dotcom boom still. And the interviewer, who did a fairly good job, was asking Bezos about his plans for Amazon. Remember, it's the late 90s. It's still a small firm.

Speaker4: But I believe that if you can focus obsessively enough on customer experience selection, ease of use, low prices, more information to make purchase decisions with. If you can give customers all that. Plus, he says, we're.

Hart: Going to make customers really happy. And he goes on and on about this, and then he says, and we're going to spend billions of dollars building warehouses all around the country.

Speaker3: You're not really a pure internet company anymore either, are you? I mean, you've got millions of square feet now of real estate. You've got a growing.

Hart: And the interviewer interrupts Bezos and says, you know, Jeff, if you go build all those billions of dollars of warehouses, you're not going to be an internet firm anymore. Remember, this is the late 90s. And Bezos response was brilliant.

Speaker4: It doesn't matter to me whether we're a pure internet play. What matters to me is that we provide the best customer service internet, internet that's, you know, that doesn't matter.

Hart: Internet. Some internet. And what he was saying was, I know what will make customers happy and we'll make customers happy. Remember, this is the late 90s, so delivery was like two weeks if you're lucky. Whereas if we can get that delivery time down from two weeks to ten days to seven days to five days to two days to one day to same day, darn it, that'll make customers happy. So the problem to be solved at Amazon is a problem of outbound logistics.

Speaker4: What we're trying to do is very complicated. There's huge execution risk involved. We have a terribly complicated business. We're growing, you know, historically very rapidly.

Hart: And they had.

Speaker4: To.

Hart: Raise tens of billions of dollars to build these facilities. They had to invent all the technologies needed to do it.

Speaker4: This is the less risky of the two approaches.

Hart: And so, you know, it's easy to ask about product market fit, but sometimes the problems that need to be solved have nothing directly to do with the customer. In fact, in many cases, the problems that need to be solved have little to do with the customer. Not always, of course. And so the book is about that. And then the link to this. My work on learning is how do you figure that out? How do you think about that?

Kirk: I'd like to thank my guest, Hart Posen. You can find his paper in the show notes. You've been listening to Knowledge and Practice, a podcast from the Tuck School of Business at Dartmouth. Please like and subscribe to the show. And if you enjoyed it, then please write a review as it helps people find the show. This show was recorded by me, Kirk Kardashian. It was produced and sound designed by Tom Whalley. See you next time.

Speaker5: Yo, T-Bone, did you produce this?

Speaker6: Sounds good. Right?

There's no guarantee that Amazon.com can be a successful company.