Iteration Times Matter More Than Ever!

I wanted to update one of my previous reflections on iteration times. It matters so much, yet people goof this up all the time!

Biotech timelines are different from computational timelines. Coding has some feedback loops that happen on the order of minutes (coding, unit tests) and some feedback loops that happen on the order of weeks (iterating on product features/framing towards market fit). Good biotech feedback loops happen on the order of days (each protocol is usually at least a few hours, and most experiments are a stack of several different day+ protocols being ran back to back, where a delay or failure at any step cascades forwards), and quarters.

Knowing when to be patient is hard. In software it seems like you often, as a CEO / CTO can (and should!) have a decent sense of how long things are supposed to take. Not true in bio! In bio, experimental failure is overdetermined. Great scientists can debug faster than you can. You can support by making suggestions / making introductions to experts / sending along papers, but generally speaking every milestone has large error bars on how long it takes to actually get done. When the project is done, you can often look back and say “there was clearly a direct route to get here that could have happened at least 2x faster” but this is sort of inevitable when its hard to accurately prioritize the list of good fixes, and it’s impossible to truly parallelize debugging in every direction you should be trying.

How do you deal with timelines then? the biggest advantage I’ve seen in biotech R&D is to just make your iteration loops faster. I hesitate to name names here, but I know of multiple occasions where a should-be frontrunner, with substantially more funding or a multi-year head start, has been caught up to by a second-string team that designed experiments that could be done more quickly. A recurring theme here revolves around choice of model organism, where the front-runner picks a more biologically-accurate model, and the catch-up player chooses a worse model that can be scaled faster (in vitro vs NHP, fast growing cell types vs slow growing cells). A slightly less informative experimental design, which can be run >2x as fast, can compound into a huge informational advantage if you’re willing and able to run many iterations.

This sounds like it should be obvious dogma — any good engineering school will insist that you think first and foremost about your iteration times. Of course, biologists don’t take engineering classes! I’m just teasing, mostly... These front-running innovators are often full of very bright scientists. In thinking about this, I think there is some urge to scientific elegance that pulls scientists towards wanting complete, orderly data, avoiding surprises. Surely, you wouldn’t want to spoil your lead by getting to the end and finding out that your model was imprecise! And you have all this money and time, surely you can afford it. The catch-up player has no such allusion that their success is assured; as a consequence they take on a cleverer approach (or maybe this is a selection effect; unless you were confident you had an edge, perhaps a would-be david immediately folds against a goliath). Optimizing for faster iteration times also, frankly, can be a bit uncomfortable for the team. Most scientists have been trained at grad school, and many have had the experience of pushing an aggressive timeline. But then they graduate, perhaps they want to re-prioritize family, friendships, hobbies getting better sleep. Working fast is uncomfortable! Obviously it helps to be willing to do this though if you want to do some of the best work of your lives. It’s perhaps worth pointing out that, due to how the grad school system works (nearly impossible to fire a grad student, nearly all come in as minimally-skilled trainees), most scientists have never actually had the experience of sustained work in a uniformly high-talent-density team, and often underestimate how much is really possible in that environment.

Getting back to iteration times — the other significant way to deal with timelines is to run multiple attempts-on-goal in parallel. This sounds great in theory, but can get expensive in practice! The more money you spend, the faster you need to compound your returns to hit those >20% risk adjusted returns that investors expect. Therefore, even well funded orgs should think hard about how they can do things cheaply. In my experience (although I’m sure experience varies dramatically across subdisciplines), faster experimental designs tend to be correlated with more in-house labor costs, since fast iteration loops tend to building more yourself, from raw materials and without external suppliers. If labor costs are a major part of your cost function, finding ways to get labor costs down should be a high order operational concern, since unlocking low labor costs may open up project avenues that wouldn’t otherwise be feasible to parallelize. Labor costs are absurdly high in San Francisco and Boston, relative to the rest of the world, but most people in the SF/Boston bubbles underappreciate this (perhaps in part because scientists in SF/Boston are exposed to their exceptionally well-paid FAANG neighbors, who are perhaps the best paid sporadically-hard-working community in the world). Extending your operational reach beyond the SF bubble can help projects move faster and more freely, especially at the earliest stages when capital is tight, and when you’re doing your most unique-to-you work.

Lastly — why does it matter to be fast? Some obvious reasons — patents only last 20 years, capital costs aren’t what they used to be in ZIRP, competitive advantages compound if you can move quickly. But maybe the most important thing is that talented scientists want to be at the frontier, where the action is. Ultra-high-talent density is a fundamental force for progress in technology, but one which biotech tends to not really practice (and only occasionally preach) — for mostly idiosyncratic historical reasons.

For funders, here is my best advice: when things kick off, genuinely give the team 1-3 years to find it’s footing. Some projects will shoot off and start hitting milestones within 6 months, but many great projects (some of the best ones!) may genuinely take a few years in the wilderness before a path emerges. Try your hardest not to shake the box, or even necessarily open the box, while that process is happening — you risk distracting the team with pleasing you in the short term rather than finding something that will generate value in the long term. Scientists are often trained in academic environments; top donors often tour and are given goofy demos to wow them; I promise you that once scientists set on the task of giving you a sexy demo, they can most certainly do that, and it will also almost certainly be a waste of your time. The much better thing you can do is to promise that, provided they solve real problems for the real market, surely there will be funding available to continue their work (note this works regardless of whether you personally are providing that funding or whether you will help them pitch for further funding, or simply a statement that you have confidence that the funding market will be there — your mark of approval will do a lot). Your vouch of confidence and trust that they’ll pick something good will avoid distracting the best teams, and it also won’t save the worst teams, who usually lack the taste to pick good problems anyways.

One final note — this discussion here is really all about optimizing across a few dimensions of what it means to innovate in biotechnology, with an inventor’s/engineer’s mindset. Many biotech companies do amazing work, will help patients tremendously, but aren’t really innovating at the bench level. This is great for those companies — innovating genuinely new technologies is hard, risky, and therefore best to avoid when at all possible. But in biotech, it seems likely that sustained competitive advantage is likely to accrue to companies with the hard-earned knowledge, intellectual property, and recruiting edge due to their reputation as genuine innovators. I continue to believe that the cultivation of new “Catalytic Biotechnologies” is one of the most important things that someone like me, and perhaps you, the reader, can do with a life, and with the time you’re given for it.