Back to Blog

Brad Feld and I Discuss Data

April 22, 2012

What do you do when you have to make decisions in an uncertain environment with only mediocre data?  Startup founders and investors face this question all the time.I had an interesting email exchange on this topic with Brad Feld of Foundry Group. First, let me say that I like Brad and his firm.  If I were the founder of a startup for whom VC funding made sense, Foundry would be on my short list.

Now, Brad has an Master's in Management Science from MIT and was in the PhD program. I have a Master's in Engineering-Economic Systems from Stanford, specializing in Decision Theory.  So we both have substantial formal training in analyzing data and are both focused on investing in startups.

But we evidently take opposing sides on the question of how data should inform decision-making. Here's a highly condensed version of our recent conversation on my latest "Seed Bubble" post (don't worry, I got Brad's permission to excerpt):

Brad: Do you have a detailed spreadsheet of the angel seed data or are you using aggregated data for this?... I'd be worried if you are basing your analysis... without cleaning the underlying data.

Kevin:  It's aggregated angel data....   I'm generally skeptical of the quality of data collection in both... data sets.... But the only thing worse than using mediocre data is using no data.

Brad: I hope you don't believe that. Seriously - if the data has selection bias or survivor bias, which this data likely does, any conclusions you draw from it will be invalid.

Kevin: ...of course I believe it....  Obviously, you have to assess and take into account the data's limitations... But there's always some chance of learning something from a non-empty data set.  There's precisely zero chance of learning something from nothing.

Brad: ... As a result, I always apply a qualitative lens to any data (e.g. "does this fit my experience"), which I know breaks the heart of anyone who is purely quantitative (e.g. "humans make mistakes, they let emotions cloud their analysis and judgement").

I don't want to focus on these particular data sets.  Suffice it to say that I've thought reasonably carefully about their usefulness in the context of diagnosing a seed investment bubble.  If anyone is really curious, let me know in the comments.Rather, I want to focus on Brad's and my positions in general. I absolutely understand Brad's concerns.  Heck, I'm a huge fan of the "sanity check".  And I, like most people with formal data analysis training, suffer a bit from How The Sausage Is Made Syndrome.  We've seen the compromises made in practice and know there's some truth to Mark Twain's old saw about "lies, damned lies, and statistics." When data is collected by an industry group rather than an academic group (as is the case with the NVCA data) or an academic group doesn't disclose the details of their methodology (as is the case with the CVR angel data), it just feeds our suspicions.I think Brad zeroes in on our key difference in the last sentence quoted above:

...which I know breaks the heart of anyone who is purely quantitative (e.g. "humans make mistakes, they let emotions cloud their analysis and judgement").

I'm guessing that Brad thinks the quality of human judgement is mostly a matter of opinion or that it can be dramatically improved with talent/practice.  Actually, the general inability of humans to form accurate judgements in uncertain situations has been thoroughly established and highly refined by a large number of rigorous scientific studies, dating back to the 1950s.  It's not quite as "proven" as gravity or evolution, but it's getting there.At Stanford, I mostly had to read the original papers on this topic.  Many of them are, shall we say, "difficult to digest." But now, there are several very accessible treatments.  For a general audience, I recommend Daniel Kahneman's Thinking Fast and Slow, where he recounts his journey exploring this area, from young researcher to Nobel Prize winner.  For a more academic approach, I recommend Hastie's and Dawes' Rational Choice In an Uncertain World. If you need to make decisions in uncertain environments and aren't already familiar with the literature, I cannot recommend strongly enough reading at least one of these books.

But in the meantime, I will sum up.  Human's are awful at forming accurate judgements in situations where there's a lot of uncertainty and diversity (known as low validity environments).  It doesn't matter if you're incredibly smart.  It doesn't matter if you're highly experienced.  It doesn't even matter if you know a lot about cognitive biases.  The fast, intuitive mechanisms your brain uses to reach conclusions just don't work well in these situations. If the way quantitative data analysis works in practice gives you pause, the way your brain intuitively processes data should have you screaming in horror.

Even the most primitive and ad hoc quantitative methods  (such as checklists) generally outperform expert judgements, precisely because they disengage the intuitive judgment mechanisms. So if you actually have a systematically collected data set, even if you think it almost certainly has some issues, I say the smart money still heavily favors the data rather than the expert.

By the way, lots of studies also show that people tend to be overconfident. So thinking that you have a special ability or enough expertise so that this evidence doesn't apply to you... is probably a cognitive illusion too. I say this as a naturally confident guy who constantly struggles to listen to the evidence rather than my gut.

My recommendation: if you're in the startup world, by all means, have the confidence to believe you will eventually overcome all obstacles. But when you have to make an important estimate or a decision, please, please, please, sit down and calculate using whatever data is available.  Even if it's just making a checklist of your own beliefs.

Further Reading

Enjoyed this post? Here are a few more posts that you might find just as insightful and engaging.

For Founders

Minimum Viable Investor Updates

For pre-seed and seed stage startups, investor updates are a challenge. Often, founders try to make them too ornate and end up getting behind. Similarly, investors don't always have the time to fully digest a finely crafted narrative and lose track of what's happening. At RSCM, our portfolio of pre-seed and seed-stage investments is at about 2000 today, so we have lots of experience with updates. Not only do we read them all, we write a 3-7 line internal summary and each one goes into our CRM system so we have a complete history at our fingertips.In my opinion, useful investor updates have three requirements: they must get done, they must be easy to produce, and they must be easy to consume.

Anatomy of an Update

You can deliver on all three requirements by breaking updates into modules and putting the most important modules first. That way, you need only produce the modules you have time for and we need only consume the modules we have time for. Everybody wins.Here are the modules and order I recommend:

[Company Name] Investor Update for Month Ending [Last Day of Month]
  • Metrics
  • Highlights (Optional)
  • Asks (Optional)
  • Thank Yous (Optional)
  • Commentary (Optional)

Notice that the only required module is "Metrics". This should be easy to produce because, at any given moment, you should have a handful of Key Performance Indicators (KPIs) you track anyway. This should be easy to consume because most investors have lots of experience absorbing tabular business data. This should be easy to get done because, in our modern software-driven world, KPIs are at your fingertips. Most importantly, if they are the metrics you are actually tracking to run your business, then they will be reasonably informative to investors. Requirements satisfied!More detail on metrics in a minute, but first some quick notes on Highlights, Asks, and Thank Yous. If you opt to include these modules, do them as bullets. Easier to produce and easier to consume. But, as with PowerPoint slides, no more than 7 bullets per section! Even then, only go to 7 on rare occasions. No more than 5 most of the time. It's easy for people to get saturated and when they get saturated, they flush the entire list from their attention. If you've got more to say, put it in the Commentary.Everything after Metrics really is optional. Better to get the update out the door quickly than wait until you come up with points for every section. If you ever find yourself thinking something like, "I'll crank out the Asks later," stop! Just hit send. Then if you do think of important items later, put them in a notes file and include them in the next update. Or send out a specific Asks email.

Universal Metrics

Now for some depth on metrics. There are really two types: (1) those that are universal to all pre-seed/seed startups and (2) those that are particular to your business. Investors need both. The first type gives us a general sense of how things are going for you relative to the typical startup lifecycle.  Kind of like the vital signs that all doctors want to know regardless of patient or condition.  They help us triage our attention. So start with them:

  • Revenues: [revenues | date when you plan to start selling] (+/- ?% MoM)
  • Total Expenses: [expenses] (+/- ?% MoM)
  • Net Burn: [total revenues - total expenses] (+/- ?% MoM)
  • Fundraising Status: [not raising | planning to raise | raising | raised]
  • Fundraising Details: [how much, what structure, valuation/cap, who]
  • Ending Cash: [last month's Ending Cash - this month's Net Burn + this month's Amount Raised] (+/- ?% MoM)
  • Full Time Employees: [FTEs, including founders] (+/- # MoM)

Note 1: we strongly encourage a monthly update cycle. Anything longer means we get data that's too stale. Anything shorter, and the financial metrics don't really make sense. Though if you're part of an accelerator that encourages weekly updates, we'd love to see them. Just make sure we also get the monthly metrics!Note 2: always put the percentage or absolute month-over-month changes in parentheses next to each entry. It turns out that highlighting the deltas make updates dramatically easier for us to absorb by drawing immediate attention to the most volatile areas.A couple of quick explanations. Always have a Revenues line. If your product isn't finished or you aren't actively trying to generate revenues, just put the target date for when you do plan to start selling. Either piece of information is enormously helpful to us. Also, provide an FTE number that logically reflects the labor resources at your disposal. A full time contractor is a unit of full time labor that you can call on. Two half-time employees are also one unit. An intern may or may not be a unit or fraction of a unit depending on how much time he/she is putting in and whether the output is roughly equivalent to what a regular employee would produce. Don't exclude people based on technicalities, but don't pad your numbers either.Now, some detail about fundraising status. This topic turns out to be pretty important to existing investors. First, it lets us know that you're on top of your working capital needs. Second, some investors like to participate in future rounds and even the ones that don't are a great source of warm leads. Third, it makes us feel good to know that other people have or will be validating our previous investment. Here are a couple of example fundraising bullets:

  • Fundraising Status: planning to raise in 4Q2015
  • Fundraising Details: $750K - $1M Series Seed at a $5M-$6M pre-money from a small fund and/or local angels
  • Fundraising Status: raising
  • Fundraising Details: $300K - $500K on a convertible note at a $2.75M cap with $175K soft committed from [prominent angel name] and other local angels
  • Fundraising Status: raised and raising
  • Fundraising Details: $400K closed of a $600K convertible note at a $4M cap from [small fund name], [AngelList syndicate name], and local angels.

Custom Metrics

At any point in time, there should be a handful of top-level KPIs that you monitor to help run your particular startup. Of course, they vary across lifecycle stage, technology area, and business model. Just pick the most important 2-6 and give them to us. Feel free to change them as you pivot and mature.Here's an example for a pre-product enterprise SaaS company:

  • Projected Alpha Delivery Date: 11/30/2015 (+15 days)
  • Alpha Access Wait-list: 47 Companies (+8)

And one for an enterprise SaaS company that recently shipped private beta

  • Max Queries/Minute: 1,201 (+29% MoM))
  • Outstanding Critical Bugs: 3 (-2)
  • Inbound Inquiries: 481 (-17% MoM)
  • Qualified Prospects: 19 (+2)
  • Paid Pilots: 3 (New Metric!)

And finally one for a consumer Web company in full operation

  • Max Concurrent Users: 1,006 (+30% MoM)
  • Registered Users: 23,657 (+13% MoM)
  • Monthly Actives: 3,546 (+4.5% MoM)
  • Users Making Purchases: 560 (+21% MoM)
  • Total Purchase Value:$17,993 (+28% MoM)
  • CAC: $12.55 (-7% MoM)

That's it. We estimate that, if you keep your accounting system up to date and use MailChimp, producing an update with metrics and a few extra bullets should take about 15 minutes (with some practice). And you'd be heroes in our book.  Well, all entrepreneurs are already heroes.  So you'd be superheroes!

This post originally published on 10/15/2015 and was last updated on 11/10/24.

For Founders

#1 Mistake: Planning for Series A?

People sometimes ask us, "What's the #1 mistake startup founders make?" Based on our 2000 pre-seed portfolio companies, one of the prime candidates is: "Planning for Series A."I don't mean the way you plan for Series A. I mean the fact that you do it at all. We see a lot of pre-seed pitch decks. A decent fraction have a "Comparables" section that list the Series A raises for companies with similar models in the same industry. In these cases,  Series A has become an explicit planning goal, despite the fact that these companies are at least two rounds, and probably three or four, away from that milestone. But the prevalence in pitch decks vastly understates the issue. From systematically interviewing 800+ founding teams in accelerators, it's clear that Series A expectations play a substantial role in most founders' planning.

While completely understandable, even considering Series A at the accelerator stage is usually a huge mistake. As I've written before, taking Series A at the point where it's appropriate decreases your success rate (though increases your expected value). Unsurprisingly, actually working backward from a future Series A can create all sorts of planning pathology. Yes, TechCrunch makes a big deal out of Series As. Yes, lot of cool VCs blog about Series A. Yes, VC investment leads to pretty fantastic story lines on "Silicon Valley". But these sources of information inherently screen for outliers. It's still the exception. Even among successful tech startups. Fundamentally, you're trying to engineer an extreme outcome in a highly uncertain environment. On first principles, this is problematic, as Nassim Taleb so beautifully explains the The Black Swan. But let's work through the steps.

Start with a modern Series A of roughly $10M as your goal. OK, those VCs will want evidence that you can quickly grow past the $100M valuation mark. That means you'll probably need about a $3M Series Seed 12-24 months beforehand to build the necessary R&D, sales, and customer success scaffolding, as well as prove out a huge addressable market. This in turn implies a $1M angel round coming out of an accelerator to complete the full-featured version of the product and establish a firm beachhead market over the next 12-18 months.

Now, I can tell you from reading the investor updates for 2000+ pre-seed startups that such rounds are very hard to raise... unless you're a strongly pedigreed founder, have obviously anti-gravity level technology, or have crazy traction in a hot space. We like to say rounds at this stage have a "geometric" difficulty curve. A round that is twice as large is four times as hard to raise.

Even if you manage to raise that round, the failure rate at each subsequent stage is high because you're continually striving to achieve outlier levels of growth. There's not much room for error or setbacks.  It's like trying to run up a ridge that just keeps getting steeper and narrower, with a sharp drop into the abyss on either side.

So what's the alternative? We recommend you ask yourself, "What's the smallest early acquisition (but not just acqui-hire) that I'd be satisfied with?" Unless you have a significant previous exit, are already very wealthy, or have unusual risk preferences, this number is likely somewhere between a $10M and $35M acquisition where the founders still own about 1/3 to 1/2 the company. Then work backwards from that.

Now, you may be saying to yourself, "Wait a minute! If I could get acquired for $10M to $35M, I could get a Series A. It's the same thing." Not exactly. $20M is a typical Series A pre-money these days, at least from a traditional name firm. But you would also need to be able to demonstrate that you could quickly grow to be worth $100M+. And you usually get a bit of a premium on acquisitions. So it's only at the upper end of the range where a Series A would be a fit, and then only some of the time.

Importantly, acquirers mostly want to see a great business or great technology and Series A investors mostly want to see enormous growth potential, which often aren't quite the same thing.

Finally, Series A investors usually want to see extremely rapid past growth, as an indicator of rapid future growth. Acquirers care much less how much time it took you.Also, the cost of being wrong is asymmetric. Say you aim for Series A from the outset. If at any point it doesn't work out, you either fold or do a fire sale. In a fire sale, liquidation preference will kick in and founders will get zilch anyway. Conversely, say you go the smaller route and things go much better than expected. You can still "upgrade" to the Series A path. And if you go the smaller route and fail, there's some chance you'd still make a modest amount in a fire sale or acqui'hire.

So now let's work backwards from the acquisition. We'll assume that revenues, rather than technology capability, is the relevant metric because it makes the reverse induction more clear cut.

  1. In most tech sectors, a $10M to $35M acquisition means $1M to $3M per year in margin (not gross revenues, though in some sectors, the margins are so high, it's the same thing). That's low $100Ks of margin per month.
  2. Next, we like to think in terms of the "straightforward scaling factor". This is the multiple by which you can grow with straightforward scaling of your product development and sales machines. No major overhauls of the product, no completely new channels, and no huge breakthroughs. Basically keep doing what you're doing, but with more resources. In most segments, this factor is 3-4X for a target in the $100K/month order of magnitude. Obviously, it's not a sure thing. Bad things can still happen. It can turn out that you've made a mistake. But it's the difference between needing circumstances not to go strongly against you and needing circumstance to go strongly for you. That works out to $20K to $80K per month, depending on scaling factor and target outcome. Thus, your near-term goal becomes, "Build a business doing $20K to $80K per month in margin."
  3. If your minimum acceptable exit is on the higher end and your scaling factor is on the lower end, you might want to break this stage into two (though your might want to ask yourself why your minimum is higher given the lower scaling factor). In most cases, the first step therefore reduces to, "Build a business doing $20K to $40K per month in margin."

This is often a very achievable goal with a very modest amount of capital. How do you go about raising a round to support achieving this goal? Well, we have a post for that.

It's worth noting that, in terms of our expected returns, it doesn't matter too much to us one way or another whether founders follow this plan. Our funds have many hundreds of companies, so we're expected value decision makers. Though there is also some argument to be made for preserving option value by having companies survive longer. But it's not a huge difference either way at our level of diversification.

However, for founders who can only do a handful of startups in their career, understanding the difference between success probability and expected value could be literally life altering. And don't forget, once you have a modest exit under your belt, you've got the pedigree! So it's much easier to command the resources and attention necessary to go big from the start on the next one.

‍

This blog post originally published on 12/10/2020 and was last updated on 10/14/2024.

Report: How Are Pre-Seed and Seed VC Firms Investing in 2024?

The venture market bottomed out from historic highs last year. Total deal volume slumped roughly 50% from 2021’s peak, exit activity hit a ten-year low, and venture fund performance dropped across the industry. These rapid changes have created a new landscape for venture capital, and it’s affected how VCs are investing.

Right Side Capital surveyed 110 Pre-Seed and Seed VCs from February 2024 to May 2024 on their investment activity and strategies in 2023 and their plans for 2024, with a focus on Pre-Seed Rounds and Seed Rounds. VCs revealed that they are optimistic about the funding landscape in 2024 and that they have high expectations for revenue levels and growth rates from portfolio companies.

Below we share what we learned.

VCs Were Active in Pre-Seed Rounds in 2023

Surveyed VCs revealed that they were fairly active in Pre-Seed investment last year. Of the VCs surveyed, 87.0% made at least one investment in round sizes of $1M to $2.5M, and 35.2% made more than five investments at this stage.

Seed Round Deal Volume Was Less Than Pre-Seed Round Deal Volume in 2023

VCs reported less deal volume in Seed Rounds in 2023 as compared to Pre-Seed Rounds during the same period. Only 12.1% of surveyed VCs made more than five investments at this stage, and 25.9% made no investments at all. The majority (62.0%) made between one and four investments at this stage.

Investment Outlook Is Optimistic in 2024

Nearly half (45.4%) of respondents plan to make five to nine new investments in 2024, which is a significant increase from 2023, and 24.1% said they planned to make 10 or more investments this year. All respondents planned to make at least one investment, which indicates a more positive outlook from 2023.

Pre-Seed Fundraising: What VCs Expect from Founders in 2024

At the Pre-Seed fundraising stage, only 46.3% of surveyed VCs will invest in a pre-revenue startup, 27.4% will invest in a startup with sub-$150K annual recurring revenue (ARR), and 14.7% require $150K – $499K in ARR. For some surveyed VCs, revenue expectations can be even higher: 11.7% said they required startups to have $500K or more in ARR.

Growth expectations are high for Pre-Seed Rounds, with 34.8% of surveyed VCs expecting startups to double year over year at this stage, and 37% expecting startups to triple year over year.

Seed Fundraising: What VCs Expect from Founders in 2024

Expectations vary a lot for startups raising their seed rounds. At this stage, 17% of surveyed VCs will invest at pre-revenue, but 24% want to see ARR of $1M or more. That’s a big change from four years ago, when $1M or more in ARR was the criteria for Series A funding.

Surveyed VCs expect aggressive growth at this stage, with 47% investing in startups that are doubling year over year and 34% investing in startups that are tripling year over year.

Most VCs Recommend 6-12 Months of Runway

The majority (53.7%) of surveyed VCs advise their portfolio companies to maintain six to twelve months of runway before raising their next round. Only 29.6% of VCs advise startups to have over 18 months of runway.

Capital Efficiency Is More Important Than Ever

VCs reported that, in this leaner landscape, they are placing a greater emphasis on capital efficiency for portfolio companies. For 81.5% of respondents, capital efficiency is more important than ever before. The survey included an option for respondents to indicate that capital efficiency was unimportant, but not a single respondent selected it.

Roughly One Third of VCs Have Changed Their Investment Thesis

We asked respondents to write in answers about how their firm’s investment thesis has changed in 2024. Below we break down the results of those write-in answers.

Summary of Investment Thesis Changes in 2024

No Change (58%) The majority respondents indicated that their investment thesis has not changed significantly from 2023.

More Focus on Specific Areas (15%) Some VCs have an increased focus on specific sectors such as health, cyber, AI, and cybersecurity. They’re putting a greater emphasis on software, particularly AI-powered applications, and avoiding certain sectors like consumer and hardware.

“Like everyone else, [we have] more interest in AI-powered applications.”

– Survey respondent

Adjustments in Investment Strategy (10%) Some VCs are shifting to smaller check sizes. They indicated more capital allocation for Pre-Seed and they are rightsizing investment amounts to achieve more significant ownership.

Greater Sensitivity to Valuations and Due Diligence (7%) VCs are more sensitive to valuations, ensuring companies have more runway, and conducting more thorough due diligence. They’re also focusing on financing risk, revenue, traction KPIs, and efficient use of capital.

“[We’re] thinking more about financing risk and making sure companies have more runway.”

– Survey respondent

Increased Sector Preferences and Deal Dynamics (5%) A small subset of VCs have a growing preference for companies with experienced founders, significant revenue, and efficient burn rates. They’re avoiding overinvested spaces like sales-enablement software and sectors that are seen as high risk for next-round funding.

“[We’re] rarely taking pre-product risk unless the team has prior operating experience.”

– Survey respondent

No Specific Answer or N/A (5%) Some responses were “N/A” or did not specify a change in investment thesis.

Final Conclusions from the RSCM 2024 VC Survey

The venture capital landscape in 2024 has adapted to a leaner and more cautious environment. Right Side Capital’s survey reveals a higher bar for revenue expectations and a greater emphasis on capital efficiency than in more bullish periods.

Despite the challenges of 2023, VCs are optimistic about 2024 and plan to increase new investment volume. Overall, VCs are adopting a resilient and forward-looking approach, emphasizing sustainability and capital efficiency to navigate the transformed economic landscape.

‍