Product Principles - Part I: Decision Making

Recently, I’ve been spending a lot of time bringing my written notes from the last few years into the present.

Most of those notes are from my time at Choose Energy as part of the product team. Thanks to the power of Roam, I’ve been able to organize those notes, capture common themes across them, and combine them with what I’m learning in venture.

This post and the three that follow will focus on my product management principles. I divided them into the following sections:

Before we get started, a note on the misnomer that product managers are the “CEO of the product”.

Admittedly, I was a sucker for this metaphor and it’s why I wanted to get into product in the first place. The responsibility of owning the vision and resulting P&L is exciting for anyone who wants accountability.

However, a more appropriate explanation for product management is “assistant coach”. I know it will not inflate anyone’s ego in the same way, but the analogy is a better fit. Why?

With that being said, here are the principles I think served me well in product and I hope will be useful to you as you craft your teams and vision going forward.

Like almost everyone, I’m a big fan of Annie Duke’s “Thinking in Bets”. I largely put myself through college playing online poker and she was a fixture at the World Series of Poker during those days. Some of the skills I learned while playing have stayed with me and still influence my thinking.

One of those concepts is “pot-odds” which is the size of the pot v. the size of a contemplated bet. For example, if the pot (potential winnings) is 4 and a call (bet to continue seeing cards) is 1 the “pot-odds” are 4 to 1. In poker, you compare these odds with the probability of winning a hand to estimate the expected value.

Product management has a lot in common with poker and finance except the bets are made in resources and time in addition to capital to arrive at an expected value. The basic concept of right-sizing resource allocation in relation to the expected outcome is fundamental to great product management.

Sizing “bets” in product management is dependent upon your confidence which then determines the speed and quality of execution.

The more confidence you have in any one bet, the higher the bar for quality in execution because the product or feature is more likely to create long-term value. The lower your confidence, the more important it becomes to prioritize speed and, in turn, learning which I’ll discuss later on.

The second-order thinking on confidence is metaprobability - the probability that your confidence interval is correct. You should track this regularly, and ask yourself how often is your confidence accurate?

The last aspect of sizing bets correctly is time to results. Some bets take a long time to play out and some take only a few weeks.

Think of it this way, investing in a private fund often requires a 10-year commitment and the public stock market is liquid. As a result, the bets you take in these markets are very different due to the potential for re-allocating resources. Product is the same way.

The default state for most product teams is to optimize for the practical over the valuable. As a result, decisions are made primarily on effort and value alone while neglecting potential upside.

In early-stage companies, the low effort, immediate value features are abundant. You can easily increase top-of-funnel, create more engagement, or increase conversion with a few small tweaks.

However, value is a near-term construct, and lasting businesses are built on the upside. The ability to capitalize on upside results from understanding deeply where you can miss and where you should take risks.

As Jeff Bezos and the Amazon team said recently, “to invent you have to experiment, and if you know in advance that it’s going to work, it’s not an experiment.” Great product teams experiment where they can afford to miss, but where a hit could have massive implications for growth.

You shouldn’t rush decisions that have large implications or pass on all low-effort features, but you can’t forget the calculated risks that could unlock growth for your business.

When building products there are three buckets to consider:

The goal should be to spend as much time as possible in the first two buckets as possible. The highest performing software companies don’t just nail top-line growth, they retain customers at outstanding rates.

Spending too much time in maintenance mode indicates you are not focused on value creation opportunities and that you’ve grown past your current capabilities.

The “right” ratio of time spent on each bucket depends on where you are within your company’s lifecycle, but a good goal should be somewhere around 60-30-10 respectively.

If you find yourself spending more than 10% of points on maintenance regularly, think about taking a step back to understand the underlying problems and how you can fix them. If you’ve built good customer relationships (see below), you should be able to manage through the pain.

If you interrogate data long enough, it will tell you what you want to hear. Consequently, we cannot assume data equals reality, but instead represents something at a single-point in time and by definition can no longer exist.

Software lives in a dynamic world, therefore data is a guide, not a law and we must be willing to consider additional information that is often qualitative. Understanding what data means, and how it is moving takes a fair amount of understanding to fully grasp.

Take this example regarding time spent on a website. We’d assumed time spent on our site was a good thing while a customer was shopping, and a bad one when they were checking out.

However, customers weren’t really spending a lot of time in our shopping experience. How did we know? We started recording browsing sessions, and it turned out consumers would shop, but leave tabs open for HOURS as a “bookmark” to get the task done later.

This completely changed how we prioritized time spent on our site, and how we handled presenting offers to customers. We started refreshing inventory more regularly, and using features like timeout clocks used on ticketing websites to create urgency.

Data is the map, reality is the terrain, act accordingly.

Now that we have some basic principles for how to think about making decisions, in the next post I’ll cover how we can execute on those decisions.