About · Posts · Categories · Projects · Mailing List

What does it mean to be a data-driven Product Manager?

SQL queries? VLOOKUPs? Definitely. Add to that: metrics, data, KPIs. These terms have become commonplace at technology companies. If you're interviewing for a Product Manager role in 2019, I guarantee you'll be asked some of these questions about your past experience:

  • What were your KPIs? Why did you pick those?
  • Talk about a time when you used data to make a decision.
  • What metrics do you use to illustrate if a feature is successful or not?
  • When is it not appropriate to use data to make a decision?
  • Sketch out your data model.
  • Talk about a time when the data suggested you should go in a different direction from your strategy.

You'll need to succinctly explain what you measured, how you measured it, and most importantly why you measured it. You should demonstrate the ability to hypothesize and associate metric(s) to evaluate.

Demonstrate an ability to focus. "Out of the 10 things I could have measured these three were most important". Focus may be the biggest value a Product Manager can offer. The ability to say these are the few things we should measure and why. And then have the awareness to know when those metrics have served their purpose and it's time to measure something else.

Demonstrate the ability to question. Was there a time when the data misled you? How did you adapt? What was your goal and why was monitoring metrics part of the solution? A concerning answer for why you measured a certain metric/KPI is "we've always done it this way". Even if you do resort to status-quo industry standard measurements, explain the reason for that. It will demonstrate that you at some point questioned the status-quo, and received a sufficient answer that resulted in you maintaining it.

Some practical examples.

Today, tech companies are vying for your attention. YouTube prefers you watch their videos instead of Netflix's, or going to the movies, or reading a book. They want your time allocated to YouTube. This is why when you finish a video the next one is already queued up and a long list of tantalizing recommended videos is in clear view.

The way to measure attention is through Retention & Engagement.

Retention, getting you to come back (e.g. open YouTube X times per month). Engagement, getting you to use the product (e.g. watch 10 videos per day on YouTube).

The gold standard Retention measurement is "N-Day Retention". The goal with measuring Retention is to understand who and how often is coming back to your product. Amplitude, a tool I currently use has a great overview of measuring N-Day Retention.

Engagement is about measuring who and how often is performing the "key action" in your app. In YouTube's case one of those actions may be "watch video". The gold standard Engagement measures are: DAU ("dow"), WAU ("wow"), MAU ("mm-ow") DAU/MAU ("dow-mm-ow"). These metrics measure: Daily Active Users, Weekly Active Users, Monthly Active Users. These are the unique number of people that perform the key action (such as "watch video") on a daily,  weekly and monthly basis.

DAU/MAU will demonstrate how engaged your user base is by reflecting the % of monthly active users that come back everyday. This can also be a measure of your apps "stickiness". Again Amplitude has great overview of this concept. It's also worth noting that although DAU/MAU is an industry standard metric for Engagement, it has shortcomings.

If your focus is Engagement & Retention, DAU, WAU, MAU, and DAU/MAU are great pulse metrics. Define a company-wide standard to an active user. Be very specific. For example an active user is an account holder that watches at least 10 seconds of video in 24 hour period. Then measure them consistently. They will help track if you're product is improving over time, and signal if things are getting better or worse.

You will also need to come up with metrics that are more specialized to your product and goals. What metrics are you going to try to improve that will directly impact DAU, WAU, MAU?

Here is a famous example from the early days of Facebook. When Facebook opened up beyond colleges, they entered hyper user acquisition and retention mode. Facebook's growth team united around the following insight: 7 friends in 10 days. The team discovered that users that added 7 friends within 10 days of creating a Facebook account were likely to remain an active Facebook user. Therefore all of their focus (features, experiments, design decisions) became around getting as many users as they could into the "7 friends in 10 days" cohort. To maximize this metric, which became their key measure for engagement and retention.

There is a group of individuals I'll label as "wake up early individuals" (WUEIs). People that get an early start in order to tackle goals before the day begins.

For some it's waking up early and exercising. Jocko Willink consistently posts photos starting his day at 4:30 AM. The book "How Children Succeed" gives an example of a middle school chess prodigy who woke up early to practice chess. Joe Satriani, electric guitar extraordinaire practiced in the mornings before school

When I was a kid, I’d get up and practice guitar for an hour before school, and during that hour I’d do all the boring stuff just to get it over with. That way I could come home, do my homework and then jam with my friends.

How are they able to do it? In a world of distractions (mobile phones, YouTube, etc.) WUEIs find a way to go to sleep early and pull themselves out of bed to get after it. Jocko in his book "Disciple Equals Freedom" argues that discipline is the enabler:

Discipline: The root of all good qualities. The driver of daily execution. The core principle that overcomes laziness and lethargy and excuses.

And that waking up early is the starting point:

Discipline starts with waking up early. It really does. But that is just the beginning; you absolutely have to apply it to things beyond waking up early.

Discipline is one common trait WUEIs share. Fuse the desire to achieve a goal with discipline and you get an individual that will wake up at 4:30 AM. Someone that will do whatever it takes.

Yet discipline is only an enabler. It's a starting point. Showing up isn't enough.

Before discipline you set a goal(s). I want to be a: entrepreneur, author, musician, fit individual, etc. This broad goal (musician) may start to become a bit more specific: 80s shred guitar player.

And thus with your goal you channel discipline to show up and put in time towards reaching your goal. This alone will not be enough. For you can show up everyday at 6 AM and practice guitar, but if the practice isn't focused and the goal is open-ended, one year later you may have not made the progress you imagined.

You must set yourself up for success. So when you do show up you take full advantage.

Break down your goal by setting mini-goals with deadlines. This month I'll learn three 80s metal guitar riffs and will write three original ones. I'll also learn to play one full song. Even more specific: by the end of this week I'll learn one riff and the first 2 sections of the song. With clear goals you now have a roadmap towards where you want to be.

To fulfill the roadmap you'll need a system. The system may even impact how you define the roadmap (the path to reaching your goal(s)). Once you define the path your system is how you divide your time. If I have 90 minutes in the morning, my system may be 20 minutes guitar exercises, 30 minutes learning the song, and 40 minutes composing.

Your focus and attention must be deliberate. It's easy to fall into a habit of repeating the same system everyday. But you're showing up so it must be enough right? Just put in the time and results will follow. This is dangerous and you'll likely stagnate. Today you may need to spend 45 minutes learning the song and 45 minutes composing. Tomorrow it may need to shift again.

With deliberate focus you are constantly aware of the goal, your system, and the progress you are making. You make adjustments as necessary so you don't fall into a mindset that just showing up is enough.

If you combine discipline, goals, deadlines, systems, and deliberate focus, you will significantly increase the likelihood of achieving your goals.

In my teens and twenties I played a lot of pickup basketball. Didn't matter where or with whom - as long as the rim had a net and the ball had air I was game.

I've written this in the past, but I continue to marvel at the life lessons you can identify in a pickup basketball game. Obvious ones such as teamwork and sportsmanship to less obvious ones such as mindfulness, confidence and restraint.

Certain days (or every summer day in NYC) the court would get crowded. It could be a two to three game wait before you get to play. So once you get on the court both teams have a lot of incentive to not lose. In basketball recognition comes to the scorers. Players that put the ball in the basket. So naturally the moment you get the ball you're making a play with the intention to score. The problem is all your teammates likely share this mindset.

Add in the influence of razzle-dazzle players like Steph Curry and Kyrie Irving whose effortless handles and pure scoring abilities are viewed with such awe that the moment a player touches the ball, they feel they must replicate Curry or Irving in order to mesmerize and garner the respect of teammates and opponents. It becomes less about the game and more about showmanship. It's a formula for losing your spot on the court and waiting another two to three games until you get back on.

So what do you do? Complain the person guarding you that your teammates are ball hogs. Or get confrontational and call out your superstar teammates to shoot less, pass the ball and make the right play. But this is pickup basketball, no one likes a player coach.

So shift your mindset, shift your responsibility: plug the hole.

If your team can't get a rebound, get in the paint and get rebounds. If an opposing player is dominating offensively offer to guard that player (always offer nicely). [1] If you need to set some picks to get teammates open, set the picks. The mindset is to do whatever is necessary to set your team up for success.

In most cases that means refraining from launching a three ten feet behind the line and turning around and celebrating prematurely as the ball drifts through the air. [2] It requires swallowing some pride and realizing that in this game your team doesn't need you to take ten shots. Identify the hole and make it a personal challenge to plug it. See how much you can influence the game even if you don't take a single shot. Teammates that know the game will appreciate you and will go out of there way to set you up offensively.

And if you win the game you get to stay on the court - and maybe in the next game you'll plug some offensive holes.



[1] Do it strategically and nicely. There is a lot of pride on the basketball court. If you just got scored on a couple times in a row it's embarrassing and frustrating to have a teammate tell you to switch on D. By telling you to switch with them are implying that you are not good enough to guard that player and they will now take over defensive responsibilities to show you how it's done. Don't be that teammate. Instead tell them they are doing a solid job, but you are trying to get better at defense and want to improve your skills. Ask them if they would be OK with you trying to guard their defensive assignment for a few plays. If they say no back off. But in most cases they will agree and appreciate you not showing them up on the court. It may also reciprocate back to you as they become more likely to pass you the ball in appreciation for your help defensively.

[2] Actually in all cases. Just don't do that. I'm a fan of confidence. But not at the cost of fundamentals. This new trend of shooting the ball and immediately celebrating by turning around and walking away because you "just know" that you made the shot is ridiculous. It's showboating and should be avoided.

In his essay "How to Get Startup Ideas" Paul Graham writes:

The verb you want to be using with respect to startup ideas is not "think up" but "notice." Since what you need to do here is loosen up your own mind, it may be best not to make too much of a direct frontal attack on the problem—i.e. to sit down and try to think of ideas. The best plan may be just to keep a background process running, looking for things that seem to be missing.

Awareness, the ability to notice is a fundamental life skill that can release you to pursue greater things. For Paul Graham it's a strategy to unlock startup ideas. For those that practice mindfulness it's the foundation of the practice.

Thich Nhat Hanh in his seminal book on meditation "The Miracle of Mindfulness" introduces the reader to meditation through awareness. Having a process running where you are constantly aware of what you are doing presently. Even when putting a book back on the shelf:

While placing a book on the shelf, look at the book, be aware of what book it is, know that you are in the process of placing it on the shelf, intending to put it in that specific place.

What is awareness? How do you channel it? Tara Brach in "Radical Acceptance" writes:

When thoughts arise, where do they come from, where do they go to? As you explore looking into the space between thoughts, through the holes in the net, you are looking into awareness itself. You might sit quietly and simply listen for a few moments. Notice how sounds arise and dissolve back into formless awareness. Can you notice the beginnings of sounds, the ends of sounds? The spaces between? It is all happening in awareness, known by awareness.

As you begin practicing awareness you'll notice a transformation in your outlook to the world. Signs you may have missed previously become clearer. Feelings you've suppressed may become nurtured. Your relationships with others, with food, with experiences may become more significant.

Starting small and finding awareness in moments everyday can lead to more awareness throughout your experiences. It's a habit that can easily be dismissed or neglected. And yet finding ways to remind yourself, to continue to notice, can establish a habit that can have a profound impact on how you live life.

Tuesday night, a long workday has passed and you have an hour before bed. You try to muster up the energy to work on your personal project but it doesn't happen. You put it off - I'll have time and energy on the weekend you say to yourself.

The weekend arrives and you've slept in. You have brunch plans. You go for a walk after. You have to buy groceries. You go out Saturday night. Sunday is laundry and gym day. You clean the house and meal prep for the week. Game of Thrones starts in an hour. That project from Tuesday night? You'll have Monday night to catch up on it.

For those working full time jobs, the weekend is a sacred bucket where all procrastinations from the week go. We imagine the bucket will be easier to empty on the days we've labeled Saturday and Sunday. It's as if the bucket feels twice as heavy on a Tuesday compared to a Saturday.

The problem with this approach is it becomes an endless cycle. Life and social priorities come up and those uninterrupted chunks of time during the weekend dissipate. Your tasks go back into the procrastination bucket and on and on it goes.

I strive not to separate a weekday from a weekend. They are all just days. Some have more free time than others. I visualize time as blocks on a calendar. What's the difference between Wednesday and Saturday? On Wednesday I'm in the office between 9 AM - 6 PM. On Saturday I have that block of time open.

So technically the only difference is I have fewer open blocks of time on Wednesday. And thus if I schedule personal project time from 8-9 PM on Wednesday, it feels no different than if I scheduled that time from 12 - 1 PM on Saturday.

The other aspect is the perception of time. I used to perceive that a weekend minute was different from a weekday minute. Weekend minutes were more flexible and productive. More appropriate for personal projects. And yet to procrastination, a weekday minute is no different from a weekend minute. It's just a minute.

If you start viewing your time as blocks of time, it wont matter which day of the week you assign them to. Instead of routinely procrastinating projects to weekends, assign them to the earliest block of time you can commit to. You'll then get in a habit of being focused and getting to work during your block of time. The day of the week wont matter. A day is just a day. A minute is just a minute.

And you'll find that once the weekend does come, the only difference is you just have more blocks of time to work with.