{"id":19133,"date":"2024-07-22T16:27:52","date_gmt":"2024-07-22T16:27:52","guid":{"rendered":"https:\/\/productcollective.com\/?p=19133"},"modified":"2024-07-22T16:27:52","modified_gmt":"2024-07-22T16:27:52","slug":"data-informed-decision-making","status":"publish","type":"post","link":"https:\/\/productcollective.com\/data-informed-decision-making\/","title":{"rendered":"Deep-Dive: Data Informed Decision Making"},"content":{"rendered":"

As a product manager, you’re no stranger to tough decisions. Should you prioritize that new feature your biggest customer is clamoring for or focus on improving the onboarding experience to boost activation rates? Is it time to overhaul your pricing strategy or double down on customer retention initiatives?<\/span><\/p>\n

The answer is simple in today’s data-rich world: follow the data. After all, isn’t that what being “data-driven” is all about? But here’s a thought that might give you pause: what if relying too heavily on data could lead you astray?<\/span><\/p>\n

Welcome to the world of data-informed decision\u2013making, a nuanced approach that’s revolutionizing how savvy product managers navigate the complex landscape of product development and strategy.<\/span><\/p>\n

Imagine you’re at the helm of a ship. Data is your compass, providing crucial direction. But you’re not blindly following its every twitch and turn. Instead, you combine its guidance with your knowledge of the seas, the weather, and your crew’s capabilities. That’s the essence of being data-informed.<\/span><\/p>\n

This essay will explore why this approach is gaining traction among successful product teams. We’ll dive into the art and science of collecting quantitative and qualitative data and how to strike the right balance between the two. You’ll discover strategies for turning the firehose of incoming data into actionable insights and learn how to avoid common pitfalls that can lead even the most well-intentioned product managers astray.<\/span><\/p>\n

But here’s where it gets really interesting: we’ll challenge some commonly held beliefs about data in product management. Can having too much data hinder decision-making? How do you reconcile conflicting data points? And in a world obsessed with metrics, is there still room for intuition and experience?<\/span><\/p>\n

Whether leading product at a scrappy startup or a Fortune 500 company, you’ll find practical advice for building a data-informed culture that empowers your team to make better decisions faster.\u00a0<\/span><\/p>\n

Let’s dive in and discover how being data-informed can transform the way you work \u2013 and the products you build.<\/span><\/p>\n

Understanding Data-Informed Decision Making<\/span><\/h3>\n

\"Understanding<\/p>\n

In the world of product management, the mantra “data-driven decision-making” has been repeated so often it’s practically become gospel. However, as Richard White, the founder <\/span>of businesses like Fathom and UserVoice, points out<\/span><\/a>, a subtle yet crucial shift is happening: the move from being data-driven to data-informed.<\/span><\/p>\n

At first glance, this might seem like a trivial distinction. After all, isn’t using data to make decisions the whole point? But dig a little deeper, and you’ll find that being data-<\/span>informed<\/span><\/i> is a more nuanced and ultimately more effective approach for product managers.<\/span><\/p>\n

Let’s start with a definition: <\/span>Data-informed decision-making is the practice of using data as a key input in the decision-making process<\/span><\/a> while also considering other factors such as domain expertise, intuition, and qualitative insights. It’s about striking a balance between the cold, hard numbers and the human elements that make product management as much an art as it is a science.<\/span><\/p>\n

The fundamental principles of data-informed decision-making include:<\/span><\/p>\n

    \n
  1. Using data as a guide, not a dictator<\/span><\/li>\n
  2. Combining quantitative and qualitative insights<\/span><\/li>\n
  3. Considering context and nuance<\/span><\/li>\n
  4. Embracing uncertainty and experimentation<\/span><\/li>\n
  5. Continuously learning and adapting based on outcomes<\/span><\/li>\n<\/ol>\n

    So, how does this differ from a purely data-driven approach? <\/span>Milene Davis, a Product Manager now at You Need a Budget, offers a valuable perspective<\/span><\/a>. She cautions against blindly following data, pointing out that this can lead to overlooking essential nuances or misinterpreting results. Instead, she advocates for challenging the data and using it as a tool in the product manager’s toolkit rather than the sole determinant of decisions.<\/span><\/p>\n

    This approach acknowledges a fundamental truth about data: it’s not infallible. Data can be incomplete, biased, or misinterpreted. By being data-informed rather than data-driven, product managers can avoid falling into the trap of false certainty that often comes with an overreliance on metrics.<\/span><\/p>\n

    The benefits of adopting a data-informed approach are numerous:<\/span><\/p>\n

      \n
    1. More holistic decision making:<\/strong> By considering quantitative data and qualitative insights, product managers can make more well-rounded decisions considering the full product and market complexity.<\/span><\/li>\n
    2. Increased agility:<\/strong> Data-informed decision-making allows faster pivots when necessary, as it doesn’t require waiting for statistically significant results for every decision.<\/span><\/li>\n
    3. Better risk management:<\/strong> By combining data with experience and intuition, product managers can often spot potential issues that pure data analysis might miss.<\/span><\/li>\n
    4. Improved stakeholder communication:<\/strong> A data-informed approach provides a framework for explaining decisions to stakeholders, balancing hard numbers with strategic reasoning.<\/span><\/li>\n
    5. Enhanced creativity:<\/strong> Product teams have more room for innovative thinking and bold experiments by not being constrained solely by what the data dictates.<\/span><\/li>\n<\/ol>\n

      Ruben Ugarte, founder of Practico Analytics, emphasizes the importance of this approach, particularly for smaller companies or those in the early stages. <\/span>He notes that <\/span><\/a>relying too heavily on data can be misleading when it is limited. Instead, he suggests using data to inform hypotheses and guide experimentation rather than as the final arbiter of decisions.<\/span><\/p>\n

      It’s also worth noting that the data-informed approach aligns well with the concept of “product sense” \u2013 that almost intuitive understanding of what will work for users that many successful product managers possess. By combining this product sense with data insights, managers can make analytically sound and intuitively right decisions.<\/span><\/p>\n

      Being data-informed is about elevating the role of data by integrating it more thoughtfully into the complex, often messy reality of product management. It’s about using data not as a crutch but as a powerful tool in the product manager’s arsenal \u2013 one that, when wielded skillfully, can lead to better products, happier users, and more successful businesses.<\/span><\/p>\n

      Types of Data for Product Managers<\/span><\/h3>\n

      \"Types<\/p>\n

      Not all data is created equal, and knowing which types to focus on can make or break your decision-making process. Let’s dive into the two main categories of data you’ll be working with: quantitative and qualitative.<\/span><\/p>\n

      Quantitative data is all about the numbers. It’s measurable, countable, and often comes from statistics and metrics. Here are some key types of quantitative data you’ll encounter:<\/span><\/p>\n

        \n
      1. Behavioral Analytics:<\/strong> This is the bread and butter of product metrics. It tells you what users are doing with your product. How many people signed up today? Which features are they using most? Where are they dropping off in the onboarding flow? Tools like <\/span>Mixpanel<\/span><\/a> or <\/span>Amplitude<\/span><\/a> are two platforms that can give you this granular insight into user behavior.<\/span><\/li>\n
      2. Usage Metrics:<\/strong> These metrics give you a broader view of how your product is being used. Think daily active users (DAU), monthly active users (MAU), session length, or time spent in-app. These numbers can give you a sense of overall engagement and product health.<\/span><\/li>\n
      3. Performance Indicators:<\/strong> These metrics directly tie to your business goals. Conversion rates, customer lifetime value (LTV), churn rate, and revenue per user are all examples of performance indicators that can help you gauge the success of your product.<\/span><\/li>\n<\/ol>\n

        But here’s the thing about quantitative data: while it’s great at telling you what’s happening, it often needs to be revised to explain why. That’s where qualitative data comes in.<\/span><\/p>\n

        Qualitative data is all about the why. The rich, descriptive information gives context to your quantitative data. Here are some key types of qualitative data:<\/span><\/p>\n

          \n
        1. Customer Feedback:<\/strong> This includes everything from app store reviews to survey responses. It’s the unfiltered voice of your customers telling you what they love, hate, or wish your product could do.<\/span><\/li>\n
        2. User Interviews:<\/strong> These are in-depth conversations with users that can uncover insights you’d never get from quantitative data alone. They can help you understand user motivations, pain points, and desires in a way that numbers simply can’t.<\/span><\/li>\n
        3. Support Tickets:<\/strong> Users’ issues with your product can be a goldmine of qualitative data. They can highlight usability issues, feature gaps, or confusion that might not appear in your usage metrics.<\/span><\/li>\n<\/ol>\n

          Richard White emphasizes the importance of qualitative data<\/span><\/a>, especially in B2B environments where usage doesn’t always equal satisfaction. He points out that while quantitative data can tell you what’s happening, qualitative data helps you understand why and what to do about it.<\/span><\/p>\n

          The real magic happens when you combine quantitative and qualitative data. Let’s look at an example:<\/span><\/p>\n

          Imagine you’re seeing a spike in user churn (quantitative data). Your first instinct might be to panic, but hold on. You dig into your customer feedback (qualitative data) and discover that users leave because they find your new feature confusing. Now you have the what (increased churn) and the why (confusing new feature), giving you a clear direction for your next move.<\/span><\/p>\n

          Chris Abad, Director of UX at Google, advocates for this balanced approach in this keynote talk he gave at INDUSTRY: The Product Conference<\/span><\/a>. He suggests using quantitative data to identify trends and patterns and then diving into qualitative data to understand the reasons behind those trends. This combination allows you to make decisions that are both data-informed and user-centric.<\/span><\/p>\n

          But how do you decide which data to prioritize? <\/span>Ruben Ugarte suggests thinking about your current goals<\/span><\/a>. If you’re focused on winning new business, prioritize data from prospective customers. If retention is your main concern, focus on feedback from existing users.<\/span><\/p>\n

          The key is to avoid getting stuck in analysis paralysis. It’s about finding that sweet spot where you have enough data to make an informed decision but not so much that you’re overwhelmed. Remember, the goal isn’t to collect every possible data point. It’s gathering the correct data to help you make better decisions for your product and users.<\/span><\/p>\n

          Strategies for Collecting Data<\/span><\/h3>\n

          \"Strategies<\/p>\n

          Alright, you’re sold on the importance of both quantitative and qualitative data. But how do you get your hands on this goldmine of information? Let’s dive into some effective strategies for data collection that will turn you into a veritable product detective.<\/span><\/p>\n

          Direct Customer Feedback: Straight from the Horse’s Mouth<\/strong><\/p>\n

            \n
          1. In-app feedback tools:<\/strong> These are your secret weapons for capturing user sentiment in real time. Tools like <\/span>UserVoice<\/span><\/a> or <\/span>Canny<\/span><\/a> allow users to submit ideas, report issues, or give feedback without leaving your product. <\/span><\/li>\n
          2. Surveys and questionnaires:<\/strong> Don’t write these off as old school. When used strategically, surveys can provide valuable insights. The key is to keep them short, focused, and timely. For instance, try sending a quick survey right after a user completes an essential action in your product. Their experience is fresh, and they’re more likely to give meaningful feedback.<\/span><\/li>\n
          3. User testing sessions:<\/strong> Nothing beats watching real users interact with your product. Tools like <\/span>UserTesting<\/span><\/a> or <\/span>Maze<\/span><\/a> allow you to set up user scenarios and tasks and observe how they navigate your product. It’s like being a fly on the wall in your users’ homes (but, you know, less creepy).<\/span><\/li>\n<\/ol>\n

            Internal Feedback Channels: Tapping into Your Team’s Collective Wisdom<\/strong><\/p>\n

            Your colleagues are a treasure trove of user insights. Here’s how to tap into that knowledge:<\/span><\/p>\n

              \n
            1. Sales team insights:<\/strong> Your sales team is on the front lines, hearing directly from potential customers about their needs and pain points. Set up a regular cadence for collecting their insights. But don’t just ask for their top feature requests. Instead, have them log the feedback they’re hearing, word for word. This helps you avoid the “telephone game” effect, where information gets distorted as it’s passed along.<\/span><\/li>\n
            2. Customer support data:<\/strong> Support tickets are like the canary in the coal mine for your product. They can alert you to usability issues or feature gaps before they become significant problems. Work with your support team to categorize and quantify the issues they see. But don’t stop at the categories \u2013 dig into the actual conversations to understand the context behind the numbers.<\/span><\/li>\n
            3. Success team observations:<\/strong> Your customer success team has a unique perspective on how customers use your product in the wild. Regular check-ins with this team can reveal patterns in how successful customers are using your product and common stumbling blocks.<\/span><\/li>\n<\/ol>\n