• Stef the PM
  • Posts
  • We Tell Our Customers Data Is Everything. Here’s Why We Need to Take Our Own Advice.

We Tell Our Customers Data Is Everything. Here’s Why We Need to Take Our Own Advice.

Why clean data matters even more for PMs in the AI era

Author

Stefanie Brown
October 03, 2025

👋 Hey friends,

Last week I met up with some friends, and Ritika, another PM who works at a financial institution, brought up something that’s been bugging both of us.

We were talking about building AI features for our users, specifically around user profiling and recommendations. She’s working in securities, I’m on customer-facing experience, but we kept running into the same problem: how do you build good user profiles when your data is messy?

The conversation got really interesting when we realized the irony. At HubSpot, we constantly tell our customers how important clean data is for their CRM. “Clean data drives better insights, better segmentation, better customer experience.”

But here we were, two PMs at data-focused companies, struggling with exactly the same data quality issues we help our customers solve.

The Universal Data Problem

What struck me about our conversation is that regardless of what area of product you’re in, data cleanliness becomes your foundation. Whether you’re building fraud detection or customer recommendations, it all comes down to the same thing: garbage in, garbage out.

At a small company, data cleanliness is almost easier. You probably sit next to the person responsible for the database, which is also probably your single source of truth.

The downsides? You’re probably not tracking everything you’d like, or the numbers are so small they’re hard to trust when making decisions.

At a big company, there are entire teams dedicated to data problems, and you probably have access to data specialists.

But your tracking might be split between platforms, there’s too much to track, or you’re looking at the wrong event because every team sets up their own measurement differently.

Joke for the marketers in the room: ask three different teams how they define a “session” and you’ll get three different answers. Same metric name, completely different measurements.

💡 Quick tip: When in doubt, ask “What exactly triggers this event?” You’d be surprised how often teams are measuring slightly different things with the same metric name.

The Stakes Are Higher Now

This problem isn’t just about dashboards anymore. When Ritika and I were discussing AI features, we realized that bad data doesn’t just give you wrong numbers. It teaches your AI the wrong patterns.

I’m currently working on a feature that makes intelligent recommendations based on user profiles and interactions. If our user engagement data is messy, our AI might think a power user is actually a casual user because we’re tracking the wrong engagement events.

In Ritika’s world in securities, one mislabeled data point could mean missing actual fraud or flagging legitimate users.

The same data discipline we preach to customers becomes even more critical when you’re building features that learn from that data. The AI doesn’t know the data is wrong, it just learns from whatever you feed it.

📌 Try this: Next time you look at a metric, ask yourself: “If I used this to train an AI system, what would it learn?” Sometimes that perspective reveals data quality issues you wouldn’t notice otherwise.

What Good Data Discipline Actually Looks Like

Here’s what I’ve learned from working at a company where data is our product:

1. Question the source before you question the story We recently launched a new grid view option on our reports list page. Our initial data showed a 53% retention rate, which would have been incredible for a new feature.

I was about to present this as a huge win until I decided to double-check what exactly we were measuring. Turns out the event we were using was completely wrong. We weren’t actually tracking retention at all.

Use the wrong event in your dashboard, and you end up telling a completely different story about user behavior. If I hadn’t questioned the source, I would have been celebrating fake success.

2. Start with the business question, not the available data It’s tempting to build analyses around the data you have easy access to. But the right question is: “What do I need to know to make this decision?” Then work backward to figure out how to measure it.

3. Test your assumptions with small samples Before you build a whole feature based on data insights, spot-check your findings with actual user conversations. Data tells you what’s happening, customers tell you why.

💡 Quick tip: When you’re unsure about data quality, pick a few specific users and manually verify their journey. It’s tedious but reveals gaps you’d never catch in aggregate.

The PM Data Skills That Actually Matter

You don’t need to become a data scientist to be a good PM. What you need is data literacy: the ability to ask the right questions about data quality, understand what you’re actually measuring, and know when to dig deeper.

Questions I ask about any metric:

  • What specific user action triggers this event?

  • Are there edge cases where this might not capture what we think it does?

  • How does this compare to what users are telling us directly?

  • If this number changed dramatically tomorrow, what would we actually do about it?

That last question is key. If a metric wouldn’t change your decisions, you probably don’t need to spend time perfecting it.

📌 Try this: Pick one metric you look at regularly. Write down exactly what user behavior you think it represents. Then verify that assumption with someone who set up the tracking.

Why This Matters for Aspiring PMs

If you’re coming from customer success, support, or marketing, you already have something many PMs struggle with: direct exposure to real user behavior.

Use that as your data quality check. When the numbers say one thing but customers are telling you something different, trust your customer intuition and dig into the data.

You don’t need to be a SQL expert to be a good PM. But you do need to be skeptical enough to question whether you’re measuring the right thing.

The future of AI customer service is at Pioneer

There’s only one place where CS leaders at the cutting edge will gather to explore the incredible opportunities presented by AI Agents: Pioneer.

Pioneer is a summit for AI customer service leaders to come together and discuss the trends and trajectory of AI and customer service. You’ll hear from innovators at Anthropic, Toast, Rocket Money, Boston Consulting Group, and more—plus a special guest keynote delivered by Gary Vaynerchuk.

You’ll also get the chance to meet the team behind Fin, the #1 AI Agent for customer service. The whole team will be on site, from Intercom’s PhD AI engineers, to product executives and leaders, and the solutions engineers deploying Fin in the market.

Quick Reads for Data-Curious PMs

This Week’s Challenge

Pick one dashboard or metric you rely on for product decisions. Spend 30 minutes investigating:

  1. What exactly triggers this measurement? Ask the person who set it up.

  2. What edge cases might skew the results? Think about user behaviors that might game the metric accidentally.

  3. How does this compare to qualitative feedback? Do customers describe their experience the way your data suggests?

You might discover your metric is rock solid. Or you might find a gap that’s been quietly affecting your product decisions for months.

Either way, you’ll have more confidence in the story your data is telling.

The best PMs aren’t the ones with the most sophisticated analytics. They’re the ones who know when to trust their data and when to dig deeper.

Working with messy data in your current role? I’d love to hear about your approach to data quality. Hit reply and share your biggest data challenge, it might spark next week’s newsletter.

See you next Friday,

– Stef

Questions about data, AI features, or PM work in general? I’m on ADPList doing free mentoring sessions. Would love to chat. [Book here]

📬 Other Newsletters You Might Like

If you like Stef the PM, here are a few other reads worth checking out: