Case study · HubSpot
Instead of perfecting an AI feature behind closed doors, I shipped it deliberately early to learn from real usage. It became HubSpot's #2 most-adopted AI feature among Enterprise customers.
By the numbers
#2
Most-adopted AI feature among HubSpot Enterprise customers, by unique seats
50+
AI tools it was ranked against
Early
shipped to learn, then iterated
COMPANY
HubSpot
MY ROLE
Product Manager
AREA
AI in reporting
RESULT
#2 of 50+ AI tools
THE CONTEXT
Why was finding a report so hard?
In discovery calls, customers kept telling us they couldn't find the report they needed. The data backed it up: Reports list engagement was lower in larger mid-market and Enterprise portals. The more reports a portal had, the harder the right one was to find, even with sorting and filtering.
What was I asked to solve?
Bring AI into report discovery, so people could find a report without already knowing its name. I chose to ship early and learn from real usage.
01 · HOW IT CAME TOGETHER
We started with one question: what makes search good? Then came a bigger one: could we help you find what you need without searching at all?
Phase 1
WHY CAN'T PEOPLE FIND THEIR REPORTS?
→ General discovery calls kept surfacing the same frustration: finding the right report.
→ Engagement with the Reports list page was lower in larger mid-market and Enterprise portals.
→ The more reports a portal had, the harder it got, even with the sorting and filters we already had.
Phase 2
WHAT MAKES SEARCH GOOD?
→ Regular search: not helpful at all the moment you misspelled something.
→ Fuzzy search: much better, but it only works if you already know what you're looking for.
→ Fuzzy search plus RAG: tied to each report's underlying metadata, not just a title match. We built it with HubSpot's AI team, with an engineer from my team embedded with them.
→ Recommendations: could we get you to the right report without searching at all?
Phase 3
FROM “CORRECT” TO “GOOD”
→ We followed HubSpot's portal-wide search, but it had to make room for AI. After design teardowns and feedback sessions, AI got its own section of the results.
→ We knew we needed search either way, so we A/B tested regular search against AI search. AI search performed better.
→ Testing showed early recommendations were correct for what was typed, but ignored who was searching and what they were trying to do.
→ We tuned them with context and ran evals with the RAG team to keep results consistent.
Phase 4
HOW WE KNEW IT WORKED
→ Searches that ended in opening a report, and fewer error or empty states.
→ AI report interactions, plus funnels from search to adding a report to a dashboard, and from search to activating on a report.
→ Overall search usage. It became HubSpot's #2 most-adopted AI feature among Enterprise customers, out of 50+ AI tools.
→ The recommendations that started here later moved into the Add to Dashboard panel.
02 · THE KEY DECISION
IT WASN'T
“Recommend whatever matches what was typed.”
IT WAS
“Recommend what this person is most likely trying to do.”
Early recommendations were correct but generic. We added context: who's searching, the dashboard they're on, their recent and most-used reports, what they can access, the apps they use most and their recent searches.
03 · THE TRADE-OFF
✓ In v1
Plain-language AI search across reports and templates, so customers could describe what they needed instead of guessing a report name.
→ What came next
Taking the AI recommendations beyond search, starting with the Add to Dashboard panel. Most recently, we wrapped up bringing reports and dashboards into HubSpot's global search, so you can find a report the same way you'd search for a contact.
My role: I worked out the requirements with another PM and partnered with HubSpot's AI team on the RAG system, with an engineer from my team embedded with them. I worked with design on the search landing page and results, and ran evals with the RAG team.
04 · THE OUTCOME
#2
most-adopted AI feature among Enterprise seats
50+
AI tools it was ranked against
A/B
AI search beat regular search head to head
It also validated AI recommendations as a pattern to scale across other reporting surfaces.
05 · WHAT TO TAKE FROM IT
1
An early, incomplete release teaches you more than a month of internal debate.
2
Ranking against 50+ AI tools told us more than raw usage alone.
3
A result can match the words and still miss the person. Ask who is searching and what they're trying to get done.
Sources: My work at HubSpot.
Your turn.
I mentor career switchers from CS, support, marketing and analytics. Start with a free 30-min clarity call. We'll map your story and your next step. No sales pitch.