Making the wrong hire isn’t just a hit to the budget and the calendar—it can shake the very foundation of your product, costing you users, customers, and that all-important product-market fit. But let’s be real, interviewing product managers is a whole different ball game compared to, say, hiring a good Customer Success Manager or a stellar engineer.
Why’s it so tough? Well, for starters, product managers are masters of storytelling. They can spin a yarn that’ll have you nodding along in agreement. But separating the talkers from the doers, especially when it comes to nailing down solid requirements, is a whole other story.
So, here’s a game-changer: a simple test that can weed out the real deal from the smooth talkers.
The need for a reliable testing method
I stumbled upon this gem back in my days at an early-stage startup. We were on a hiring spree for product managers, sifting through five to ten candidates daily. Our interviews covered everything from culture fit to creativity, but we were missing a crucial piece: a way to put their hard skills to the test.
Think about it—would you hire an engineer based solely on their description of coding prowess? Heck no! You’d throw them a practical test to see if they could walk the walk. And that’s exactly what we needed for product management.
Despite the inherently subjective nature of product management, certain core skills—like creating user personas, delineating use cases, and documenting requirements—can be objectively assessed.
Enter the structured assessment framework. We crafted a concise scenario, handed it to candidates before their interview, and invited them to tackle it on a whiteboard during the session. Our focus wasn’t on the perfect solution, but rather on their approach. Could they distill complex requirements? Stay true to the scenario? Navigate the problem-solving journey with finesse?
A self-contained capsule scenario
Picture this: a bite-sized problem that anyone can understand, with multiple solutions that don’t require a PhD in our product space. We’d give this scenario out to candidates a week before their interview, no pre-work required. Then, during the interview, we’d slap it on a whiteboard and say, “Let’s tackle this together.”
Take a look! (Google Docs link)
Given the concise scenario, a candidate should be able to tackle it live without any trouble. Importantly, our focus wasn’t on the perfect solution, but rather on their approach. Could they distill complex requirements? Stay true to the scenario? Navigate the problem-solving journey with finesse? Or did they freeze like a deer in headlights? The key was to see if they could keep it simple, stick to the scenario, and get the job done efficiently.
Evaluating candidate results
This approach gave us invaluable insights. Candidates who breezed through the exercise demonstrated a deep understanding and practical application of product management principles. It wasn’t just about what they delivered, but how they got there.
As we put the test into practice, we noticed a few common candidate stumbling blocks. Surprisingly, quite a few were a bit green when it came to developing requirements—a more common issue than you might imagine! Others dove straight into the weeds, obsessing over intricate details like security and permissions from the get-go. Those candidates routinely ran out of time without making much headway, which was a red flag for us.
A savvy product manager could breeze through the task in under 10 minutes, including all the brainstorming and explaining. If it took them between 30 to 60 minutes, that was still within the ballpark, but it often signaled a hiccup along the way—either forgetting their own process or getting bogged down in a tangent. And those who hit a wall on time? Well, they typically either lacked familiarity with crafting solid requirements or veered way off course early on and never recovered.
The impact of a structured assessment framework for product management
This test has become my touchstone for identifying strong product managers. It gives me a clear-cut way to separate the wheat from the chaff. By providing a reliable metric for assessing candidates’ suitability for the role, it removes one of the big risk factors in product hires, and lets me spend time on the strongest candidates. And its simplicity made it easy for interviewers across the team to adopt, fostering consistency and objectivity in candidate evaluation.
This test dramatically sped up our hiring process, and it enabled us to make same-day decisions on promising candidates. So, if you’re grappling with hiring top-tier product managers, give this approach a shot. Trust me, you won’t be disappointed.
Reality-Based Product Management 