Ask a PM: AMA with Vlado Hruda, Sr. Product Manager at Mixpanel

Question

Answer

How do you define design decisions without having analytics at the product designing stage, as product designing is very subjective and depends on your user base? Is there any framework that can be used to base design decision?

• I've actually found that in these situations, a super valuable analysis is Funnels. Your goal is to track adoption of each feature as a “new product” from 1. discoverability, 2. set-up, 3. aha moment, to 4. habit moment. You can benchmark desired adoption levels vs. existing features (e.g. Query Builder example), and from here, you want to understand where is the biggest gap in existing solutions or what users perceive as a missing piece. 

• This will provide you with a stack-ranking of goals & problems you want to address that you can guide your designers with so they know if they need to double down on discoverability, or ease of use, or a particular flow in that feature. Each of these might have a very different outcome and will lead you to a different solution.

How can we find out the time spent in a particular section of our product?

• That's relatively easy. You can create a 2-step funnel to understand how long users take to complete a set of events. If you track navigation, you can see this information by session/time. If you’re already a Mixpanel user, you can check out how this works in our demo datasets.

How do you make sure to always have time for user tests?

• Actually, it's even more important for change management than in new features, because you can get a great sense of how much friction you are removing, and how much you are introducing for the existing users. If this is done right, it should not take more than 5 hours of work and can potentially save you 2-4 weeks of redoing work, so it's always a great time investment.

As a Product Manager, it gets easier with each day to let go of my own ideas I once thought were awesome that may not work with the direction the product is headed anymore. How do you get the idea of "killing your darlings" across to users or, more importantly, stakeholders after presenting findings that their old ideas just don't quite jive with the direction of the product?

• The biggest learning as a PM for me was that I need to be solution-agnostic and problem-specific. Unless you have experience across Eng, Design & Domain expertise, you are statistically unlikely to discover the best solution all by yourself. As a PM you can provide the role of a guide to help your team to navigate towards the best solution. One of the best approaches is to be product-driven, and quickly try to validate multiple tactics. We usually use an ICE prioritization to understand which are the low hanging fruits we can try, and then pursue them accordingly.

• For stakeholders, it’s important to have a social contract where they agree they care about the results or how you do it. And you should definitely push back on it. If there is too much disagreement, figure out how you can test each of these ideas, and focus on spending time validating them, instead in discussion. You need new information to help with the indecision.

• For large projects, if you get a sense you cannot be objective in your judgment, you might want to ask another PM to step in if necessary.

Which feature(s) in Mixpanel is best for gaining UX insights?

• It’s actually a combined use of Funnels, Flows and Retention reports for discovering activation moment and habit moment. Once you understand the path in Flows where users are trying to go and why they get stuck, you can figure out which is the biggest roadblock preventing them from being successful. You can then optimize this path via Funnels. For instance, once we optimized AI feature adoption by 500% by just focusing on the discoverability of the CTA and optimizing the content for consumption on the right device.

• Also, if you are focused on optimizing just adoption, insights can be a great resource to benchmark your adoption with other similar UI controls within the product to better understand what’s the size of the discoverability gap.

What are your favorite tricks for creating user stories within Mixpanel that most users don't know about?

• The best solution is actually getting real customer quotes. There is nothing more powerful than if you can bring the real customer story to your team and have everyone think of the challenges that a user is facing in that particular situation. We've found making zoom recordings of customer calls very powerful, as we can always go back to the video to gather more detail on a particular topic. Also, we have a great process to collect customer gaps that helps us collect this information at scale.