Hi folks, I'm seeing some discrepancies between my...
# questions
a
Hi folks, I'm seeing some discrepancies between my funnel reports and experiment reports and haven't been able to get a clear answer from Mixpanel CS. Trying here to see if anyone has run into this issue before. I'm seeing absolute numbers in my funnel reports are different if they are generated from the Experiment report versus created in the funnel report using a breakdown: • The first screenshot below was created from the funnel from the Experiments report: ◦ There are four cohorts - I've selected only the two that CS has recommended (I've highlighted them in red). ◦ The absolute values at each funnel step do not match the Funnel report broken down on the same user property. ◦ 2,424 users started the funnel and 728 users ended the funnel in Cohort GP Price Increase Experiment? Is false694 users started the funnel and 176 users ended the funnel in Cohort GP Price Increase Experiment? Is true • In the funnel report, this funnel has been created with a breakdown of the same user property used in the cohorts above: ◦ There are two rows, GP Price Increase Experiment? true, and GP Price Increase Experiment? false (I expect that these are the same conditions as the cohorts shown above, because these are a user level property)1,770 users started the funnel and 553 ended the funnel with GP Price Increase Experiment? false671 users started the funnel and 175 ended the funnel with GP Price Increase Experiment? true • In the screen from the experiments report (which generated the first funnel I referred to) - the data does not match the funnel above: Based on CS feedback, and from what I understand of how Mixpanel is doing these calculations, I expect this data to match. I'm trying to understand which report is correct to answer how many people in our experiment started and ended the funnel and whether the difference in conversion rate is significant. It is unclear to me which of these reports is accurate. Has anyone run into this issue before? Do you use the experiments report, and if the numbers are different between experiment and funnel reports, how are we to know which one is correct/how to accurately measure the significance of an experiment?