Hi everyone! I'm stuck with the root interpretation of some numbers, and was hoping we could bee-hive this. I am looking at 3 metrics in the graph attached:
🟠 orange line: # unique users - metric 1
🟣 purple line: # unique users - metric 2
🟢 green line: % Conversion Rate for Funnel metric 1 -> metric 2 (30 day conversion window)
I am struggling with the following: How can the relation between metric 1 and metric 2 remain the same, while I see an overly steep decline in the funnel % conversion rate for the same reference metrics? Did anyone else experience something like that?
Is it possible for someone to do metric 2 but not metric 1 first? Is there another conversion path that is interrupting the conversion from 1 to 2? Also, is there a direct relation between unique users and your conversions? If you do total users do you see a related drop off?
Hey Andrew, thanks for reaching out! To your questions:
It is not possible to do metric 2 before metric 1, also not in any alternative path - so all users that have performed metric 2 must have done metric 1 before
The relation between metric 1 and metric 2 is the same for total events as well, frequency per user is higher for metric 2 than for metric 1
So if the frequency on 2 is more than 1 is it possible they fall outside the 30 day window on the funnel?
Also I am seeing a dip in the orange that coincides with the green drop. Which if I’m right, without a corresponding drop in purple would mean your repeat metrics 2 with a drop in new metric 1. So that could translate into your drop in the green. They are different axes so the magnitude of the green drop doesn’t relate to the magnitude of the orange and purple lines
That dip you see was in February, where we had an issue with metric 2 (basically not tracking it any more for 2 weeks). I am more interested in the drop of the green line (the conversion metric) since beginning of december that I am struggling to find an explanation for
So much appreciate your throughs on this Andrew! Feels like an early Christmas gift, that someones putting their mind to this as well 🎅
I wonder if you did another metric. S formula of 1 over 2. Does that formulaic metric show a declining ratio which would be that you have more repeat metrics 2 for the same unique user (1/1 vs 1/2 vs 1/3…). If so then your funnel metric may be declining bc it’s reflecting that
you can then try changing your second element in the funnel to have "first time" selected and see if you get different results (to try to maybe prove out the repeat metric 2's
I think you lost me here. What would that formula look like? If I do a simple (metric2 / metric1)*100, which eliminates the conversion window, then I do not see the drop
That dip you see was in February, where we had an issue with metric 2 (basically not tracking it any more for 2 weeks)
this is almost certainly your issue here, you have a cohort of users who did metric 1 during those two weeks but weren't "allowed" to convert until you fixed the bug, and after you fixed it only had two weeks left to convert, which is half the time the other users have also note that your conversion rate only drops from 60% to 45%, so it's zoomed in, since the y axis is not indexed at zero
