I noticed a weird thing earlier when looking at first-time retention report.
- When breaking down by time-based cohort (based on date of first action, this is the default view), the retention curves flatten before dropping directly to 0 in month-N (side note: it should discontinue instead). This is normal because the first event simply haven’t passed for N months yet – hence the data don’t exist.
- However, when breaking by a property, the retention curves will gradually decrease to 0, and all segments will reach 0 around the same time.
After some digging, I suspected this is the formula system is using:
(# of users with X property who did action B in month-N) / (# of user with X property who did action A at any time in reporting period)
This present a big problem because some users didn’t perform action-A that long ago, but they’re part of the denominator. In my opinion, a user should only be included in the denominator for month-N retention when he performed action at least N months ago.
Is my interpretation correct?