Mixpanel Community Icon

The Importance of Confidence Intervals in Data Reporting

·
·

every once in a while I run into situations where it would be terribly convenient to display confidence intervals or error bars on graphs; for example, every time I'm looking at a conversion rate or retention rate, it's very hard to judge what the meaningful part of the graph is, given that sample sizes can vary significantly between points; same with any breakdowns where the underlying data for each group is of very different sizes; this is particularly important when I have stakeholders look at the reports who do not understand sampling error, at which point I have to explain why a corner case in the data has the higher conversion rate and why it's not meaningful to the overall discussion, essentially having me teach stats rather than find and explain patterns in the data currently, I have to download the data and create my own reports to get those confidence intervals, which is kind of a painful process and not something that can scale, as I don't expect most of the people in my company to have the skills and know how to do that