Question

A/B Testing for Web

  • 16 August 2019
  • 4 replies
  • 326 views

Badge

Hello,

I know that this is not something that MixPanel offers, but given how great the tracking is on MixPanel, I'm looking for a super simple way to implement A/B testing without having to use another vendor (Those A/B testing services are very expensive >$500/month!).

Here's how I'm thinking to implement web-based A/B testing with MP:
Creating a super simple JS process that shows at random the A or B variation for every visitor. Then would track that variation in MP and then can probably cobble together a report that looks at conversion rates for each variation and will be able to compare.

Has anyone had any experience doing something like this? Are there any super simple open-source JS frameworks out there that would help facilitate serving the different variations? Let me know.




4 replies

Userlevel 1
Badge

Hi @sisrani,


On the web, it is easier to rapidly deploy changes in your code because you do not need to submit it to any third party review system for approval. I can see why Mixpanel built special features in our iOS and Android libraries to deploy and analyze A/B tests since dealing with app store will take a while!

I am not aware of a JS framework, but It looks like @shawn.barrett gave an example on this old community post here on how set up and use Mixpanel to help analyze the results of any A/B tests that you perform on your site. Hopefully Shawn's word of wisdom help you set up your test!


A/B testing on web would be great. We are a cordova web app running of localhost so we cannot use Google Optimize and you have built such good tools for native, seems like it would be simple enough to have a ‘start experiement’ flag for the js-sdk that allows us to use the same reporting features.

Should be possible to just send events like  

$experiment_started 

with properties

 $experiment_id 

I just don’t know how to specify the variants? Can someone explain how that works?

Userlevel 1
Badge

I found this JS a/b test library that you could use to solve this problem:

Are there any super simple open-source JS frameworks out there that would help facilitate serving the different variations?

https://github.com/dollarshaveclub/study#usage

I haven’t used it before, or tested it, but it seems promising for what you described.

Reply