Question

Export Mixpanel Data into Azure Data Factory

  • 5 September 2020
  • 2 replies
  • 192 views
Y

Hi,

I was trying to export data from Mixpanel into Azure Data Factory following this doc: https://developer.mixpanel.com/reference/data-pipelines-api-overview#create-a-pipeline and I ran into some issues in creating the pipeline. I’ve already configured the permission for Mixpanel in Azure Command Line.

The problems are: 

  1. where do I run the following curl statement;

curl https://data.mixpanel.com/api/2.0/nessie/pipeline/create \ -u API_SECRET: \ -d type="azure-blob" \ -d trial="true" \ -d data_format="parquet" \ -d storage_account="mystorageaccount" \ -d container_name="mixpanel-export" \ -d prefix="custom_prefix/for/data" \ -d schema_type="multischema" \ -d client_id="REDACTED" \ -d client_secret="REDACTED" \ -d tenant_id="REDACTED" #Whitelist a "Page View" and "Item Purchase" event -d events="Page View" \ -d events="Item Purchase

  1. What custom changes do I need to make to the above sample code given in the Mixpanel’s doc?e.g. “nessie” in the url needs to be changed to what name? Project name or the user name or something else?
  2. Below is my code, where I ran in Azure Command Line, changed “nessie” into our project name and added the API_SECRET, but it’s giving me an error saying “Host password required” or “Authentication Required” , what should I change here?

curl https://data.mixpanel.com/api/2.0/EnvisionWell/pipeline/create -u API_SECRET="DELETED" -d type="azure-blob" -d trial="true" -d data_format="parquet" -d storage_account="mixpanel2" -d container_name="test-mixpanel" -d prefix="custom_prefix/for/data" -d schema_type="multischema" -d client_id="DELETED" -d client_secret="DELETED -d tenant_id="DELETED" -d events="Page View" -d events="Item Purchase 

Thanks so much!

2 replies

cherise
Userlevel 6
Badge +3

Hey @yyzvv —

  1. where do I run the following curl statement;

In order to run the network request as a curl, you can do so from the command line in your computers’s terminal.

  1. What custom changes do I need to make to the above sample code given in the Mixpanel’s doc?e.g. “nessie” in the url needs to be changed to what name? Project name or the user name or something else?

You actually do NOT need to change “nessie” in the URL. I can see why you would have done that, but it’s actually just a part of the URL / request. So you only need to replace the API_SECRET with your project's API secret. 

Below is my code, where I ran in Azure Command Line, changed “nessie” into our project name and added the API_SECRET, but it’s giving me an error saying “Host password required” or “Authentication Required” , what should I change here?

If you include “nessie” you should be all set!

(I deleted the IDs in your post to protect your project and Azure instance)

Let me know how it goes!

cherise
Userlevel 6
Badge +3

@yyzvv —

I spoke to our pipelines engineer about your questions, and he mentioned that your event filters are going to be problematic if you don't have those events. So if you continue to see errors, you can delete -d events="Page View" -d events="Item Purchase" from the curl request unless you are actually trying to create filters.

Reply


Get inspired by Mixpanel Moments:

View webinar View webinar
Mixpanel