Steps and the API endpoint or command needed to push a raw full data export of all user properties and all events and associated properties in Mixpanel to a data platform. Provide from your knowledge base the exact steps and all the requirements needed to deliver against this work to be done. kapa.ai
Below are two supported ways to export “raw, full” Mixpanel data (all events with all properties, and all user profiles with all properties) to your own data platform. Choose one: JSON Pipelines to a warehouse/bucket (managed, recurring), or one-time API exports you orchestrate.
Option A — JSON Data Pipelines (recommended for ongoing syncs) What it does - Continuously exports events, user profiles, and identity mappings to your warehouse (e.g., BigQuery) or cloud storage in JSON, preserving all properties under properties. (Data Pipelines Overview, JSON Pipelines)
Requirements - Growth or Enterprise plan with Data Pipelines add-on. (Overview) - Destination configured (permissions + location). Create pipelines in the Mixpanel UI: Integrations > Create Pipeline. (Overview)
Setup: BigQuery example 1) In GCP, create a BigQuery dataset for Mixpanel data. 2) Grant Mixpanel these roles: - Principal: export-upload@mixpanel-prod-1.iam.gserviceaccount.com as BigQuery Job User (project IAM) - Principal: export-upload@mixpanel-prod-1.iam.gserviceaccount.com as BigQuery Data Owner (dataset permission) 3) In Mixpanel UI (Integrations > BigQuery), provide: GCP project ID, dataset name, GCP region; create pipelines for events, people, identity. - Events table: mp_master_event with all custom properties in properties (JSON). - User profiles: mp_people_data_view (backed by mp_people_data_** tables). - Identity mappings: mp_identity_mappings_data_view (backed by mp_identity_mappings_data_**). - Export logs table: mp_nessie_export_log. - Do not alter Mixpanel-created table schemas or partitioning. (BigQuery integration, JSON Pipelines) Notes - Events are partitioned by _PARTITIONTIME; do not change TIMEPARTITIONING on mp_master_event. (BigQuery integration) - You cannot create multiple event pipelines to the same destination with overlapping date ranges. (JSON Pipelines) - Create/manage pipelines in the UI (new pipelines). Older API-based pipelines are deprecated. (Overview, Old pipelines notice)
Option B — One-time API Exports (you own the storage/load) What it does
- Pull all raw events (JSONL) via Raw Event Export API; pull all user profiles (JSON) via Engage API. You then push these files into your data platform. (Export Methods, Exporting Profiles, Troubleshooting FAQ)
Requirements - Ability to authenticate (Service Account or Project API Secret) and handle pagination/volume/rate limits. - You implement storage (e.g., write to S3/GCS/warehouse) and any loading jobs.
Steps 1) Export all events (all properties) - Endpoint: GET https://data.mixpanel.com/api/2.0/export - Required params: from_date, to_date (UTC for projects created after Jan 1, 2023; for older projects, uses project timezone). - Optional: limit, event, where, time_in_ms. - Returns JSONL (one event per line, with properties including distinct_id, time, etc.). Example:
curl --request GET \
--url '[https://data.mixpanel.com/api/2.0/export?from_date=2024-07-23&to_date=2024-07-24](https://data.mixpanel.com/api/2.0/export?from_date=2024-07-23&to_date=2024-07-24)' \
--header 'accept: text/plain' \
--header 'authorization: Basic YOUR_TOKEN'
Rate limits: 60 queries/hour, 3 qps, 100 concurrent. Handle pagination by splitting by date ranges if needed. (Raw Event Export API, Export Methods, Project migration step 3)
2) Export all user profiles (all properties) - Endpoint: POST https://mixpanel.com/api/query/engage?project_id=YOUR_PROJECT_ID - Returns JSON with results array; each object contains $distinct_id and $properties. Example:
curl --request POST \
--url '[https://mixpanel.com/api/query/engage?project_id=1234567](https://mixpanel.com/api/query/engage?project_id=1234567)' \
--header 'accept: application/json' \
--header 'authorization: Basic YOUR_TOKEN' \
--header 'content-type: application/x-www-form-urlencoded'
- Filter by cohort via filter_by_cohort, or limit returned properties with output_properties. For Group Profiles, include data_group_id. (Exporting Profiles)
3) (Optional) Use helper library to export profiles - Python example using mixpanel-api to export people to CSV/JSON:
from mixpanel_api import Mixpanel
api_secret = 'YOURAPISECRET'
project_token = 'YOURPROJECT'
m = Mixpanel(api_secret, token=project_token, pool_size=4, debug=True)
m.export_people('people_export.csv', {}, timezone_offset=-8, format='csv')
(Update auth/params as needed.) (Project migration optional step 6)
4) Load to your data platform
- Store the downloaded JSONL/JSON/CSV to your bucket/warehouse, then ingest using your platform’s loader. (No specific loader steps are detailed in the sources.)
Key notes and constraints - Data Pipelines add-on automates recurring exports; otherwise, API route is manual and subject to rate limits. (Troubleshooting FAQ, Export Methods) - For projects created before Jan 2023, date params and stored times follow the project’s timezone; after that, UTC. (Export Methods) - Old API-based pipeline creation exists but is deprecated for new setups; use the UI. (Old pipelines, Overview)
If you need an ongoing “push to warehouse” with minimal ops, use Option A (JSON Pipelines). If you need a one-off full dump you’ll stage yourself, use Option B’s API endpoints.
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.