Selective Export of Mixpanel User Properties to Data Platforms
Is it possible to specify the Mixpanel user properties to export to the data platform or lake and not a complete raw extract?
Is it possible to specify the Mixpanel user properties to export to the data platform or lake and not a complete raw extract?
Hi event, Please stand by as I search the Mixpanel knowledge base for an AI-powered answer. I normally take about 15 seconds or so. In case AI cannot answer your question, feel free to reach out to Mixpanel support at mixpanel.com/get-support. If you have follow-up questions for this thread, please tag me again with your question: @kapa.ai <follow-up question> 👋
Feel free to ask a follow-up question once I've answered by mentioning me with @kapa.ai
Yes, it is possible to specify which Mixpanel user properties to export in certain scenarios, but the level of control depends on the export method and integration: - Cohort Sync Integrations: For some integrations (e.g., CleverTap), you can select which user properties to export by clicking on "Edit Columns" and choosing the properties you wish to include in the export. The default user properties exported are documented, and you can customize this selection for the cohort export to the partner platform see here.- Lexicon Export: When exporting Lexicon data (your data dictionary), you can choose to export events & properties and/or user profile properties. The export will include only the selected data see here.- API Exports: When exporting user profiles via the Engage API, you can use the output_properties parameter to specify which profile properties to return, rather than exporting all properties. For example:
```LBKS This allows you to limit the export to only the properties you need [see here](https://docs.mixpanel.com/docs/export-methods#exporting-profiles).LBKS- **Data Pipelines (Warehouse/Lake Exports)**: For Data Pipelines exporting to data warehouses or lakes, the documentation does not explicitly state that you can select a subset of user properties for export. Typically, these exports are designed to provide a complete, schematized extract of your data for downstream analysis, and all properties are included by default [see here](https://docs.mixpanel.com/docs/data-pipelines/json-pipelines).LBKSIn summary:
- For API and some cohort sync integrations, you can specify which user properties to export.
- For Data Pipelines to data warehouses or lakes, the knowledge sources do not explicitly mention the ability to select only specific user properties; these exports are generally full extracts.LBKSIf you need fine-grained control over exported properties for a data warehouse or lake, you may need to filter or transform the data after export.
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
Let me check the knowledge base for you again. One moment! 📚
Certainly! Here’s how you can use the Engage API to export user profile data (both raw and filtered), and how to export raw event data from Mixpanel, based strictly on the provided knowledge sources.---
1. Export All User Profiles (Raw Data Export) via Engage API
To export all user profiles, make a POST request to the Engage API endpoint. This will return all user profiles in your project.**Sample cURL request:**LBKScurl --request POST \ --url '[https://mixpanel.com/api/query/engage?project_id=1234567](https://mixpanel.com/api/query/engage?project_id=1234567)' \ --header 'accept: application/json' \ --header 'authorization: Basic YOUR_TOKEN' \ --header 'content-type: application/x-www-form-urlencoded'LBKSThe response will contain all user profiles in the results array, with all properties for each profile Exporting Profiles.---
2. Export Selected/Filtered User Data via Engage API
You can filter which user profiles to export and/or specify which properties to return using the where and output_properties parameters.**Sample cURL request (filtering and selecting properties):**LBKScurl --request POST \ --url '[https://mixpanel.com/api/query/engage?project_id=1234567](https://mixpanel.com/api/query/engage?project_id=1234567)' \ --header 'accept: application/json' \ --header 'authorization: Basic YOUR_TOKEN' \ --header 'content-type: application/x-www-form-urlencoded' \ --data-urlencode 'where=properties["$email"] == "example@[mixpanel.com](http://mixpanel.com)"' \ --data-urlencode 'output_properties=["$email","$last_name"]'LBKS- where filters the profiles (e.g., only users with a specific email).
- output_properties limits the returned properties to only those specified Exporting Profiles.---
3. Export Raw Event Data from Mixpanel
To export raw event data, use the Raw Event Export API. This returns events as JSONL (one JSON object per line).Sample cURL request:References integrated above as per your instructions. If you need more advanced filtering or paging, see the Engage API documentation for additional parameters.
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
Let me check the knowledge base for you again. One moment! 📚
To confirm the table and column names for user properties (like User ID, Location, etc.), as well as event names, event properties, and event descriptions in Mixpanel, you should use the Lexicon feature in the Mixpanel UI.
How to Confirm Names and Descriptions
1. User Properties (User Profiles)
- Table Name: In Mixpanel’s data model, user profiles are stored in a table where each row represents a user and columns represent user properties (e.g., Name, Email, Department). The unique identifier for a user is the Distinct ID (column name: distinct_id) User Profile: Demographic attributes for your users.
- Column Names: You can view all user profile properties (including User ID, Location, etc.) in the Profile Properties section of Lexicon. The "Name" field is the database (column) name, and the "Display Name" is what appears in the UI. You’ll also see descriptions and example values for each property Lexicon: Profile Properties.
2. Event Names, Event Properties, and Descriptions
- Event Name: In Lexicon’s "Events" view, the "Event Name" is the database name of the event. The "Display Name" is what appears in the UI. The "Description" field explains what triggers the event or what properties are sent with it Lexicon: Overview.
- Event Properties: In Lexicon’s "Event Properties" view, you can see the database name (column name), display name, description, and example value for each property. This includes properties like distinct_id (User ID), location properties (e.g., $city, $region), and any custom properties you track.
3. How to Access Lexicon
- Open Mixpanel. - Click the Data Management icon in the top navigation. - Select Lexicon. - Browse the "Events", "Event Properties", and "Profile Properties" tabs to see all names and descriptions.
4. Default and Reserved Properties
- Some properties, like User ID (distinct_id), Device ID ($device_id), and location properties ($city, $region, mp_country_code), are default or reserved properties in Mixpanel. Their names and usage are documented in the Reserved Event Properties and Default Properties sections.---Summary:
Use the Lexicon in Mixpanel to confirm the exact table/column names and descriptions for user and event properties. The "Name" field in Lexicon corresponds to the database (column) name, and the "Description" field provides details about each property or event. This is the authoritative source for your project’s schema and metadata Lexicon: Overview.
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
Export Raw Event Data from Mixpanel What if I don't want a time filter
est GET \
--url 'https://data.mixpanel.com/api/2.0/export?from_date=2024-07-23&to_date=2024-07-24&limit=100&event=sample_event'
and want to extract all event datas in Mixpanel and from the since date and continue exporting during schedule used kapa.ai
Let me check the knowledge base for you again. One moment! 📚
The Mixpanel Raw Event Export API requires both from_date and to_date parameters for every export request. There is no supported way to omit these parameters and export all event data without a time filter. You must specify a date range, and the API will return events within that range only. The parameters are required and must be in the format YYYY-MM-DD Export API Reference.If you want to export all historical data, you need to:
- Set from_date to the earliest date you want to export from (up to the start of your project).
- Set to_date to the latest date you want to export (e.g., today’s date).
- Repeat the export process on a schedule, updating the date range each time to fetch new data.Automated Scheduling:
To continuously export new data, you should schedule regular exports (e.g., daily or hourly) and update the from_date and to_date accordingly for each run. There is no built-in "continuous" or "since last export" mode in the API; you must manage the date ranges yourself Export Methods.Note:
- The API has rate limits: 60 queries per hour, 3 queries per second, and a maximum of 100 concurrent queries.
- Each request must include both from_date and to_date Export API Reference.Summary:
You cannot omit the time filter. To export all data, start with the earliest date and incrementally export new data by updating the date range in your scheduled jobs.
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
Seeking an update on the attribute mapping list I shared by email for MixPanel. At this stage, it is the field mapping source/target mapping confirmation that I need/require. Does this question make sense, this is regarding providing the event timestamp, event name, event description, user properties - such as location, email, User ID etc kapa.ai
Let me check the knowledge base for you again. One moment! 📚
Yes, your question makes sense in the context of Mixpanel data exports and integrations. You are seeking confirmation on the mapping between Mixpanel’s internal field names (source) and your target system’s field names for attributes such as event timestamp, event name, event description, and user properties like location, email, and User ID.Here’s how Mixpanel defines and names these fields:Field mapping confirmation: - The source field names in Mixpanel are as listed above. - You should map these to your target system’s field names as required. - For a full list of available properties and their exact names, use the Lexicon in Mixpanel, which provides the authoritative source for all event and user property names and their descriptions Lexicon: Overview.If you need confirmation for additional or custom properties, please refer to your Mixpanel Lexicon or the exported schema from your data pipeline, as these will reflect your project’s specific configuration.
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.