I used by mistake original api identity management for my project and now regret it. First of all is it recommended that i move to Simplified API. If yes could you give a step by step approach to backfilling data?
Hi Sushrut C., Please stand by as I search the Mixpanel knowledge base for an AI-powered answer. I normally take about 15 seconds or so. In case AI cannot answer your question, feel free to reach out to Mixpanel support at mixpanel.com/get-support. If you have follow-up questions for this thread, please tag me again with your question: @kapa.ai <follow-up question> 👋
Feel free to ask a follow-up question once I've answered by mentioning me with @kapa.ai
Yes, it is generally recommended to move to Simplified API for identity management in Mixpanel, as it offers a more straightforward approach to managing user identities. However, migrating from Original API to Simplified API requires setting up a new project, as it's not possible to directly convert an existing project with data to Simplified API. Here's a step-by-step approach to backfilling data when migrating to Simplified API: 1. Set up a new Mixpanel project: - Create a new project in your organization - Enable Simplified API in the new project's Identity Merge settings [Migrating to Simplified ID Merge](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#set-up-a-new-mixpanel-project) 2. Determine the volume of historical data to migrate: - Decide on how much historical data you need to backfill - Consider migrating only recent, actively queried data for efficiency [Backfilling Historical Data](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#backfilling-historical-data) 3. Choose a backfilling method: - Use Mixpanel APIs (Raw Export API for exporting, Import API for importing) - Use Mixpanel Warehouse Connector if your data is in a data warehouse - Use your Customer Data Platform (CDP) to replay historical data [Backfilling Historical Data](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#backfilling-historical-data) 4. Transform your data: - Ensure your historical data includes both $device_id and $user_id properties - If these properties are missing, consider creating dummy events with both IDs to enable identity merging [Legacy ID Management](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#legacy-id-management) 5. Import the transformed data into the new project: - Use the chosen backfilling method to import your historical data - Ensure each imported event includes a $insert_id to prevent data duplication [Backfilling Historical Data](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#backfilling-historical-data) 6. Migrate reports and non-data entities: - Manually recreate cohorts, custom events, and custom properties - Use the Move Board feature to transfer boards between projects - Migrate Lexicon schema definitions using Schemas API or CSV Export/Import [Migrating Reports and Non-Data Entities](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#migrating-reports-and-non-data-entities)
7. Update your implementation for live data:
- Update your tech stack with the new project's token and credentials
- Upgrade to the latest SDK versions supporting Simplified ID Merge
- Modify your implementation to include $device_id and $user_id in events
Sending Live Data Part 1Remember to thoroughly test both live and historical backfilled data in a development environment before deploying to production. Also, be prepared for potential data delays during the transition and communicate this to your Mixpanel users.
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
kapa.ai 4. Transform your data: - Ensure your historical data includes both $device_id and $user_id properties - If these properties are missing, consider creating dummy events with both IDs to enable identity merging [Legacy ID Management](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#legacy-id-management) Elaborate what you mean by dummy event
Let me check the knowledge base for you again. One moment! 📚
A dummy event in the context of migrating from Original ID Merge to Simplified ID Merge is a specially crafted event that includes both $device_id and $user_id properties to enable identity merging. Here's an elaboration on what this means: 1. Purpose: The dummy event is used when your historical data doesn't include both $device_id and $user_id properties, which are required for identity merging in Simplified ID Merge.2. Content: According to the documentation, "Instrument a dummy event that includes both $device_id and $user_id based on your ID mappings and send that to the new project to enable identity merging." Backfilling Historical Data3. Naming: You can choose any name for the dummy event, except for "$identify", "$create_alias", and "$merge". These reserved event names are not used in Simplified ID Merge.4. Example: A dummy event might look like this:
"event": "ID_Mapping",
"properties": {
"$device_id": "anonymous111",
"$user_id": "user123",
"token": "YOUR_PROJECT_TOKEN"
}
}
```LBKSLBKS5. Frequency: You only need to send this dummy event once per user to establish the link between their `$device_id` and `$user_id`.LBKS6. Timing: Typically, you would send this dummy event at the point when a user switches from an anonymous to an identified state, such as when they log in.LBKS7. Purpose: By sending this dummy event, you're telling Mixpanel that the anonymous user with `$device_id` "anonymous111" is the same as the identified user with `$user_id` "user123". This allows Mixpanel to merge the user's anonymous and identified event streams.LBKSRemember, this approach is only necessary if your historical data doesn't already include both `$device_id` and `$user_id` properties. If your data already includes these properties, you can import it directly without creating dummy events [Original ID Merge](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#original-id-merge).
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
kapa.ai can you give an example? and also explaiin how this can help mappting specially if the user is inactive
Let me check the knowledge base for you again. One moment! 📚
Certainly! I'll provide an example of a dummy event and explain how it can help with mapping, especially for inactive users. Example of a dummy event:
json
{
"event": "ID_Mapping",
"properties": {
"$device_id": "anonymous_device_123",
"$user_id": "registered_user_456",
"token": "YOUR_PROJECT_TOKEN",
"$insert_id": "unique_event_id_789",
"timestamp": 1634567890
}
}
In this example: - The event name is "ID_Mapping" - $device_id represents the anonymous identifier - $user_id represents the registered user identifier - $insert_id is included to prevent duplication [Backfilling Historical Data](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#backfilling-historical-data) - A timestamp is added to indicate when this mapping occurred Now, let's explain how this can help with mapping, especially for inactive users: 1. Historical data mapping: This dummy event allows you to establish a link between the anonymous $device_id and the known $user_id for historical data. This is particularly useful when your original data doesn't contain both identifiers in the same event [Backfilling Historical Data](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#backfilling-historical-data). 2. Inactive user mapping: For inactive users who haven't logged in recently, you might not have recent events containing both identifiers. The dummy event allows you to create this mapping without requiring the user to perform any new actions. 3. Retroactive merging: Simplified ID Merge supports retroactive merging. This means that once you send this dummy event, Mixpanel will merge all past events associated with either the $device_id or $user_id into a single user profile [Understanding Simplified ID Merge](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#understanding-simplified-id-merge). 4. Unlimited merging: Simplified ID Merge allows you to merge an unlimited number of anonymous IDs ($device_id) to a user ($user_id). This is particularly helpful for inactive users who might have multiple anonymous identifiers from different devices or sessions [Understanding Simplified ID Merge](https://docs.mixpanel.com/docs/tracking-methods/id-management/migrating-to-simplified-id-merge-system#understanding-simplified-id-merge). 5. No time limit: Unlike some systems, there's no time limit on when you can send this mapping event. You can send it for users who have been inactive for long periods, ensuring their historical data is properly attributed. 6. One-time operation: You only need to send this dummy event once per user to establish the link. This makes it efficient for mapping large numbers of inactive users without generating unnecessary event volume.
By using these dummy events, you can ensure that all user data, including that of inactive users, is properly mapped and merged in your new Simplified ID Merge project. This provides a more accurate and complete view of user behavior over time, even for users who haven't been active recently.
kapa.ai is still learning and improving, please let me know how I did by reacting below.
ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.