Server Ingestion Time in API?

  • 3 March 2020
  • 3 replies
  • 190 views

Mixpanel's (offline) events can be delayed in the raw export for up to 5 days as described here: https://help.mixpanel.com/hc/en-us/articles/115004616486-Tracking-If-Users-Are-Offline

The same is also mentioned in the export API docs https://developer.mixpanel.com/docs/exporting-raw-data. It would be great if in the event export API, Mixpanel could give us the ability to query based on server ingestion time of the event instead of event creation time on the client side. Then we wouldn't need to have a lagging cursor in place and could reduce our sync times. Right now this option does not seem to be present, we can only query for events with the time event was created at and filter on some event parameters (which does not include server ingestion time). Does anyone know if Mixpanel plans to build support for such querying based on server ingestion time in their API? Or do they have any alternatives to address this issue?


3 replies

Userlevel 5
Badge +4

@fivetrandev We filed a product gap on your behalf with our Engineering and Product Development team for review.

 

Please make sure you are subscribed to this Idea - via the Subscribe star button at the bottom of the post - to get emailed updates on its progress.

Userlevel 1
Badge

@fivetrandev I was browsing the Community and thought I’d like to share some good news about this request - this is actually something you can do today!

In order to filter on the ingestion time, you want to use the property `mp_processing_time_ms`  within the `where` clause of an export request. An example filtering might look like: 

where=(properties["mp_processing_time_ms"] >=1579141800000 and properties["mp_processing_time_ms"]<=1579148999999)

Do note that this property is in GMT and you would still need the appropriate to and from dates on the request to make an export. Using this property you can, for example, export all data ingested today that was actually sent yesterday. This means you can utilize the to date, from date, and ingestion time to export data and selectively filter to avoid any duplication in automated processes.

@ryan awesome, thanks so much!

Reply