Hi ,I have pulled data from mixpanel with event ex...
# questions
b
Hi ,I have pulled data from mixpanel with event export API and it is around 8.8Gb. I got
MemoryError
for loading the content in json. Any recommendation on loading the large json? Many thanks! Here is my code:
Copy code
parameters = set()
 
    # Calls to the data export API do not return JSON.  They return
    # records separated by newlines, where each record is valid JSON.
    # The event parameters are in the properties field
    events = []
    for line in requests.text.split('\n'):
        try:
            event = json.loads(line)
            ev = event['properties']
            ev['event'] = event['event']
        except ValueError:  # Not valid JSON
            continue
 
        parameters.update(ev.keys())
        events.append(ev)