Careful What You Eat, Monkey! |
One of Monkey's co-workers (Troy) was able to provide investigators with a location of interest by looking at the device owner's Google Takeout "Location History.json".
Specifically, Troy looked at the DetectedActivity classification strings which Google (Play Services) uses to estimate whether the device was still, in a vehicle, on foot etc. There was a transition from IN_VEHICLE to STILL which indicated the vehicle had stopped at a location (and not just passed through).
Being part of Google Play services means this DetectedActivity data is likely only available for Google Accounts involving Android devices. We have yet to see any DetectedActivity in iOS device related Google Takeouts.
Special Thanks to:
- Troy for sharing his findings
- Rasmus Riis Kristensen, Mike Lacombe, Heather Mahalik and Lee Crognale for checking their Google Takeout data and script testing.
- Josh Hickman for providing more test Android Takeout data and additional feedback regarding the new Takeout format & velocity / heading / altitude fields.
The main DetectedActivity categories are defined as:
IN_VEHICLE The device is in a vehicle, such as a car.
ON_BICYCLE The device is on a bicycle.
ON_FOOT The device is on a user who is walking or running.
RUNNING The device is on a user who is running.
STILL The device is still (not moving).
TILTING The device angle relative to gravity changed significantly.
UNKNOWN Unable to detect the current activity.
WALKING The device is on a user who is walking.
For further details see here.
So ass-uming any Takeouts will now export to "Records.json", Monkey wrote a script to process the "Records.json" file which can be described as follows:
- 1 locations list which can store many location element records.
- Each location element has the following fields:
- "source" (usually "UNKNOWN" but have also seen "CELL" here)
- "deviceTag" (device identifier)
- "platformType" (usually "ANDROID")
- "formFactor" (eg "PHONE")
- "serverTimestamp" (eg "2022-02-04T04:40:17.685Z", not always present, can be same for multiple location elements)
- "deviceTimestamp" (eg "2022-02-04T04:40:16.214Z", not always present, can be same for multiple location elements)
- "timestamp" (eg "2022-02-02T00:55:06.311Z", changes with each element record, to avoid confusion Monkey is calling this the "element timestamp")
- "latitudeE7" (in degrees scaled by 10 000 000)
- "longitudeE7" (in degrees scaled by 10 000 000)
- "altitude" (not always present, units are in metres per Josh Hickman)
- "heading" (not always present, units are degrees clockwise from True North per Josh Hickman eg 90 = East, 180 = South)
- "velocity" (not always present, units are metres per second per Josh Hickman)
- "accuracy" (units unknown, suspected to be in m)
- "verticalAccuracy" (not always present, units unknown, suspected to be in m)
- Each location element may/may not have an "activity" list.
- Each activity list has an activity "timestamp" and can store 1 or more "subactivitys".
- Note: "subactivity" is a monkey term used to label any child activitys which are listed in the activity list (see example below).
- Each "subactivity" can have multiple type and confidence (percentage) pairs (eg type = ON_FOOT, confidence = 3 and type = STILL, confidence = 89).
"activity": [{"activity": [{"type": "STILL","confidence": 89}, {"type": "ON_FOOT","confidence": 3}, {"type": "WALKING","confidence": 3}, {"type": "UNKNOWN","confidence": 3}, {"type": "IN_VEHICLE","confidence": 2}, {"type": "ON_BICYCLE","confidence": 2}, {"type": "IN_RAIL_VEHICLE","confidence": 2}, {"type": "IN_ROAD_VEHICLE","confidence": 1}, {"type": "IN_FOUR_WHEELER_VEHICLE","confidence": 1}, {"type": "IN_CAR","confidence": 1}],"timestamp": "2022-02-04T05:52:43.071Z"}]
Scripting
python3 gRecordsActivity_ijson_date.py -husage: gRecordsActivity_ijson_date.py [-i input_file -o output_dir -a start_isodate -b end_isodate]Extracts/parses "Detected Activity" data from Google Takeout "Records.json" (large files) and outputs TSV and KML files to given output diroptional arguments:-h, --help show this help message and exit-i INPUT Input Records filename-o OUTPUT Output KML/TSV directory-a START Filter FROM (inclusive) Start ISO date (YYYY-MM-DD)-b END Filter BEFORE (inclusive) End ISO date (YYYY-MM-DD)
python3 gRecordsActivity_ijson_date.py -i Records-4.json -o records4_outputRunning gRecordsActivity_ijson_date.py v2022-02-26ACTIVITY =>[{'activity': [{'type': 'STILL', 'confidence': 99}, {'type': 'UNKNOWN', 'confidence': 1}], 'timestamp': '2022-02-04T03:49:57.246Z'}]Element timestamp: 2022-02-04T03:49:55.263ZElement serverTimestamp: 2022-02-04T04:22:27.214ZElement deviceTimestamp: 2022-02-04T04:22:25.715ZNo. (sub)Activitys => 1(sub)Activity #1 timestamp = 2022-02-04T03:49:57.246ZNo. (sub)Activity types: 2ACTIVITY =>[{'activity': [{'type': 'STILL', 'confidence': 99}, {'type': 'UNKNOWN', 'confidence': 1}], 'timestamp': '2022-02-04T03:51:01.973Z'}]Element timestamp: 2022-02-04T03:50:48.614ZElement serverTimestamp: 2022-02-04T04:22:27.214ZElement deviceTimestamp: 2022-02-04T04:22:25.715ZNo. (sub)Activitys => 1(sub)Activity #1 timestamp = 2022-02-04T03:51:01.973ZNo. (sub)Activity types: 2[Output Redacted for brevity ...]ACTIVITY =>[{'activity': [{'type': 'TILTING', 'confidence': 100}], 'timestamp': '2022-02-04T04:24:24.209Z'}, {'activity': [{'type': 'STILL', 'confidence': 99}, {'type': 'UNKNOWN', 'confidence': 1}], 'timestamp': '2022-02-04T04:26:35.525Z'}, {'activity': [{'type': 'UNKNOWN', 'confidence': 40}, {'type': 'IN_VEHICLE', 'confidence': 10}, {'type': 'ON_BICYCLE', 'confidence': 10}, {'type': 'ON_FOOT', 'confidence': 10}, {'type': 'WALKING', 'confidence': 10}, {'type': 'RUNNING', 'confidence': 10}, {'type': 'STILL', 'confidence': 10}, {'type': 'IN_ROAD_VEHICLE', 'confidence': 10}, {'type': 'IN_RAIL_VEHICLE', 'confidence': 10}, {'type': 'IN_FOUR_WHEELER_VEHICLE', 'confidence': 10}, {'type': 'IN_CAR', 'confidence': 10}], 'timestamp': '2022-02-04T04:30:11.757Z'}]Element timestamp: 2022-02-04T04:30:36.434ZElement serverTimestamp: 2022-02-04T04:40:17.685ZElement deviceTimestamp: 2022-02-04T04:40:16.214ZNo. (sub)Activitys => 3(sub)Activity #1 timestamp = 2022-02-04T04:24:24.209ZNo. (sub)Activity types: 1(sub)Activity #2 timestamp = 2022-02-04T04:26:35.525ZNo. (sub)Activity types: 2(sub)Activity #3 timestamp = 2022-02-04T04:30:11.757ZNo. (sub)Activity types: 11[Output Redacted for brevity ...]ACTIVITY =>[{'activity': [{'type': 'STILL', 'confidence': 99}, {'type': 'UNKNOWN', 'confidence': 1}], 'timestamp': '2022-02-04T10:00:45.110Z'}]Element timestamp: 2022-02-04T10:00:32.756ZElement serverTimestamp: 2022-02-04T10:45:20.367ZElement deviceTimestamp: 2022-02-04T10:45:18.854ZNo. (sub)Activitys => 1(sub)Activity #1 timestamp = 2022-02-04T10:00:45.110ZNo. (sub)Activity types: 2Total no. of elements with at least one Activity = 36No. of elements with multiple Activitys = 2Processing Activitys ... Number of days = 1Processing 2022-02-04 = 39 entriesProcessed/Wrote 39 Total Activity entries to: records4_outputExiting ...
Example TSV Output |
Redacted Example KML output viewed in Google Earth Desktop |