Real Roads Don’t Have Four Wheels: Exploring MOTOR in FiftyOne

Every Driving Dataset Was Built Around a Car

Every driving dataset you've probably worked with was built around a car. nuScenes, Waymo, Cityscapes, IDD-X - four wheels, a stable chassis, a windshield-height sensor rig.

That's not what most of the world's roads look like. In India, Indonesia, and much of the Global South, motorcycles and scooters are the dominant mode of transport, and two-wheelers account for a disproportionate share of road fatalities relative to their numbers.

Riders lean hard into turns, weave between cars, brake on a dime, and share lanes with pedestrians, pushcarts, and parked trucks - none of which a four-wheeler dataset was ever designed to capture.

What is the MOTOR dataset?

MOTOR (MOtorized TwO-wheeler Rider), from Varun Paturkar, Shankar Gangisetty, and C.V. Jawahar at IIIT Hyderabad's CVIT lab, was built to close that gap.

The full release covers 1,629 annotated maneuver sequences (25+ hours) from 16 riders, captured over four weeks of real Indian traffic. A 324-clip subset of it has been imported into FiftyOne's grouped-dataset format and published on Hugging Face as Voxel51/motor_two_wheel_rider.

That's what we'll dig into here: four synchronized camera views per clip, GPS routes, gyroscope telemetry, and rider gaze heatmaps, all queryable in a handful of lines of Python.

Key Takeaways

• Voxel51/motor_two_wheel_rider contains 324 grouped clips × 4 camera slices (front, helmet, rear, eye_tracker) = 1,296 video samples, drawn from 14 of the original rides in the source corpus, size-budgeted to roughly 10 GB.
• Every clip contains eight emporalDetectionsT fields (event, legality, head_pose, road_type, road_marking, divider, traffic_density, n_lanes), a sample-level GPS route on the front slice, and - in 308 of 324 clips (95%) - frame-level gaze heatmaps from eye_tracker.
• FiftyOne's Map panel only renders a GeoLocation field (singular), not GeoLocations - a real API gotcha the import pipeline has to work around, and one worth knowing before you build your own geo-enabled dataset.
• In this build, weaving through traffic, distracted driving, and violation clips are annotated 100% illegal ; going straight, lane change, left turn, and stop are 100% legal - a clean structural split that says a lot about how "unspecified" maneuvers like U-turns get handled.
• In the full MOTOR benchmark, fusing RGB + gaze + telemetry on a Video Swin Transformer backbone reaches 52.9% accuracy on 11-way rider-behavior classification and 69.0% on 3-way legality classification - the best of four backbones tested and meaningfully better than RGB alone in both tasks.
• This is a size-budgeted subset of the 1,629-sequence corpus, round-robin sampled across event types for balance - great for exploring the schema and multi-view structure, not a substitute for the full corpus if you're trying to reproduce the paper's benchmark.

MOTOR Corpus Details

MOTOR is a research dataset from CVIT, IIIT Hyderabad ( paper , ICRA 2026). The full corpus comprises 1,629 annotated maneuver sequences (25+ hours of video) collected from 16 riders (13 male, 3 female, with 2-20 years of riding experience) over 4 weeks of naturalistic riding in India.

Each ride was captured with three GoPro Hero 10 cameras (front, rear, helmet - all 1920×1080 at 30 FPS with stabilization) plus wearable eye-tracking glasses (Project Aria or Pupil Labs), synchronized GPS at 10 Hz, and gyroscope/accelerometer data at 200 Hz.

The full corpus also includes on-road audio, which is not included in the FiftyOne build below.

Annotations cover four categories: traffic scene context (road type, lane count, markings, divider, density), rider state (gaze direction, GPS, speed), 12 riding maneuvers split into six conventional (going straight, left/right turn, lane change, u-turn, stop) and six unconventional behaviors (overtake, weaving through traffic, obstruction avoidance, distracted driving, traffic violation, near collision), and a legality label (Legal, Illegal, Unspecified) grounded in the Indian Motor Vehicle Act (2017).

The Hugging Face repository Voxel51/motor_two_wheel_rider is a curated, size-budgeted subset of that corpus - 324 clips (~9.7 GB), selected in a round-robin fashion across event types so no single maneuver dominates - imported into FiftyOne's grouped-dataset format for easy multi-view exploration.

It's the FiftyOne build we're exploring in this post, and it's a different (smaller) thing than the full 1,629-sequence release at varunpaturkar/MOTOR.

Why FiftyOne Fits a Multimodal Dataset Like MOTOR

A dataset like MOTOR is awkward to explore with ordinary tooling for three reasons:

• It's grouped and multi-view. Each clip is really four synchronized videos (front, helmet, rear, eye-tracker). A plain file browser can't show you all four keyed to the same frame.
• It's multimodal . Beyond pixels, you have GPS routes, per-frame telemetry, and 224×224 gaze heatmaps.
• Its labels are temporal. Behaviors like "lane change" or "weaving through traffic" span a range of frames, not a single bounding box.

FiftyOne handles all three natively. Grouped datasets (fo.Group) keep the four camera slices linked to a single clip and let you switch between them in the App using a single slice selector. TemporalDetections render frame-spanning event labels without a custom viewer. And GPS data plugs directly into the Map panel - with one real caveat.

Good to know. Why does the import script insist on fo.GeoLocation, not fo.GeoLocations? FiftyOne's Map panel currently only looks for a GeoLocation field (singular) on your samples - it doesn't render GeoLocations (plural), even though "a route made of many points" sounds like it should be the plural type.

This is a known, open constraint (see voxel51/fiftyone#5824 , confirmed by a FiftyOne core maintainer), and it's why MOTOR's import pipeline stores each clip's route as a single GeoLocation with a point (the Map panel anchor) and a line (the longest unbroken GPS segment) rather than the more semantically obvious GeoLocations. If you're building your own geo-enabled dataset, this is the kind of thing that costs an afternoon if you don't already know it.

Loading the Dataset

import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
dataset = load_from_hub("Voxel51/motor_two_wheel_rider")
print(dataset)
print(dataset.count_values("event.detections.label"))

That single call pulls a 324-group, 1,296-sample FiftyOne dataset with four slices already wired up:

• front: Front-mounted - GPS route + frame-level telemetry (default view)
• helmet: Helmet-mounted - First-person rider view
• rear: Rear-mounted - What's behind the rider
• eye_tracker: Eye-tracker - Rider POV + 224×224 gaze heatmaps (308/324 clips)

Seeing It: Four Views, One Timeline

Because the data is grouped, the most useful way to view a clip is synchronized four-pane playback - front, helmet, rear, and eye-tracker all advancing together. FiftyOne renders this natively once you launch the App on a grouped dataset:

import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
dataset = load_from_hub("Voxel51/motor_two_wheel_rider")
session = fo.launch_app(dataset)

Figure 1 - A real clip from the Voxel51/motor_two_wheel_rider FiftyOne build, shown as its four grouped slices. The front and helmet views capture near-identical road context from two mounting points; the rear view monitors the blind spot; the eye-tracker view provides the rider's own gaze data.

What's Actually in the Schema

Each sample - regardless of camera slice - carries the same clip-level metadata and the same eight TemporalDetections fields (they're duplicated across all four slices during import, since they describe the clip, not the camera):

• clip_id String Unique clip ID (e.g. "01_012")
• video_id Integer Source ride ID
• camera String front / helmet / rear / eye_tracker
• source_timestamp String Position in the full ride (MM:SS-MM:SS)
• duration_s Float Clip duration in seconds
• location GeoLocation GPS route (front slice only)
• event TemporalDetections Primary riding maneuver
• legality TemporalDetections Legal / illegal / unspecified
• head_pose TemporalDetections Rider head direction
• road_type TemporalDetections Paved / unpaved
• road_marking TemporalDetections Marked / unmarked lanes
• divider TemporalDetections Lane divider yes/no
• traffic_density TemporalDetections Low / medium / high
• n_lanes TemporalDetections Number of lanes (1-5)

Per-sample schema in the Voxel51/motor_two_wheel_rider FiftyOne build. On the front slice, frames also carry telemetry: gps (a per-frame GeoLocation point), speed_2d_mps, speed_3d_mps, gyro_x/gyro_y/gyro_z, gps_alt_m, and telemetry_timestamp_s. On eye_tracker, frames carry gaze, a Gaussian heatmap field.

Keep it honest. A few things worth knowing before you build on top of this subset:

• All 324 clips have a GPS route on location, but gaze heatmaps were only available for 308 of 324 clips (95%) - extraction from the red-marker overlay on the eye-tracker video doesn't succeed in every clip.
• The near-collision class has only 12 examples here (vs. ~28-29 for most other event types), because it's genuinely rare in the source corpus - the paper itself excludes it from its 11-way behavior classification benchmark for the same reason.
• One clip (01_012) has "High" recorded as its event label instead of a maneuver name - a stray annotation-import artifact affecting 1 of 324 clips, not a systemic issue.
• Clip durations run from 1 to 30 seconds, with 85% of clips under 6 seconds - shorter than the "typically 1-20 seconds" description on the dataset card, which describes the full corpus rather than this specific subset.
• On-road audio, part of the original MOTOR capture rig, isn't included in this FiftyOne build.

The Label Distributions, for Real

Querying the annotations directly (dataset.count_values(field)) on the 324 front-slice samples gives you the actual shape of this subset:

• Legality - 165 legal (51%), 91 illegal (28%), 68 unspecified (21%)
• Traffic density - 139 low (43%), 66 medium (20%), 119 high (37%)
• Road type - 312 paved (96%), 12 unpaved (4%)
• Road markings - 224 marked (69%), 100 unmarked (31%)
• Lane divider - 252 yes (78%), 72 no (22%)
• Number of lanes - 80 one-lane, 94 two-lane, 120 three-lane, 28 four-lane, 2 five-lane
• Head pose - 214 on the road (66%), 58 left (18%), 32 right (10%), 20 either side (6%)

from fiftyone import ViewField as F
illegal = dataset.match(F("legality.detections.label") == "illegal")
print(len(illegal), "of", len(dataset), "clips are annotated illegal")

Cross-Tabulating Labels: Where the Built-in Dashboard Needs Help

FiftyOne's built-in Dashboard panel handles single-field distributions out of the box, but a few of the more interesting questions about MOTOR require zipping two parallel TemporalDetections fields together - something the built-in form can't do, but a few lines of dataset.values() can:

dataset = ctx.dataset# inside a custom Dashboard plot
event_labels = dataset.values("event.detections.label")
legality_labels = dataset.values("legality.detections.label")

Running that cross-tab over this subset surfaces a clean three-way split: distracted driving, violation, and weaving through traffic clips are 100% illegal ; going straight, lane change, left turn, and stop clips are 100% legal ; and u-turn (illegal in only 1 of 29 clips) and obstruction avoidance (illegal in only 2 of 29) are overwhelmingly annotated unspecified rather than illegal - maneuvers the Indian Motor Vehicle Act doesn't explicitly regulate.

The same technique applied to head_pose vs. event shows rider attention tracks maneuver risk in an intuitive way: distracted driving is 100% off-road by definition, but u-turn (62% off-road) and violation (50%) show riders looking away far more than lane change (7%) or overtake (4%) - maneuvers that demand active attention to execute.

And event vs. duration_s shows maneuver complexity in seconds: u-turn clips average 8.0s and stop clips 5.8s, while obstruction avoidance (1.1s) and lane change (1.25s) are essentially instantaneous - reactive swerves rather than deliberate, sustained maneuvers. near collision clips average 2.4 seconds in this subset, which lines up closely with the 2.4-second figure the MOTOR paper reports for the full 1,629-sequence corpus.

Why the Benchmark Numbers Matter

The MOTOR paper benchmarks four video backbones - CNN-based S3D and ResNet3D, transformer-based MViTv2 and Video Swin Transformer (SwinT) - on the full 1,629-sequence corpus, with and without gaze and telemetry fusion. A few results worth knowing if you're evaluating this data for your own research:

Rider behavior classification (11 classes, near-collision excluded for data sparsity): SwinT with RGB+gaze+telemetry reaches 52.9% accuracy / 51.5% F1 , the best of the four backbones. Removing gaze costs 1.6% accuracy; removing telemetry costs 2.6%; removing both costs 5.2% - telemetry is the stronger standalone signal, but gaze and telemetry together beat either alone.

Legality classification (Legal / Illegal / Unspecified): SwinT with full fusion reaches 69.0% accuracy / 53.6% F1 . Here the gap is bigger - removing gaze costs 4.0% accuracy, removing telemetry costs 6.3%