geomap-engine: fusing AR pose + object detections into a 2D map, in Rust

Every AR toolkit (ARKit, ARCore, RoomPlan, Polycam) bundles SLAM, object detection, and a UI into one closed, heavyweight package. If you just want the middle piece - turn a stream of {camera pose, object detections} into a clean, deduplicated 2D map of "here's a chair, here's a door, here's where they are" - there's nothing small and embeddable that does just that. So I built geomap-engine: an open-source Rust crate that does exactly this one job.

What it does, what it doesn't

Feed it a Frame (camera pose + intrinsics + a list of 2D bounding-box detections) and it maintains a SceneMap: one stable entry per real object, not one per detection.

It explicitly does not:

• run SLAM (bring your own ARKit/ARCore/ORB-SLAM3/whatever)
• run object detection (bring your own model)
• have any UI

It's the glue layer that sits between those and your app/robot/tool.

How it works

• Projection - a 2D bbox + camera pose + intrinsics becomes an estimated 2D world position, via ray-plane intersection with the ground plane (v0.1 assumes a known/fixed camera height).
• Association - nearest-neighbor match against existing tracked objects, by label + position proximity.
• Fusion - repeated observations merge into one object: running-average position, confidence combined as independent evidence so it grows toward 1 with repeated sightings instead of just averaging.
• Map maintenance - a single staleness rule handles pruning one-off noise, stale objects, and objects that moved, all at once.

Try it

use geomap_engine::Engine;
use geomap_engine::proto::{CameraIntrinsics, CameraPose, Detection, Frame};

let mut engine = Engine::new();

let frame = Frame {
    pose: Some(CameraPose {
        timestamp: 0.0,
        x: 0.0,
        y: 0.0,
        z: 1.5, // 1.5m camera height above the floor
        qx: 1.0,
        qy: 0.0,
        qz: 0.0,
        qw: 0.0, // looking straight down
    }),
    intrinsics: Some(CameraIntrinsics {
        fx: 500.0,
        fy: 500.0,
        cx: 320.0,
        cy: 240.0,
    }),
    detections: vec![Detection {
        label: "chair".into(),
        confidence: 0.8,
        bbox_x: 0.45,
        bbox_y: 0.45,
        bbox_w: 0.1,
        bbox_h: 0.1,
    }],
};

let scene_map = engine.ingest_frame(frame);
println!("{}", scene_map.to_geojson_json().unwrap());

Output is GeoJSON-shaped (FeatureCollection of Point features), so you can drop it straight into geopandas/matplotlib or any map viewer without writing a converter.

Why Rust + Protobuf

Rust for portability and eventual FFI into iOS/Android. The Frame/SceneMap schema is Protobuf, so any front-end - Swift, Kotlin, a Python test harness - can feed it language-agnostically without caring what the engine is written in.

Status

v0.1 is implemented and tested - 11 tests, cargo build / cargo test / cargo clippy all clean, CI running on every push. It's tested against both a hand-written fixture and a real camera trajectory from the TUM RGB-D benchmark (with synthetic detections and realistic detector noise layered on top), to make sure the association/fusion logic holds up against actual sensor jitter, not just clean synthetic input. Pre-1.0, so the API may still shift a bit.

Get it

geomap-engine = https://github.com/ghimirebibek/geomap-engine

Repo (MIT licensed). Feedback, issues, and PRs welcome - especially from anyone who's fought with ARKit/ARCore object placement before and has opinions about the projection/association assumptions.