2D/3D Spatial Pixel Mapping & Ground-Plane Localization Engine
- Project
- 22006 SINTRA
- Type
- Open Source Software
- Description
TAV Technologies developed and open-sourced a lightweight 2D-to-3D spatial pixel mapping and ground-plane calibration component (aruco-lidar-ground-plane) to bridge perspective CCTV imagery with metric 3D physical coordinates in indoor terminal environments. In multi-sensor security deployments, operators struggle to correlate a 2D bounding box on a camera screen with physical positions reported by 3D LiDAR or mmWave radar sensors. By combining visual ArUco fiducial markers with 3D LiDAR point-cloud ground-plane estimation, this component computes accurate homography and extrinsic transformation matrices. Once calibrated, any 2D pixel detection (such as a fallen passenger, unattended fixture, or security anomaly) is instantaneously projected onto a unified 3D floor-plan coordinate frame without requiring continuous depth hardware on every camera.
- Contact
- Selim Sarı
- selim.sari@tav.aero
- Research area(s)
- Sensors / IoT, Artificial Intelligence / Data analytics, Safety & Security, Smart mobility / Transport
- Technical features
Cross-modal extrinsic calibration: Aligns monocular RGB camera perspectives with 3D LiDAR ground-plane point clouds using ArUco fiducial marker detection. Real-time 2D-to-3D projection: Maps 2D image pixel coordinates (u, v) (e.g., object bounding-box footpoints) into metric 3D floor-plane coordinates [x, y, z] with negligible CPU overhead. Sensor-fusion readiness: Enables spatial alignment and late/mid-level fusion between standard CCTV video analytics and privacy-preserving 3D range sensors (LiDAR and mmWave radar) in a shared terminal coordinate system. Calibration-once workflow: Requires 3D LiDAR and ArUco markers only during the initial setup/calibration phase; runtime inference operates purely on standard 2D camera outputs using the exported transformation matrices.
- Integration constraints
Software dependencies: Python 3 environment with OpenCV (including aruco module), NumPy, and 3D point-cloud processing libraries. Scene geometry: Assumes a locally planar floor surface within the calibrated zone; multi-level scenes (stairs, escalators, ramps, or split-level mezzanines) require separate plane segmentation or per-zone calibration matrices. Installation stability: Fixed-mount cameras are required (PTZ cameras invalidate static extrinsic matrices unless preset-specific calibrations are stored); physical camera displacement requires recalibration. Calibration setup: Temporary placement of printed ArUco markers within the overlapping field of view of the camera and reference 3D LiDAR during commissioning.
- Targeted customer(s)
Computer vision and multi-sensor fusion R&D teams, system integrators, and smart infrastructure developers. Airport, retail, and industrial digital-twin platform providers requiring metric floor-plan localization from existing 2D CCTV networks.
- Conditions for reuse
Open source under the MIT License (maintained by TAV Technologies). GitHub Repository (https://github.com/gmzeztrkkk/aruco-lidar-ground-plane)
- Confidentiality
- Public
- Publication date
- 10-11-2026
- Involved partners
- TAV Technologies (TUR)