Dynamic Obstacle Detection and Tracking

UAV + Unitree Go2
LiDAR + RGB-D
Jetson Orin NX
ROS Noetic

A robust and efficient dynamic obstacle detection and tracking framework integrating LiDAR and visual sensors. The LiDAR module uses DBSCAN for initial 3D obstacle detection, giving a rough estimate of the scene. The visual module, leveraging YOLO, refines those detections by identifying objects in the camera’s field of view and adding semantic information. Results from both are fused through a Kalman filter for accurate, continuous tracking. The framework is validated on both UAV and Unitree Go2 platforms, and tested across obstacle sizes, velocities, and object types. UAV and Unitree Go2 robot platforms used for validation

My Role

Designed and implemented the LiDAR-visual dynamic obstacle detection system.
Integrated DBSCAN, Kalman filtering, and computer vision techniques for stable, efficient detection.
Worked across sensor fusion, robotics, and algorithm optimisation.

System overview of the LiDAR-visual detection and tracking pipeline

Detection and tracking results on a sequence of LiDAR and camera frames — Detection and tracking results. Play the full animation (73 MB), or watch the demo video.

Tech Stack

Key Features

Dynamic detection & avoidance

A dual-sensor approach combining LiDAR and vision for real-time, robust obstacle detection in complex environments.

High odometry frequency

FAST-LIO2 fused with the PX4 IMU raises the odometry rate, giving accurate and stable flight control.

Light-weight design

Optimised for minimal weight, extending flight duration and improving manoeuvrability.

Modular architecture

Hardware and software components can be upgraded or maintained independently.

Haoyu Shen