Dynamic Obstacle Detection and Tracking

• Description: Develop a robust and efficient dynamic obstacle detection and tracking framework by integrating Lidar and Visual sensors. The Lidar detection module uses DBSCAN for initial 3D obstacle detection, providing a rough estimate of the scene. The Visual module, leveraging YOLO, refines these Lidar-based detections by identifying objects in the camera’s field of view, offering additional semantic information. The results from both modules are then processed using a Kalman Filter to ensure accurate and continuous tracking of obstacles. The framework has been successfully validated on both UAV and Unitree Go2 platforms, demonstrating cross-platform compatibility and adaptability. Extensive testing has been conducted to evaluate the detection performance across various obstacle characteristics, including different sizes, velocities, and object types, ensuring robust performance in diverse real-world scenarios.

• Github Repo: Github
• Demo Video: Google Drive
• Role and Contribution:
  • Designed and implemented a LiDAR-Visual dynamic obstacle detection system.
  • Integrated advanced algorithms including DBSCAN, Kalman filters, and computer vision techniques to achieve stable and efficient detection performance.
  • Demonstrated in-depth expertise in sensor fusion, robotics, and algorithm optimization.

Tech Stack

• Hardware:
  • Sensors:
    • LiDAR: For precise distance measurements and obstacle detection.
    • Intel RealSense D435i: Provides RGB-D information
    • PX4 IMU: High-precision IMU(Inertial Measurement Unit) for motion tracking.
  • Flight Controller: PX4 Flight Controller for managing UAV operations.
  • Processing Unit: NVIDIA Jetson Orin NX for onboard computing.
  • Communication: Wi-Fi and telemetry modules for data transmission and remote control.
• Software:
  • Operating System: Ubuntu 20.04 LTS
  • Flight Stack: PX4 Autopilot for flight control and navigation.
  • Localization & Mapping: FAST-LIO2 for real-time localization.
  • Programming Languages: C++, Python
  • Middleware: ROS Noetic for managing communication between different system components.
  • Obstacle Avoidance Algorithms: Custom algorithms integrated with sensor data for real-time obstacle detection and avoidance.

Key Features

• Dynamic Obstacle Detection & Avoidance: Utilizes a dual-sensor approach combining LiDAR and visual sensors for real-time, robust obstacle detection and avoidance, ensuring safe navigation in complex environments.
• High Odometry Frequency: Enhanced odometry frequency through the integration of Fast-LIO2 and PX4 IMU, ensuring accurate and stable flight control.
• Light-Weight Design: Optimized for minimal weight to extend flight duration and improve maneuverability.
• Modular Architecture: Designed with modularity in mind, allowing for easy upgrades and maintenance of hardware and software components.