Dynamic Obstacle Detection and Tracking

• Description: Develop a robust and efficient dynamic obstacle detection and tracking framework by integrating Lidar and Visual sensors. The Lidar detection module uses DBSCAN for initial 3D obstacle detection, providing a rough estimate of the scene. The Visual module, leveraging YOLO, refines these Lidar-based detections by identifying objects in the camera’s field of view, offering additional semantic information. The results from both modules are then processed using a Kalman Filter to ensure accurate and continuous tracking of obstacles.
• Github Repo: Github
• Demo Video: Youtube
• Role and Contribution:
  • Designed and implemented a LiDAR-Visual dynamic obstacle detection system.
  • Integrated advanced algorithms including DBSCAN, Kalman filters, and computer vision techniques to achieve stable and efficient detection performance.
  • Demonstrated in-depth expertise in sensor fusion, robotics, and algorithm optimization.

Tech Stack

• Hardware:
  • Sensors:
    • LiDAR: For precise distance measurements and obstacle detection.
    • Intel RealSense D435i: Provides RGB-D information
    • PX4 IMU: High-precision IMU(Inertial Measurement Unit) for motion tracking.
  • Flight Controller: PX4 Flight Controller for managing UAV operations.
  • Processing Unit: NVIDIA Jetson Orin NX for onboard computing.
  • Communication: Wi-Fi and telemetry modules for data transmission and remote control.
• Software:
  • Operating System: Ubuntu 20.04 LTS
  • Flight Stack: PX4 Autopilot for flight control and navigation.
  • Localization & Mapping: FAST-LIO2 for real-time localization.
  • Programming Languages: C++, Python
  • Middleware: ROS Noetic for managing communication between different system components.
  • Obstacle Avoidance Algorithms: Custom algorithms integrated with sensor data for real-time obstacle detection and avoidance.

Key Features

• Dynamic Obstacle Detection & Avoidance: Utilizes a dual-sensor approach combining LiDAR and visual sensors for real-time, robust obstacle detection and avoidance, ensuring safe navigation in complex environments.
• High Odometry Frequency: Enhanced odometry frequency through the integration of Fast-LIO2 and PX4 IMU, ensuring accurate and stable flight control.
• Light-Weight Design: Optimized for minimal weight to extend flight duration and improve maneuverability.
• Modular Architecture: Designed with modularity in mind, allowing for easy upgrades and maintenance of hardware and software components.