Case Study: Lyft improves 3D detection with Weights & Biases

Lyft’s High-Capacity End-to-End Camera-Lidar Fusion for 3D Detection

Lyft Level 5, the autonomous vehicle division of the ride-sharing company, faced the challenge of improving the 3D detection of objects for its self-driving cars. Their existing two-stage sensor fusion approach, which processed camera and lidar data separately, limited performance and was not fault-tolerant against missing data. To develop a more accurate and robust perception system, they utilized their internal ML framework, Jadoo, in conjunction with Weights & Biases.

With Weights & Biases, Lyft implemented a high-capacity, end-to-end fusion model that learns to combine camera and lidar data directly. This solution, which included strategies to prevent overfitting and speed up training, resulted in a model that significantly outperformed their previous method, boosting detection accuracy for pedestrians by approximately 5%. The use of Weights & Biases helped the team achieve faster iteration and a 4.3x speedup in training, enabling them to develop a more reliable perception system for autonomous vehicles.

Lyft