Case Study: U.S. Department of Transportation achieves real-time ML aviation insights and 90% compute-cost reduction with Databricks

Activating aviation data with real-time ML‌

The U.S. Department of Transportation needed to build a real-time commercial aviation flight database to meet updated reporting requirements and accurately measure aviation system performance. The challenge was unifying fragmented SWIM data (weather, flight, aeronautical and surveillance) from multiple publishers and on‑prem systems, ensuring real-time data quality and scalable ingestion for analytics and ML.

By adopting Microsoft Azure and the Databricks Lakehouse Platform with Delta Lake and Apache Kafka, USDOT automated batch and streaming pipelines, centralized data for Tableau/Power BI dashboards, and enabled ML-powered forecasts. The solution improved team collaboration, delivered more accurate traffic and passenger predictions at scale, and cut streaming data compute costs by 90%.

U.S. Department of Transportation

Mehdi Hashemipour

Data Scientist