Case Study: Together AI accelerates agent-authored data pipelines with Astro by Astronomer

Together AI cuts pipeline development from weeks to hours with Astronomer, Inc.

Together AI, a company that operates an AI acceleration cloud, faced significant challenges with its fragmented data orchestration infrastructure. They were managing nine separate MWAA environments and 17 bespoke Argo workflows, which resulted in slow manual CI/CD deployments, a lack of observability, and no unified platform. This setup was a major impediment to their goal of having AI agents autonomously build data pipelines. To overcome this, they turned to the vendor Astronomer and its product Astro for a modern Airflow foundation.

Astronomer implemented Astro to consolidate all orchestration onto a single platform with automated CI/CD and integrated observability. This provided the stable, programmable foundation needed for AI agents to author pipelines via the Airflow MCP server. The results were immediate and substantial: CI/CD deploy times were reduced 60x from 10 minutes to seconds, and agent-authored DAGs cut pipeline development time from weeks to hours, a 20x improvement. The agents have now autonomously built and maintain over 700 dbt models in production, processing billions of rows of data weekly.

Together AI