Case Study: OLMo achieves open-source state-of-the-art model training with Databricks

OLMo Is Here, Powered by Databricks

OLMo, the open-source large language model from the Allen Institute for AI, wanted to build and train a state-of-the-art model while keeping its work open and reproducible. To support that effort, OLMo used Databricks and its Databricks Model Training Platform to scale model training and experimentation.

Databricks enabled the OLMo team to run training and ablations more easily and to launch multi-node jobs quickly from their laptops, speeding iteration and simplifying large-model development. The result was OLMo 7B, an open-source state-of-the-art LLM, with Databricks citing the collaboration as proof that its platform can make building and fine-tuning large models more straightforward and efficient.

OLMo

Pete Walsh

Senior Software Engineer