Case Study: AT&T achieves significant cost reduction and accurate natural-language-to-SQL query generation with H2O.ai's H2O LLM Studio

World’s leading telco leverages H2O LLM Studio to finetune models and enhance query generation with reduced costs

AT&T’s CDO Office needed a simple, precise way for non-technical business users to query company databases in plain English (for example: “Number of prospective customers that are fiber-eligible, limited to residential, excluding employees and vacant addresses”) so teams could uncover campaign and expansion opportunities. Facing strict accuracy requirements and high operational costs from large LLMs, AT&T worked with H2O.ai using H2O LLM Studio to explore fine‑tuning smaller models as a lower‑cost, lower‑latency alternative.

Using H2O.ai’s H2O LLM Studio, the team fine‑tuned a Llama‑Transformer SQL coder 8B model via data curation, semantic similarity search, grid search hyperparameter tuning, and deployment. The fine‑tuned model delivered performance comparable to GPT‑4 for the task while cutting costs dramatically (previously ~ $0.02 per call versus effectively free for the fine‑tuned 8B model), improved latency, and increased ROI; AT&T also validated text‑to‑SQL quality on the BIRD‑SQL benchmark (holding a top rank), demonstrating measurable accuracy and efficiency gains from the H2O.ai solution.

AT&T