Case Study: Intercom accelerates Fin.ai and reduces LLM waste with Honeycomb

Intercom cuts median time to first token by 60% with Honeycomb

Intercom, a software company, faced challenges with the performance of its AI customer service agent, Fin. Customers reported that Fin was slow, and despite internal optimizations to individual components, the engineering team lacked a customer-centric signal to understand the true end-user experience. They partnered with observability vendor Honeycomb to address this latency issue without sacrificing the agent's resolution efficiency.

Honeycomb helped Intercom implement a new observability strategy centered on "time to first token," a metric measured from the frontend to accurately represent customer wait times. By building detailed distributed traces that connected this high-level signal to low-level telemetry, Honeycomb enabled engineers to identify and optimize critical paths. This solution led to a 60% reduction in median time to first token, bringing it below eight seconds, and provided real-time cost visibility to prevent wasteful LLM calls, ultimately improving both customer satisfaction and business efficiency.

Intercom