Case Study: Apple achieves on-device reply suggestions with Stream Chat and Apple Foundation Models

Apple builds on-device reply suggestions with Stream’s ~3B model and zero token cost

Apple partnered with Stream Chat to build a feature for on-device reply suggestions within a SwiftUI chat application. The challenge was to implement a responsive and private suggestion feature that avoids network latency and API costs, while ensuring conversation data never leaves the user's device.

Stream provided its Chat SDK to handle the core, cloud-based messaging infrastructure. The solution integrated Apple's Foundation Models framework to run a local AI model for generating reply suggestions, resulting in a seamless user experience with no token cost, low latency, and enhanced privacy. The integration was achieved with minimal code, demonstrating how Stream's reliable cloud services can effectively complement on-device AI for specific, high-frequency tasks.

Apple