Case Study: Sina Weibo achieves scalable, Kafka-compatible streaming with StreamNative’s Apache Pulsar and KoP

How Sina Weibo Migrated from Kafka to Pulsar

Sina Weibo, a gaming and social media company, was operating large Kafka clusters to process more than 1 trillion messages per day across logs, orders, ads, and key events. As traffic surged, the team faced major challenges with manual scaling, uneven data distribution, extra network load during rebalancing, and difficult migration requirements across many business systems and Kafka versions.

StreamNative helped Sina Weibo adopt Apache Pulsar with KoP (Kafka on Pulsar), giving them native Kafka compatibility while benefiting from Pulsar’s separated storage and compute architecture for better elasticity and faster rebalancing. The team also built additional metadata handling to improve consistency across brokers, and the environment has been running securely and stably since September 2021. StreamNative’s approach allowed Sina Weibo to migrate without major code changes while supporting older Kafka clients and improving operational stability.

Sina Weibo

Wenbing Shen

Data Platform Engineer