Case Study: GitHub achieves faster semantic issue search with CloudQuery, BigQuery, and Vertex AI

GitHub runs semantic issue search in 31 seconds with CloudQuery

The customer, GitHub, faced the challenge of managing a rapidly growing backlog of issues for their cloudquery/cloudquery repository. Finding themes and specific requests within this backlog required inefficient manual searching. To address this, they worked with vendor CloudQuery to implement a solution using its GitHub source plugin and BigQuery destination plugin, integrated with Google Cloud's Vertex AI and BigQuery.

CloudQuery's solution automatically synced all open GitHub issues into BigQuery. As part of the sync, BigQuery used a remote model to call Vertex AI for generating text embeddings, which were stored in a separate table. This enabled a semantic search capability where a Python script could embed a user's query, perform a vector search in BigQuery to find the most relevant issue chunks, and then use Gemini to generate a comprehensive answer. The result was a powerful, fully managed semantic search system that required no additional database operations, offered elastic scale, and provided a simple pay-per-query cost model, turning the backlog into an easily searchable knowledge base.

GitHub