Case Study: Datadog achieves scalable service discovery across VMs and Kubernetes with HashiCorp Consul

Service Discovery at Datadog

Datadog, a SaaS-based monitoring company processing trillions of datapoints per day, needed a way to handle service discovery across a rapidly growing mix of tens of thousands of VMs and multiple Kubernetes clusters. As Datadog migrated from VMs to Kubernetes, it faced challenges with cross-cluster communication, VM-to-Kubernetes discovery, and the need for encryption and authorization at scale. HashiCorp products, including Consul, Vault, Terraform, and Packer, were already part of Datadog’s infrastructure.

To address these issues, Datadog used HashiCorp Consul for service discovery, including Consul DNS to reach services running on VMs from Kubernetes. For VM-to-cluster and cluster-to-cluster communication, Datadog also relied on ingresses and DNS as an interim approach, while planning to move toward service mesh for stronger authentication, encryption, and more efficient multi-platform communication. The result was a workable service discovery strategy across heterogeneous environments, supporting thousands of services, tens of thousands of nodes, and hundreds of thousands of pods, though Datadog noted that a full service mesh would better solve the remaining gaps.

Datadog

Laurent Bernaille

Staff Engineer