Case Study: C4ADS improves analysis of heterogeneous data with ClearScale on AWS

Implementing a Large AWS Data Lake for Analysis of Heterogeneous Data

C4ADS, a Washington DC nonprofit focused on data-driven analysis of global conflict and transnational security issues, needed a faster way to work through growing volumes of data from many sources and formats. As the number of datasets it ingested increased, analysts struggled to sift through information quickly enough to support timely evidence-based reporting. C4ADS worked with ClearScale and AWS to address the problem.

ClearScale implemented a secure AWS data lake in a Virtual Private Cloud, using S3 for storage, DynamoDB for file metadata, Elasticsearch for search, and Lambda and API Gateway to automate ingestion and metadata creation. The solution also included a web-based interface, SAML-based authentication through Cognito, and encryption at rest and in transit. ClearScale delivered the system in two months, including staging, production, and staff training, and launched it in October 2017.

C4ADS

David W. Lynch

Chief of Analysis