Case Study: EmPushy Improves Push Notification Classification with Appen Data Annotation Services

EmPushy Leverages Appen Data Annotation Services to Improve User Experience

EmPushy, an organization providing audience intelligence for marketers, faced the challenge of analyzing push-notifications in a noisy mobile environment. They needed a tool to perform multi-label data annotation for their NLP models with a focus on quality, speed, and scale, which had never been done before in their domain. They turned to vendor Appen to find a partner that could meet these novel criteria.

Appen provided a combination of its Open Crowd and a Dedicated Crowd to annotate the text of push-notifications, classifying their call-to-action. This solution enabled EmPushy to significantly improve its model accuracy, raising its F1 score from 60% to over 85%. The project collected over 223,000 judgments, which helped EmPushy acquire new clients and fuel research publications.

EmPushy

Kieran Fraser

CEO & Co-Founder