Routing billions of analytics events with high deliverability in a hostile environment

Segment’s API has scaled significantly over the past three years and has grown from processing a trickle of events to tens of thousands per second. Today, Segment processes tens of billions of events each month and sends them to hundreds of partner APIs.

It can be a very hostile environment. Partners fail frequently, customers send highly variable data and instances regularly die. As a result, Segment has invested heavily in tools for monitoring, failover and fairness when it comes to routing events through its system.

In this talk, CTO Calvin French-Owen will discuss how Segment continues to maintain a high quality of service, how its infrastructure has evolved over time and where it's heading in the future.

This talk is from DataEngConf SF in April 2016.