Routing billions of analytics events with high deliverability in a hostile environment
Segment’s API has scaled significantly over the past three years and has grown from processing a trickle of events to tens of thousands per second. Today, Segment processes tens of billions of events each month and sends them to hundreds of partner APIs.
It can be a very hostile environment. Partners fail frequently, customers send highly variable data and instances regularly die. As a result, Segment has invested heavily in tools for monitoring, failover and fairness when it comes to routing events through its system.
In this talk, CTO Calvin French-Owen will discuss how Segment continues to maintain a high quality of service, how its infrastructure has evolved over time and where it's heading in the future.
This talk is from DataEngConf SF in April 2016.