Normal spotifyimage

Spotify

New York, NY https://www.spotify.com/us/
   
San Francisco, CA  
   
Stockholm, Sweden  

About Spotify Engineering

With Spotify, it’s easy to find the right music for every moment – on your phone, your computer, your tablet and more. It was launched in the fall of 2008 and currently has over 40 millions users in over 56 countries. Users create and update 5 million playlists every day. Their engineers work to connect millions of people with their favorite songs and create a service that people love to use.

Small 928322 Chris Johnson on

From Idea to Execution: Spotify's Discover Weekly

spotifydiscoverweekly

This awesome talk by Chris Johnson and Edward Newett, machine learning engineers at Spotify, shows how they imagined, tested, iterated and built the highly-popular "Discover Weekly" feature of Spotify from start to finish.

Learn how product-oriented engineers think in this talk and the tradeoffs they make as they're looking for ways to rapidly test ideas and iterate. Of course, this product was built on music recommendations, so you'll also get to see how they thought through the process to figure out exactly how to generate meaningful recommendations for their millions of users.

Dataenconfnyc2016 logos3

This talk was a talk recorded at the DataEngConf 2015 event in NYC.

Small thumb speaker nevilleli Neville Li on

Scio, a new Scala API for Google Cloud Dataflow

Learn about Scio, a Scala API for Google Cloud Dataflow (incubated as Apache Beam). Apache Beam offers a simple, unified programming model for both batch and streaming data processing while Scio brings it much closer to the high level API many data engineers are familiar with, e.g. Spark and Scalding. Neville will cover design and implementation of the framework, including features like typesafe BigQuery macros, REPL, and serialization. There will also be a live coding demo.

Neville is a software engineer at Spotify who works mainly on data infrastructure and tools for machine learning and advanced analytics. In the past few years he has been driving the adoption of Scala and new data tools for music recommendation, including Scalding, Spark, Storm and Parquet. Before that he worked on search quality at Yahoo! and old school distributed systems like MPI.

This talk was given at the NYC Data Engineering meetup in June 2016.

Small 7481924 Noel Cody on

Cassandra: Data-Driven Configuration


Interested in learning more about data engineering and data science? Don't miss our 2 day DataEngConf with top engineers in San Francisco, April 2016.

Spotify currently runs over 100 production-level Cassandra clusters. We use Cassandra across user-facing features, in our internal monitoring and analytics stack, paired with Storm for real-time processing, you name it. With scale come questions. “If I change my consistency level from ONE to QUORUM, how much performance am I sacrificing? What about a change to my data model or I/O pattern? I want to write data in really wide columns, is that ok?”

Rules of thumb lead to basic answers, but we can do better. These questions are testable, and the best answers come from pre-launch load-testing and capacity planning. Any system with a strict SLA can and should simulate production traffic in staging before launch.

Unknown author on

Underflow bug

All of us are familiar with overflow bugs. However, sometimes you write code that counts on overflow. This is a story where overflow was supposed to happen but didn’t, hence the name underflow bug.

Round-robin


In our Java implementation of the round-robin algorithm, we store the number of connections in variable size and then we call index() % size to get the index of the chosen connection. The value of index() follows the sequence 0, 1, 2, 3, … For example with size = 3, we get index() % size equal to 0, 1, 2, 0, 1, 2, 0, …, which is exactly how round-robin works.

How is index() implemented? About a month ago, it looked the following way.


int index() {

int index = this.index.get();
if (index < 0) {
index -= Integer.MAX_VALUE;
}
return index;
}


where this.index is AtomicInteger (we can’t just use an int, because the code is run in parallel). It’s value is increased later, after a sendable connection has been found.

The function index() tries to convert negative integers into positive. Before we dive into why it doesn’t work, let’s look at the representation of positive and negative integers.

Unknown author on

Protect Your Baby! How Spotify Does Testing for Mobile

Sean Kenny, Android developer at Spotify, talks about how they've reduced crash rates by building a veritable fortress of tests. Sean shares their testing for mobile methodology/process, covering their testing levels (unit [Robolectric], test automation, manual) and crash reporting, as well as how they've improved over time to create a more stable and robust application.

This video was recorded at the New York Android Developers meetup at Spotify in NYC.

Unknown author on

It's Not a Bug, It's a Feature!

The Android platform consists of a huge amount of code. With so much code, it is not uncommon that bugs and unexpected behavior occurs. In this talk, Erik Hellman explains some of the more interesting bugs and API behaviors that are present and how to deal with them. From subtle UI glitches, concurrency issues, the 64k method limit and more, this session will save you lots of time once you run into these bugs.

Code samples here.

This video was recorded at the New York Android Developers meetup at Spotify in NYC.

Small kinshuk mishra Kinshuk Mishra on

How Spotify Scales Apache Storm

Spotify has built several real-time pipelines using Apache Storm for use cases like ad targeting, music recommendation, and data visualization. Each of these real-time pipelines have Apache Storm wired to different systems like Kafka, Cassandra, Zookeeper, and other sources and sinks. Building applications for over 50 million active users globally requires perpetual thinking about scalability to ensure high availability and good system performance.


Interested in learning more about data engineering and data science? Don't miss our 2 day DataEngConf with top engineers in San Francisco, April 2016.

Small 1054171 Gandalf Hernandez on

We have an S3 bucket with 35M files

Introduction


Spotify joined forces with The Echo Nest this spring. The Echo Nest specializes in, among other things, knowing as much as possible about individual songs. For example, they can figure out the tempo and key of a song, if it is acoustic or electric, if it is instrumental or has vocals, if it is a live recording, and so on. Very exciting stuff!

During the past couple of months The Echo Nest has been running their audio analyzer over a big portion of the tracks in our catalog, uploading the resulting analysis files to an S3 bucket for storage.

As we integrate The Echo Nest into Spotify, we want to start making use of the analysis output within the main Spotify pipelines.

My problem is that I have data for 35 million tracks sitting in 35 million individual files, totaling 15-20TB, in an S3 bucket that we need to get into our Hadoop cluster, which is the starting point for most of our pipelines.

Small piyush Piyush Narang on

How Fine Should I Slice My Services?

A large reason why micro services appeal to programmers is that they resonate with the UNIX philosophy - make each program do one thing well and ensure a separation of concerns. While micro-services are quite the rage at most companies, there are a few dissenting opinions that are fans of the opposite end of the spectrum - monolithic services. This often results in both sides indulging in fairly engaging debates. Occasionally 140 characters at a time.

There are patterns to getting monolithic services to work well in production systems. Etsy is one of the places that seems to have it working reasonably well (continuous integration, multiple deployments a day, per developer VMs, monitoring, dashboards & alarms). Personally though, I'm wary about a few aspects of the monolithic setup. I'm not sure how well it can work when sizes of development  teams cross a few hundred engineers. Netflix has come out publicly about a lot of their struggles with monolithic architectures as their size grew in the early years. I’d heard similar stories about the early days while I was at Amazon (though that might not be entirely applicable as it was quite a few years back). Scaling various components might also prove to be a challenge in such architectures.

There is an aspect of micro service design that I've had my own share of lively debates over.   It centers on deciding what bits of functionality needs to be part of a service. Very strict separation of concerns can lead services to be very minimal (or the opposite - bloated services). I've listed some aspects that I’ve learnt to pay attention to while thinking about how to structure my services.

Join Us