Transforming data collaboration at SoundCloud

SoundCloud empowers artists and fans to connect and share through music. Founded in 2007, SoundCloud is an artist-first platform empowering artists to build and grow their careers by providing them with the most progressive tools, services, and resources. With over 375 million tracks from 40 million artists, the future of music is SoundCloud.

Managing an operation of this scale requires data to be the backbone of the artist-first platform. For Brian, Principal Data Analyst at SoundCloud, these challenges are all too familiar. Acting as the technical lead for the Data & Insights Team, Brian supports his team members in navigating the technical and collaborative challenges that come with projects at scale, while also taking a spot in product leadership meetings to maintain crucial business context. Along with the DISC team, the data platform team, data science team, and analytics engineering team form the Data Foundations department at SoundCloud.

With limited bandwidth and a torrent of priorities, the Data & Insights team needs to work effectively. Working in a hybrid structure means analysts are embedded in different Product & Business Domain steams (i.e. Anti-Abuse Search, Growth, Music, etc) while simultaneously being part of the core analytics team. This way, they make sure to provide crucial insights into every corner of the company.  

At SoundCloud, analysts have the freedom to work with the tools they like. While Google BigQuery is the main tooling for SQL tasks, many team members prefer using their own stack for specialized tasks. Some chose to work with Vertex AI or Google Colab, while others preferred to work on local Jupyter notebooks or other Python IDEs. This fragmentation in stack leads to fragmentation in data collaboration, knowledge sharing, and technical expertise.

Embedded in different teams, analysts at SoundCloud often collaborate on cross-functional projects using a variety of tools and dashboards to best fit their individual needs.  

As at any Company, the data team is on the receiving end of Ad-Hoc questions from the Business. The team found that many stakeholder requests were repetitive, similar to questions previously answered. However, due to the lack of a central repository for their work, analysts struggled to find and utilize previous analyses. This lack of discoverability resulted in analysts spending hours each month either searching for old work or unnecessarily starting new analyses from scratch.

In previous roles, Brian used data notebooks extensively. He recognized the power of notebooks for analytics and data science work and was naturally intrigued by Deepnote. While extremely powerful, traditional data notebooks lack the features you would expect out of modern business software. Problems with collaboration and reproducibility are notable and constant and, to even get started, a certain technical level must be achieved.

"Deepnote combines the power of data notebooks with a user-friendly interface, making it accessible for non-technical members and promoting collaboration.", solving the key issues with traditional notebooks and creating a source of truth for analysts' work. Thanks to its versatility, analysts could do a lot more in Deepnote, from simple SQL queries to complex Python projects. All while collaborating in real-time, enabling knowledge sharing, and overall reducing friction in collaboration.

Analysts were not the only ones to see the benefits of Deepnote. Working with notebooks meant that analysts could add important context to their analysis and quickly share them with stakeholders as Deepnote Apps. Likewise, stakeholders would be able to consume the insights directly from the source, leaving feedback directly where the work happens.

Deepnote AI was the “cherry on the top” for the team. Being a SQL-first team, analysts had different levels of expertise when working with Python. Deepnote AI unlocked a technical boundary for them, allowing them to work on more complex projects effectively.

Anyone who’s bought software for a larger organization knows that implementing a new tool to your stack can cost significant time and resources. But that’s not always the case. Deepnote allowed them to easily integrate with BigQuery (and any other data source) and get the team up and running from day one.

In line with how SoundCloud works, Brian didn’t want to just “enforce” a new tool to the team. Instead, he opted for a more organic approach, releasing Deepnote to a few teammates who were the most intrigued about a new notebook platform. Thanks to Deepnote's collaborative capabilities, it wasn’t long until analysts started sharing their analysis in Deepnote, including other analysts and stakeholders. In the first 3 months, 30 people were working inside Deepnote, and this number quickly grew to 100 in the first 6 months.

Fast-forward to this day, most of SoundCloud’s data analytic work happens in Deepnote. Analysts collaborate between themselves and with stakeholders constantly, and the team has gained momentum to “get things done”.