Google Cloud Platform Blog
Spotify chooses Google Cloud Platform to power data infrastructure
Tuesday, February 23, 2016
It’s not every day you move a 75 million+ user company from a home-grown infrastructure to the cloud. But if you use Spotify, more and more of your musical experience will be delivered by
Google Cloud Platform
over the coming weeks and months
—
we’re partnering on an ambitious project to move Spotify’s backend into GCP.
Spotify aims to make music special for everyone. Today, the company hosts more than
2 billion playlists
and gives consumers access to more than 30 million songs. Users can search for music across any device by artist, album, genre, playlist or record label, while features like Discover Weekly suggest personalized playlists for millions of people around the world.
While Spotify had engineers running its core infrastructure and buying or leasing data-center space, PC hardware and networking gear to provide a seamless experience for users — time and again it asked whether the tradeoff of resources that could otherwise focus on innovative features and software, was worth it.
Recently Spotify decided it didn’t want to be in the data center business, and chose Cloud Platform over the public cloud competition after careful review and testing. The company split their migration to Cloud Platform into two streams: a services track and a data track. Spotify runs their products on a multitude of tiny microservices, several of which are now being moved from on premise data centers into Google’s cloud using our Cloud Storage,
Compute Engine
and other products.
With Compute Engine, teams can rely on consistent performance from ultra high IOPS SSD and local SSD storage capabilities. And with autoscaling, they can build resilient and cost-efficient applications that use just the right amount of resources necessary at any given time. For storage, Spotify is now implementing
Google Cloud Datastore
and
Google Cloud Bigtable
. This rich fabric of storage services lets engineers work on complex back end logic, instead of focusing on how to store the data and maintain databases. Spotify is also deploying Google’s Cloud Networking services, such as
Direct Peering
,
Cloud VPN
and
Cloud Router
, to transfer petabytes of data. This results in a fast, reliable and secure experience for users around the globe.
On the data side of things, the company is adopting an entirely new technology stack. This includes moving from Hadoop, MapReduce, Hive and a series of home-grown dashboarding tools, to adopting the latest in data processing tools, including
Google Cloud Pub/Sub
,
Google Cloud Dataflow
,
Google BigQuery
, and
Google Cloud Dataproc
.
With BigQuery and Cloud Dataproc, data teams can run complex queries and get answers in a minute or two, rather than hours. This lets Spotify perform more frequent in-depth, interactive analysis, guiding product development, feature testing and more intelligent user-facing features. To gather and forward all events to its ecosystem, Spotify is using Cloud Pub/Sub, Google’s global service for messaging and streaming data. This gives teams the ability to process hundreds of thousands of messages per second, in a reliable no-ops manner. And to power its ETL workloads, Spotify is deploying Cloud Dataflow, Google’s data processing service. This lets the company rely on a single cloud-based managed service for both batch and stream processing.
What makes us most excited to work with Spotify is their company-wide focus on forward-looking user experiences. Now that they’ve begun using Google Cloud Platform, we can’t wait to see what Spotify builds next.
Join us for the
GCP NEXT 2016
opening keynote, where we’ll feature a
talk
from Nicholas Harteau, VP of Engineering and Infrastructure at Spotify. You can also attend Spotify-led technical
sessions
where you can learn more about how they’re deploying Google Cloud BigQuery and Dataflow.
- Posted by Guillaume Leygues, Lead Sales Engineer, Google Cloud Platform
No comments :
Post a Comment
Don't Miss Next '17
Use promo code NEXT1720 to save $300 off general admission
REGISTER NOW
Free Trial
GCP Blogs
Big Data & Machine Learning
Kubernetes
GCP Japan Blog
Labels
Announcements
56
Big Data & Machine Learning
91
Compute
156
Containers & Kubernetes
36
CRE
7
Customers
90
Developer Tools & Insights
80
Events
34
Infrastructure
24
Management Tools
39
Networking
18
Open Source
105
Partners
63
Pricing
24
Security & Identity
23
Solutions
16
Stackdriver
19
Storage & Databases
111
Weekly Roundups
16
Archive
2017
Feb
Jan
2016
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2015
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2014
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2013
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2012
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2011
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2010
Dec
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2009
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Feed
Subscribe by email
Technical questions? Check us out on
Stack Overflow
.
Subscribe to
our monthly newsletter
.
Google
on
Follow @googlecloud
Follow
Follow
Follow
No comments :
Post a Comment