Google Cloud Platform Blog
Offline Disk Import and the OmNomNom Machine
Thursday, November 14, 2013
Yesterday, we announced that we are
expanding our offline disk import service
to better serve users globally. With new disk upload centers in Switzerland, Japan and India, as well as our US center, it’s easier for people around the world to import large data sets by mailing hard drives to us rather than sending hundreds of terabytes over their slow, expensive or unreliable Internet connection.
But importing large amounts of data at scale isn’t simple. Our engineers have been working on the challenge for years. Originally, offline disk import was handled at our
data centers
as a way to efficiently import large amounts of data from the hard drives in our
Street View
cars - vehicles that capture terabytes of photographs and information about the landscape as they build a navigable, visual database of the world.
And although it’s technically challenging, the system we built for rapidly ingesting and processing these large data sources has a playful name. We call it
OmNomNom
. Here is one of the test OmNomNom machines that the disk import team has at its offices in Mountain View:
But rapidly ingesting and processing large data sets and making them usefully available is a bit more complex than gobbling down a cookie. As we’ve improved our ability to quickly import these drives, we dramatically reduced the time between capturing these images and making them available to users around the world.
Now, we are helping people across the world take advantage of the speed, scale and global availability of Google Cloud Storage as well as this rapid disk-upload technology. Even though it might sound like something out of Sesame Street, this is another example of how Google Cloud Platform is making the advantages of Google-sized scalable infrastructure available to you. All you need to do is send us your EncFS encrypted hard drives, and we will let you know once your encrypted bytes are imported to your designated GCS bucket. Once uploaded, we can mail your drives back to you, or if you prefer, safely and securely handle disk destruction free of charge. Check out our website for how you can be part of the
Limited Preview of International Offline Disk Import
.
(Oh, and for those of you who want to use Google Cloud Storage but don’t need offline disk import, you always have quick access Google Cloud Storage from the command line using
gsutil
).
-Posted by Benjamin Bechtolsheim, Product Marketing Manager
No comments :
Post a Comment
Don't Miss Next '17
Use promo code NEXT1720 to save $300 off general admission
REGISTER NOW
Free Trial
GCP Blogs
Big Data & Machine Learning
Kubernetes
GCP Japan Blog
Labels
Announcements
56
Big Data & Machine Learning
91
Compute
156
Containers & Kubernetes
36
CRE
7
Customers
90
Developer Tools & Insights
80
Events
34
Infrastructure
24
Management Tools
39
Networking
18
Open Source
105
Partners
63
Pricing
24
Security & Identity
23
Solutions
16
Stackdriver
19
Storage & Databases
111
Weekly Roundups
16
Archive
2017
Feb
Jan
2016
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2015
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2014
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2013
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2012
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2011
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2010
Dec
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2009
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Feed
Subscribe by email
Technical questions? Check us out on
Stack Overflow
.
Subscribe to
our monthly newsletter
.
Google
on
Follow @googlecloud
Follow
Follow
Follow
No comments :
Post a Comment