Starting on July 21, Google Cloud Platform developers and engineers are visiting 11 cities across North America, providing a technical, demo-based introduction to Google Cloud Platform.



In addition to this technical introduction, the roadshow provides a great opportunity for you to meet the Googlers who build the services you know and love.Read More

Starting on July 21, Google Cloud Platform developers and engineers are visiting 11 cities across North America, providing a technical, demo-based introduction to Google Cloud Platform.



In addition to this technical introduction, the roadshow provides a great opportunity for you to meet the Googlers who build the services you know and love.



We will be talking about how Cloud Platform is offering new ways for you to run applications in the cloud, including supporting new models of cloud computing such as Managed VMs that blur the lines between PaaS & IaaS. We’ll demonstrate new features and use cases of our Big Data platform.



We’re also going to show you a case study of how we built a world-class mobile application integrating Cloud Platform services with load balancing and replica pools.



The full list of dates is below; click the links below to join us at a roadshow near you:

Los Angeles	July 21
San Francisco	July 23
Seattle	July 30
Vancouver	July 31
New York City	August 5
Cambridge	August 7
Toronto	August 12
Boulder	August 12
Austin	August 14
Atlanta	August 19
Chicago	August 22

- Posted by Greg DeMichillie, Director of Product Management

Many mobile apps today suffer from “app-nesia” — the affliction that causes an app to forget who you are. Have you ever re-installed an app only to discover you have to re-create all your carefully crafted preferences? This is typically because the user’s app data lives only on the device.Read More

Many mobile apps today suffer from “app-nesia” — the affliction that causes an app to forget who you are. Have you ever re-installed an app only to discover you have to re-create all your carefully crafted preferences? This is typically because the user’s app data lives only on the device.



By connecting your apps to a backend platform, you can solve this issue, but it can be challenging. Whether it’s building basic plumbing, or just trying to load and save data in a network & battery-efficient way, spending time dealing with the backend can take precious time away from building an awesome app. So, we’re introducing two new features to help make your life easier.



Google Cloud Save

Google Cloud Save allows you to easily load and save user data to the cloud without needing to code up the backend. This is handy for situations where you want to save user state and have that state synchronized to multiple devices, or survive an app reinstall.



We handle all the backend logic as well as the synchronization services on the client. The synchronization services work in the background, providing offline support for the data, and minimizing impact on the battery. All you need to do is tell us when and what to save, and you do this with just 4 simple methods:

• .save(client, List<Entity>)


• .delete(client, Query)


• .query(client, Query)


• .requestSync(client)

All data is written locally first, then automatically synchronized in the background. The save, delete and query methods provide your basic CRUD operations while the requestSync method allows you to force a synchronization at any time.



On the backend the data is stored in Google Cloud Datastore which means you can access the raw data directly from a Google App Engine or Google Compute Engine instance using the existing Datastore API. Changes on the server will even be automatically synced back to client devices.

Importantly, this per-user data belongs to you, the developer, and stored in your own Google Cloud Datastore database.



Google Cloud Save is currently in private beta and will be available for general use soon. If you’re interested in participating in the private beta, you can sign up here!



Cloud Tools for Android Studio

To simplify the process of adding an App Engine backend to your app, Android Studio now provides three App Engine backend module templates which you can add to your app:

• App Engine Java Servlet Module - Minimal Backend


• App Engine Java Endpoints Module - Basic Endpoint scaffolding 


• App Engine with Google Cloud Messaging - Push notification wireup 

When you choose one of these template types your project is updated with a new Gradle module containing your new App Engine backend. All of the required dependencies/permissions will be automatically set up for you.



Built-in rich editing support for Google Cloud Endpoints

Once you have added the backend module to your Android application, you can use Google Cloud Endpoints to streamline the communication between your backend and your Android app. Cloud Endpoints automatically generates strongly-typed, mobile optimized client libraries from simple Java server-side API annotations, automates Java object marshalling to and from JSON, and provides built-in OAuth 2.0 support.



On deployment, this annotated Endpoints API definition class generates a RESTful API. You can explore this generated API (and even make calls to it) by navigating to Endpoints API explorer as shown in the image below:



To simplify calling this generated API from your Android app, Android Studio will automatically set up your project to include all compile dependencies and permissions required to consume Cloud Endpoints, and will re-generate strongly-typed client libraries if your backend changes. This means that you can start calling the client libraries from your Android app immediately after defining the server-side Endpoints API.



The underlying work-horses: Gradle, and Gradle plug-in for App Engine

Under the hood, Gradle is used to build both your app and your App Engine backend. In fact, when you add an App Engine backend to your Android app, the open-source App Engine plug-in for Gradle is automatically downloaded by Android Studio, and common App Engine tasks become available as Gradle targets. This allows you to use the same build system across your IDE, command-line or continuous integration environments.



Checkout more details on the new Cloud Endpoints features in Android Studio on the Android Developer Blog.



-Posted by Jason Polites, Product Manager

In today's world, information is being generated at an incredible rate. However, unlocking insights from large datasets can be cumbersome and costly, even for experts.



It doesn’t have to be that way. Yesterday, at ...Read More

In today's world, information is being generated at an incredible rate. However, unlocking insights from large datasets can be cumbersome and costly, even for experts.



It doesn’t have to be that way. Yesterday, at Google I/O, you got a sneak peek of Google Cloud Dataflow, the latest step in our effort to make data and analytics accessible to everyone. You can use Cloud Dataflow:

• for data integration and preparation (e.g. in preparation for interactive SQL in BigQuery)


• to examine a real-time stream of events for significant patterns and activities


• to implement advanced, multi-step processing pipelines to extract deep insight from datasets of any size

In these cases and many others, you use Cloud Dataflow’s data-centric model to easily express your data processing pipeline, monitor its execution, and get actionable insights from your data, free from the burden of deploying clusters, tuning configuration parameters, and optimizing resource usage. Just focus on your application, and leave the management, tuning, sweat and tears to Cloud Dataflow.



Cloud Dataflow is based on a highly efficient and popular model used internally at Google, which evolved from MapReduce and successor technologies like Flume and MillWheel. The underlying service is language-agnostic. Our first SDK is for Java, and allows you to write your entire pipeline in a single program using intuitive Cloud Dataflow constructs to express application semantics.



Cloud Dataflow represents all datasets, irrespective of size, uniformly via PCollections (“parallel collections”). A PCollection might be an in-memory collection, read from files on Cloud Storage, queried from a BigQuery table, read as a stream from a Pub/Sub topic, or calculated on demand by your custom code.



Because PCollections can be arbitrarily large, Cloud Dataflow includes a rich library of PTransforms (“parallel transforms”), which you can customize with your own application logic. For example, ParDo (“parallel do”) runs your code over each element in a PCollection independently (like both the Map and Reduce functions in MapReduce or WHERE in SQL), and GroupByKey takes a PCollection of key-value pairs and groups together all pairs with the same key (like the Shuffle step of MapReduce or GROUP BY and JOIN in SQL). In addition, anyone can define new custom transformations by composing other transformations -- this extensibility lets you write reusable building blocks which can be shared across programs. Cloud Dataflow provides a starter set of these composed transforms out of the box, including Count, Top, and Mean.



Writing in this modular, high-level style naturally leads to pipelines that make multiple logical passes over the same data. Cloud Dataflow automatically optimizes your data-centric pipeline code by collapsing multiple logical passes into a single execution pass. However, this doesn't turn the system into a black box: as you can see below, Cloud Dataflow’s monitoring UI uses the building block concept to show you the pipeline as you wrote it, not as the system chooses to execute it.



The same Cloud Dataflow pipeline may run in different ways, depending on the data sources. As you start designing or debugging, you can run against data local to your development environment. When you’re ready to scale up to real data, that same pipeline can run in parallel batch mode against data in Cloud Storage or in distributed real-time processing mode against data coming in via a Pub/Sub topic. This flexibility makes it trivial to transition between different stages in the application development lifecycle: to develop and test applications, to adapt an existing batch pipeline to track time-sensitive trends, or to fix a bug in a real-time pipeline and backfill the historical results.



When you use Cloud Dataflow, you can focus solely on your application logic and let us handle everything else. You should not have to choose between scalability, ease of management and a simple coding model. With Cloud Dataflow, you can have it all.



If you’d like to be notified of future updates about Cloud Dataflow, please join our Google Group.



-Posted by Frances Perry, Software Engineer

Whether it’s the next viral game, social sharing app or hit SaaS application, the velocity of your innovation is driven by the productivity of your dev team. This week at Google I/O we talked about several new tools that enable developers to understand, diagnose and improve their systems in production.Read More

Whether it’s the next viral game, social sharing app or hit SaaS application, the velocity of your innovation is driven by the productivity of your dev team. This week at Google I/O we talked about several new tools that enable developers to understand, diagnose and improve their systems in production.



Cloud Debugger

Today the state of the art of debugging for cloud applications isn’t much more than writing out diagnostic messages and spelunking the logs for them. When the right data is not being written to the logs, developers have to make a code change and redeploy the application to production. That is the last thing you want to do when investigating an issue in production. Traditional debuggers aren’t well suited for cloud-based services for two reasons. First, it is difficult to know which process to attach to. Second, stopping a process in production makes it hard to reproduce an issue and gives your end-users a bad experience.



The Cloud Debugger completely changes this model. It allows developers to start where they know best - in the code. By simply setting a watchpoint on a line of code, the next time a request on any of your servers hits that line of code, you get a snapshot of all the local variables, parameters, instance variables and a full stack trace. This works no matter how many instances you are running in production. There is zero setup up time and no complex configuration to enable. The debugger is ideal for use in production. There is no overhead for enabling the debugger on a project and when a watchpoint is hit very little noticeable performance impact is seen by your users.



Cloud Trace

Performance is an important feature of your service which directly correlates with end user satisfaction and retention. No one intends to build a slow service, but it can be extremely difficult to isolate the root cause of sluggishness when it happens. Especially when the issue hits only a fraction of your users.



Cloud Trace helps you visualize and understand the time spent by your application for request processing. This enables you to quickly identify and fix performance bottlenecks. You can even compare performance from release to release with a detailed report. You can leave Cloud Trace enabled in production because it has very little performance overhead.



In this screenshot, you can see we have investigated a particularly slow trace and we see a detailed breakdown of where the time is being spent. It looks like the problem could be these numerous sequential calls to Datastore, so maybe we should consider batching them.



So we go update our service to batch the Datastore calls, and deploy the updated service. Now we can use Cloud Trace to verify the fix.



As a developer, you can easily produce a report that shows the performance change in your service from one release to another. In the following report, the blue graph shows the performance without datastore batching and the orange graph shows the performance after releasing the change to use datastore batching. The X-axis of the graph represents the time taken (logarithmic scale) to service requests, and the left shift of the orange graph shows the significant performance gain due to Datastore batching.



Cloud Monitoring, Powered by Stackdriver

Cloud Monitoring provides rich dashboards and alerting capabilities that help developers find and fix performance problems quickly.



With minimal configuration and no separate infrastructure to maintain, Cloud Monitoring provides you with deep visibility into your Cloud Platform services. For example, you can use Cloud Monitoring dashboards to diagnose cases where your customers are reporting slow response times or errors accessing your applications:



Likewise, you can create alerting policies so that you are notified when key metrics, such as latency or error rates, pass a given threshold in the future:



You can configure alerts for any metric in the system, including those related to the performance of Cloud SQL databases, App Engine modules and versions, Pub/Sub topics and subscriptions, and Compute Engine VMs. With Compute Engine VMs, you can create alerts for both core system metrics (CPU, memory, etc.) and application services running in the VMs (Apache, Cassandra, MongoDB, etc.).



You can also create dashboards that make it easier to correlate metrics across services. For example, it takes a few clicks to create a dashboard that tracks key metrics for an App Engine module that connects to a set of Redis VMs running on Compute Engine:



Finally, you can create endpoint checks to monitor availability and response times for your end-user facing services. Endpoint checks are performed by probes in Oregon, Texas, Virginia, Amsterdam, and Singapore, enabling monitoring of latency from each of these five regions.



SSH to your VM instantly

Sometimes it is inevitable to connect directly to a VM to debug or fix a production issue. We know this can be a bit of a pain, especially when you are on the road, so now you can do that from just about anywhere. With our new browser based SSH client you can quickly and securely connect to any of your VMs from the Console. No need to install any SDK or tools. The best part is, this works from any desktop device with most major web browsers.



Ready for a Spin?

All of these features are just about ready for your applications. Stay tuned to this blog, we will post updates as they are more widely available.



-Posted by Brad Abrams, Group Product Manager

Today at Google I/O, we are introducing new services that help developers build and optimize data pipelines, create mobile applications, and debug, trace, and monitor their cloud applications in production.Read More

Today at Google I/O, we are introducing new services that help developers build and optimize data pipelines, create mobile applications, and debug, trace, and monitor their cloud applications in production.



Introducing Google Cloud Dataflow

A decade ago, Google invented MapReduce to process massive datasets using distributed computing. Since then, more devices and information require more capable analytics pipelines — though they are difficult to create and maintain.



Today at Google I/O, we are demonstrating Google Cloud Dataflow for the first time. Cloud Dataflow is a fully managed service for creating data pipelines that ingest, transform and analyze data in both batch and streaming modes. Cloud Dataflow is a successor to MapReduce, and is based on our internal technologies like Flume and MillWheel.



Cloud Dataflow makes it easy for you to get actionable insights from your data while lowering operational costs without the hassles of deploying, maintaining or scaling infrastructure. You can use Cloud Dataflow for use cases like ETL, batch data processing and streaming analytics, and it will automatically optimize, deploy and manage the code and resources required.



Debug, trace and monitor your application in production

We are also introducing several new Cloud Platform tools that let developers understand, diagnose and improve systems in production.



Google Cloud Monitoring is designed to help you find and fix unusual behavior across your application stack. Based on technology from our recent acquisition of Stackdriver, Cloud Monitoring provides rich metrics, dashboards and alerting for Cloud Platform, as well as more than a dozen popular open source apps, including Apache, Nginx, MongoDB, MySQL, Tomcat, IIS, Redis, Elasticsearch and more. For example, you can use Cloud Monitoring to identify and troubleshoot cases where users are experiencing increased error rates connecting from an App Engine module or slow query times from a Cassandra database with minimal configuration.



We know that it can be difficult to isolate the root cause of performance bottlenecks. Cloud Trace helps you visualize and understand time spent by your application for request processing. In addition, you can compare performance between various releases of your application using latency distributions.



Finally, we’re introducing Cloud Debugger, a new tool to help you debug your applications in production with effectively no performance overhead. Cloud Debugger gives you a full stack trace and snapshots of all local variables for any watchpoint that you set in your code while your application continues to run undisturbed in production. This brings modern debugging to cloud-based applications.



New features for mobile development

With rapid autoscaling, caching and other mobile friendly capabilities, many apps like Snapchat or Rising Star have built and run on Cloud Platform. We’re adding new features that make building a mobile app using Cloud Platform even better.



Today, we’re demonstrating a new version of Google Cloud Save, which gives you a simple API for saving, retrieving, and synchronizing user data to the cloud and across devices without needing to code up the backend. Data is stored in Google Cloud Datastore, making the data accessible from Google App Engine or Google Compute Engine using the existing Datastore API. Google Cloud Save is currently in private beta and will be available for general use soon.



We’ve also added tooling to Android Studio, which simplifies the process of adding an App Engine backend to your mobile app. In particular, Android Studio now has three built-in App Engine backend module templates, including Java Servlet, Java Endpoints and an App Engine backend with Google Cloud Messaging. Since this functionality is powered by the open-source App Engine plug-in for Gradle, you can use the same build configuration for both your app and your backend across IDE, CLI and Continuous Integration environments.



We’ll be doing more detailed follow-up posts about these announcements in the coming days, so stay tuned.



-Posted by Greg DeMichillie, Director of Product Management









*Apache, Nginx, MongoDB, MySQL, Tomcat, IIS, Redis, Elasticsearch and Cassandra are trademarks of their respective owners.

With over a dozen different virtual machine types, and the great price for performance of persistent disks, we think Google Compute Engine is a great place for running ...Read More

With over a dozen different virtual machine types, and the great price for performance of persistent disks, we think Google Compute Engine is a great place for running MongoDB. We are excited to be working with MongoDB to bring together documentation, resources, and partners to help you get up and running, to build, and to scale your application.



For self-managed production deployments, you have a range of software management options, such as Puppet, Chef, Salt, and Ansible. For fully-managed deployments, you can work with partners such as MongoLab.



But when you are just getting a project started, you want to get a proving ground up and running quickly. You'd also like to to find the documentation you need from one place. You'll be able to find resources you need to deploy MongoDB on Compute Engine at http://cloud.google.com/solutions/mongodb.



With Click to Deploy MongoDB available in the Developer Console, you'll be able to bring up a Compute Engine cluster running MongoDB in a few minutes. Through the web interface, you can choose how many MongoDB server and arbiter nodes, their associated virtual machine types, and the size of your data disk volumes.



We'll continue to add features to the Click to Deploy infrastructure and look to implement best practices for MongoDB on Google Compute Engine. Here's one key tip you can find in the MongoDB on Google Compute Engine solutions paper:

Put your MongoDB journal files and data files on the same disk 

It is a common recommendation to separate components onto different storage devices. However, persistent disk already stripes data across a very large number of volumes, so there is no need to do it yourself. 

MongoDB journal data is small and putting it on its own disk means either creating a small disk with insufficient performance or creating a large disk that goes mostly unused. Put your MongoDB journal files on the same disk as your data. Otherwise putting your MongoDB journal files on a small persistent disk will dramatically decrease performance of database writes.

Check it all out at http://cloud.google.com/solutions/mongodb.



-Posted by Matt Bookman, Solutions Architect



MongoDB, is registered trademarks of MongoDB, Inc. All other trademarks cited here are the property of their respective owners.

If you develop scalable applications, you often want to use a messaging system, but you may be concerned that setting up such a system is time consuming and its throughput limits are not sufficient for your growing needs. But what if the whole setup required just a few minutes and benchmarks showed that the system was capable of processing over one million messages per second? This is exactly what we are announcing today with Pivotal.Read More

If you develop scalable applications, you often want to use a messaging system, but you may be concerned that setting up such a system is time consuming and its throughput limits are not sufficient for your growing needs. But what if the whole setup required just a few minutes and benchmarks showed that the system was capable of processing over one million messages per second? This is exactly what we are announcing today with Pivotal.



When we talked about high-throughput, low-latency messaging scenarios with our customers, they were particularly interested in RabbitMQ, a popular open source messaging system that serves a variety of use cases.



We recently made it super easy for our users to deploy a dedicated RabbitMQ cluster on Google Compute Engine. All it takes is one click after providing some basic information, such as the size of the cluster and RabbitMQ username / password.



In a matter of a few minutes, you can get a RabbitMQ cluster deployed and configured. You will also get two load balancers set up for your cluster. As a next step you can securely access RabbitMQ web management console and start developing your app that uses RabbitMQ. Even if you need a very large 64-node cluster, the deployment usually takes less than 10 minutes.



In fact, the RabbitMQ cluster is so easy to deploy and configure that we’ve received feedback that for some developers the experience may be similar to having their own "Rabbit service" consisting of multiple nodes.



We also wanted to demonstrate that a RabbitMQ cluster running on Google Compute Engine is a good fit for high-throughput and low-latency scenarios. To do so, we ran a series of benchmarks in various configurations. These test clusters were able to sustain a throughput of over 1 million messages published and consumed per second (a sustained combined ingress/egress of over two million messages per second).



To put this throughput in context, one million messages per second translates to 86 billion messages per day. U.S. text messages reached 6 billion per day in 2012. Apple processes about 40 billion iMessages per day, and WhatsApp recently hit a new daily record in December when it sent 20 billion daily messages.



Google and Pivotal Benchmarks

We then invited engineers from Pivotal, the company behind RabbitMQ, to validate and endorse our results.



For our joint tests on Google Compute Engine we provisioned a cluster of 32 virtual machines with 8 vCPUs and 30GB of RAM each, and deployed RabbitMQ on Debian.



To generate load we used RabbitMQ PerfTest client tool running on a set of GCE virtual machines separate from the ones housing the RabbitMQ cluster nodes. As per RabbitMQ clustering recommendations the clients were configured to target the IP address of a GCE load balancer that was automatically created through click to deploy. This is a more flexible approach than having the clients know the IP addresses or host names of every node in the Rabbit cluster.



After the traffic generating clients warmed up, we reached a steady state as shown in RabbitMQ’s web management console: over 1.3 million messages published and consumed per second.



Under multiple hours of such load, the RabbitMQ cluster and its underlying GCE VMs remained stable, with average throughput staying at approximately the same levels as shown in the screenshot.



Conclusion

We are excited to offer our customers an easy way to deploy RabbitMQ. We are proud that both Google and Pivotal benchmarks validate that Compute Engine is an excellent choice for running RabbitMQ in a public cloud



To deploy your own dedicated RabbitMQ cluster, go to Google Developer Console and navigate to an existing project, or create a new one, and you will have an option to deploy RabbitMQ from the project dashboard:



You can also find a link at the bottom of the project landing page:



There is no extra charge for using click to deploy - you will be only billed for the underlying Compute Engine resources, so why not deploy your cluster today and tell us what you think.



-Posted by Grzegorz Gogolowicz, Cloud Solutions Architect



P.S. If you don’t want to have your own dedicated RabbitMQ cluster deployed as described in this post, and rather would like to use a managed RabbitMQ, you may want to check out CloudAMQP offering that runs on Google Compute Engine.

Our customers tell us that importing multi-terabyte datasets, such as log files and media libraries, from other cloud-based 3rd party storage providers can be slow and expensive. Today, we’re introducing a limited preview of Online Cloud Import for Google Cloud Storage which makes it faster, easier and cheaper to import online data to your Cloud Storage buckets (standard or regional in US or Europe) using Google’s high performance network. Sign up for the limited preview ...Read More

Our customers tell us that importing multi-terabyte datasets, such as log files and media libraries, from other cloud-based 3rd party storage providers can be slow and expensive. Today, we’re introducing a limited preview of Online Cloud Import for Google Cloud Storage which makes it faster, easier and cheaper to import online data to your Cloud Storage buckets (standard or regional in US or Europe) using Google’s high performance network. Sign up for the limited preview here.



Online Cloud Import provides a number of options that makes data transfers and synchronization easier. Not only can you create a one-time backup/transfer of your data to your Cloud Storage bucket, but you can also set a periodic scheduled synchronization of your 3rd party storage to Google Cloud Storage. Additionally, you can configure your synchronization using a set of advanced filters based on file creation dates, file-name filters, and the times of day you prefer to import data. Once your data is synchronized, you will receive an email notification with a summary of the entire process.



Online Cloud Import complements both gsutil and Cloud Storage Offline Disk Import. Cloud Storage Offline Disk Import is suitable if you have hundreds of terabytes of offline data -- disk-upload centers are available in Switzerland, Japan, India and the United States. If you'd like command-line control, and are dealing with less than 10TB, we suggest using gsutil (now with rsync) to transfer your data.



We’re excited to offer another method of importing data into Cloud Storage, and we invite you to sign up for Online Cloud Import and let us know what you think.



-Posted by Lamia Youseff, Product Manager

Earlier this week, we announced the limited preview of SSD persistent disk. SSD persistent disk gives you the same great features as Standard persistent disk but is backed by SSD so you get much faster IO performance—up to 30 IOPS per GB. On a per GB basis, this is 20x more write IOPS and 100x more read IOPS than Standard PD! Also, while performance on Standard persistent disk is generally very consistent, SSD persistent disk is designed to perform even more consistently; it is designed to provide 30 IOPS per GB if you follow ...Read More

Earlier this week, we announced the limited preview of SSD persistent disk. SSD persistent disk gives you the same great features as Standard persistent disk but is backed by SSD so you get much faster IO performance—up to 30 IOPS per GB. On a per GB basis, this is 20x more write IOPS and 100x more read IOPS than Standard PD! Also, while performance on Standard persistent disk is generally very consistent, SSD persistent disk is designed to perform even more consistently; it is designed to provide 30 IOPS per GB if you follow best practices.



SSD persistent disk costs only $0.325 per GB per month. Like Standard persistent disk, there is no additional cost for I/O. With SSD persistent disk, there is no need to plan for IOPS or to balance cost concerns with performance goals.



SSD persistent disk has two major benefits for customers:

1. You can run your databases and file servers on Google Cloud Platform faster than ever before and move ever more demanding applications into Google Cloud Platform.


2. You now have a simple and cost effective solution for handling small amounts of data that need high IO. When your application is I/O constrained, SSD persistent disk gives you up to 59% lower costs for read-heavy workloads, and up to 92% lower costs for write-heavy workloads.

SSD persistent disk can help build faster and more cost effective cloud applications. "Our data suggests that storage represents 20% of overall cloud spend and up to 70% of the costs of a heavily loaded database tier,” said Sebastian Stadil, CEO of Scalr. “The extreme performance offered by the new SSD PDs, when coupled with their very low price and absence of fees per IO consumed, have made us accelerate our plans to move our workloads to Google Cloud Platform."



Please note the following details about using SSD persistent disk:

1. This storage offering is targeted at highly transactional systems. Your MySQL, PostgreSQL, Mongo, Cassandra and Redis should work incredibly fast. But, this is not targeted at streaming large files sequentially. Use Standard persistent disk for bulk data or data that is read and written to in sequential streams and High IOPS PD for your heavily transactional storage with more random access.


2. High IOPS counts 16KB IOs and lower as 1 IO. Larger IO sizes will count as multiple IOs. For example, a 128KB IO will count as 8 IOs.


3. For larger volumes, VM IO capabilities will limit how much IO can be expected from the volume. VM limits today for the larger VMs are 10,000 read IOPS, 15,000 write IOPS, 180 MB/s read and 120 MB/s writes. These are numbers we are working hard to increase as we further optimize our infrastructure. Stay tuned for more here.

To learn more about SSD persistent disk, please read the documentation. If you’d like access to the limited preview, please request access here. As always, please keep in touch and give us feedback on how things are working either through our technical support, the Googe Compute Engine Discussion mailing list or the Google Compute Engine Stack Overflow forum.



-Posted by Jay Judkowitz, Senior Product Manager

Editor’s note: This post is a follow-up to the announcements we made on 6/16/2014.



A powerful load balancing solution can improve your application’s response to the dynamic nature of user traffic. In fact, last year we demonstrated how you can ...Read More

Editor’s note: This post is a follow-up to the announcements we made on 6/16/2014.



A powerful load balancing solution can improve your application’s response to the dynamic nature of user traffic. In fact, last year we demonstrated how you can serve 1 million requests per second instantaneously and affordably using Google Cloud Platform Network load balancing within a cloud region.



If you have a global user base, however, server utilization, availability, and performance can vary across regions. Traditionally developers have addressed this issue by configuring DNS-based load balancing, but that can be complex to maintain, inaccurate, and slow to respond to changes in load. We have developed a better way to load balance.



On Monday, we announced Cloud Platform HTTP Load Balancing, the world’s most scalable offering for load balancing traffic over groups of compute instances across multiple regions. It uses network proximity and backend capacity information to optimize the path between your users and your instances, and improves latency by connecting users to the closest Cloud Platform location. If your instances in one region are under heavy load or become unreachable, HTTP load balancing intelligently directs new requests to your available instances in a nearby region. Because your entire app is exposed via a single global external IP address, you can avoid the complexity and problems associated with configuring DNS-based load balancing; all you need to do is add a DNS record pointing to the global IP address.



HTTP load balancing is built on top of the same world-class infrastructure that powers Google’s own services such as Search, Gmail, and Youtube. Like Network load balancing, it uses the global Google network and doesn’t require pre-warming. Furthermore, HTTP load balancing filters out TCP SYN flood attacks. You can also route user requests to different backend groups based on host and URL path prefix.



Finally, using Google’s HTTP load balancing can help reduce costs. With cross-region support you only need to pay for one load balancing forwarding rule instead of one per region for your app. We're also simplifying pricing by offering unified, region-free rates:



HTTP load balancing is in limited preview. You can read more about it in our documentation and sign up to be one of the first to try it now. Network load balancing remains generally available.



Happy load balancing!



-Posted by Gary Ling, Product Manager

With Google I/O right around the corner, Google Cloud Platform is launching two innovations that will change the way developers use cloud resources and how they interact with their customers around the world. We are excited to announce both of these products today: HTTP Load Balancing and SSD-based Persistent Disk. Both products are examples of what Google Cloud Platform does best: performance and versatility on a global scale, with simple and competitive pricing.Read More

With Google I/O right around the corner, Google Cloud Platform is launching two innovations that will change the way developers use cloud resources and how they interact with their customers around the world. We are excited to announce both of these products today: HTTP Load Balancing and SSD-based Persistent Disk. Both products are examples of what Google Cloud Platform does best: performance and versatility on a global scale, with simple and competitive pricing.



HTTP Load Balancing

HTTP Load Balancing changes the game for how developers interact with network resources. HTTP load balancing can easily scale to support more than 1 million requests per second with no “warm up.” It also supports content-based routing and allows you to capture the benefits of Google’s global network infrastructure. More importantly, and for the first time, users can take advantage of load balancing across different regions -- balancing traffic among compute resources that are located in different data centres in different parts of the world. This creates a truly global service offering and lets our customers optimize their compute resources and reduce latency on a global scale. Google HTTP Load Balancing does this through a single external IP address, which also adds simplicity to network operations.



SSD Persistent Disk

With Persistent Disk, more and more of our customers have been asking for a solution to high IOPS use cases - and one that doesn’t break the bank. Our SSD persistent disk product sets a new bar for scalability and performance in Block Storage. Priced at flat $0.325 monthly cost per GB, SSD persistent disk supports up to 30 IOPS per GB. And while other providers count each and every IOPS and charge extra for them, SSD persistent disk includes IOPS in the base price with no extra charges or fees, making cost completely predictable and easy to model.



Both HTTP Load Balancing and SSD persistent disk are now in limited preview. Read more about SSD persistent disk or sign up for the Limited Preview, and do the same for HTTP load balancing.



-Posted by Tom Kershaw, Product Management Lead

Editor’s note: Today's guest post is from Daniel Viveiros, Head of Technology at CI&T, a Google Cloud Platform Partner of the Year LATAM 2013. In this post, Daniel describes how CI&T in partnership with Coca-Cola built the ‘Happiness Flag’ for the Coca-Cola 2014 FIFA World Cup™ campaign in Brazil. To learn more about the Happiness flag visit this website. ...Read More

Editor’s note: Today's guest post is from Daniel Viveiros, Head of Technology at CI&T, a Google Cloud Platform Partner of the Year LATAM 2013. In this post, Daniel describes how CI&T in partnership with Coca-Cola built the ‘Happiness Flag’ for the Coca-Cola 2014 FIFA World Cup™ campaign in Brazil. To learn more about the Happiness flag visit this website.



As part of the ‘The World’s Cup’ campaign, Coca-Cola wanted to do something that would visually illustrate soccer’s global reach. Coca-Cola invited fans around the world to share their photos to create the Happiness Flag -- the world’s largest mosaic flag crafted from thousands of crowdsourced images submitted by people in more than 200 countries. The flag, 3,015 square meters in size, was unveiled during the opening ceremony of the 2014 FIFA World Cup™.





A project of this scale calls for high performing and reliable technology, so when we started working with Coca-Cola to build the infrastructure for the Happiness Flag campaign, we knew we had to use Google Cloud Platform. By using Google Cloud Platform, we turned a big, innovative idea into reality on a global scale.



To create the Happiness Flag, we leveraged the whole Google Cloud Platform stack as shown below:



Google App Engine enabled us to handle the computing workload, capable of handling millions of images via Twitter, Facebook, Instagram and email, to the searches for images and view requests. The architecture was scalable to meet this kind of transaction demand and the fluctuations in traffic. We stored all the images in Google Cloud Storage, where integrated edge caching support and image services made it an ideal choice for serving the images. Meanwhile, Google Compute Engine gave us the capability for long-running processes, such as the Twitter integration and advanced image transformations. We were able to show how powerful the creation of hybrid environments can be, using both Platform-as-a-Service (Google App Engine) and raw virtual machines (Google Compute Engine) in the cloud.



We used other out-of-the-box Google Cloud Platform technologies like Memcache, Datastore and Task Queues to ensure outstanding levels of performance and scalability. We know that many fans will be viewing the Happiness Flag on their mobile devices, so we needed a platform that would offer different capacities of computational power. The system provides amazing user experience with high performance and low latency, regardless of the device and its location. Using Google Cloud Platform, the campaign runs smoothly 24/7 and includes redundancy, failover techniques, backups and state-of-the-art monitoring. Plus, it’s affordable.



After the physical flag was unveiled before the opening match, the digital mosaic was made available with a Google map-like zoom in and out with eleven levels of detail. Anyone who submitted an image can now search for themselves on the virtual flag and the search results will show up as pins in the mosaic, like locations found in a Google map. By clicking on the pin, their photos open up in an overlay and they are taken to the maximum level of zoom in to see the "neighborhood" around their image in the flag. After the match, a link to the Happiness Flag site was sent to each participant as a souvenir.



Our goal was to help Coca-Cola create a project that would celebrate the 2014 FIFA World Cup™ by enabling fans from all over the world to express their creativity in a show of unity and art. What better way to open the games than by displaying the Happiness Flag, which is a symbol of the spirit of the game and its fans.



-Posted by Daniel Viveiros, Head of Technology at CI&T

A couple of months ago, we shared some tips and tricks about bringing the power of Google Cloud Platform to your command-line using Google Cloud SDK and gcloud. Today we are announcing that ...Read More

A couple of months ago, we shared some tips and tricks about bringing the power of Google Cloud Platform to your command-line using Google Cloud SDK and gcloud. Today we are announcing that gcloud family of command-line tools is being joined by gcloud compute, a new command-line interface for Google Compute Engine.



Here are some guiding principles that we focused on while developing this new command-line tool. We certainly hope it will help you to be more productive with Google Compute Engine!



Great Windows support out-of-the-box

If you’re running on Windows, download and launch our new Google Cloud SDK installer. Once the installer completes, you should have gcloud compute available on your system.



Using gcloud compute you can easily connect to your VM instances, manage files and running processes on native Windows installations, as well as on Linux and Mac machines without any extra effort. For example, to SSH into your VM, simply run gcloud compute ssh INSTANCE_NAME --zone=us-central1-a.





To execute a process remotely on your VM, add “--command“ flag, e.g. gcloud compute ssh INSTANCE_NAME --command="ps aux" --zone=us-central1-a.



To copy files to and from your instances, use copy-files command:

gcloud compute copy-files instance-1:file-1 instance-2:file-2 ... local-directory --zone=us-central1-a



gcloud compute copy-files file-1 file-2 ... instance:remote-directory --zone=us-central1-a



Finally, if you want to use SSH-based programs, like ssh or scp directly, run gcloud compute config-ssh, which will populate your per-user SSH configuration file with “Host” entries from each instance.



Improved scripting support

You can easily combine individual commands into actions which would require tens of button clicks in the graphical interface, or tens of lines when programming directly against APIs. Here is a simple example.



Run the following command to list all of your virtual machine instances:

gcloud compute instances list



Similarly, to delete a particular VM instance (say named my-old-vm), run the following command:

gcloud compute instances delete my-old-vm



Now, combine the previous two commands to delete all your VMs in us-central1-b zone:

gcloud compute instances delete $(gcloud compute instances list --zone=us-central1-b)



Integrated documentation and command-line help

If you want to read a summary of a particular command’s usage, simply append -h to any command (for example, gcloud compute instances list -h). Similarly, append --help to any command to see its detailed man page entry on Linux or Mac. The detailed command-line reference for all platforms is also available on Cloud SDK’s documentation page.



Flexible output formatting

The gcloud compute instances list example above prints detailed information about VM instances in YAML format by default. If you want to get the same data in JSON or in condensed, human-readable text formats (for example, to send it over the wire or to import it in a spreadsheet), simply pass the “--format=json“ or “--format=text“ flag to any list operation in gcloud compute:



gcloud compute firewall-rules list --format=text or

gcloud compute zones list --format=json



Furthermore, you could use regular expressions via --regexp to get information only about objects with specific names. For example, to return information about VM instances with names my-instance-1, my-instance-2, my-instance-3, run the following command:

gcloud compute instances list --regexp my-instance-.*



Finally, if you care just about certain fields in the output, you can select them by passing “--fields“ flag. For example, to see the disks attached to all your instances and their modes, run:

gcloud compute instances list --fields name disks[].source disks[].mode



Command suggestion and autocompletion

While gcloud command-line tools are great for scripting, they’re also comfortable for humans to use. On Linux and Mac, all gcloud commands can be autocompleted by pressing (e.g. type gcloud compute fi and press to have it expanded to gcloud compute firewall-rules). Similarly, you can type gcloud compute firewall-rules and press twice to see the operations which you can perform with firewall-rules. Also, if you mistype a command, gcloud will suggest a nearest match.

Today’s guest blog comes from Eli Uzan, CEO of Screenz. Screenz joined forces with Google to revolutionize the TV viewing experience. Their Real Time Platform is the state-of-the-art technology powering the upcoming ABC television show Rising Star. ...Read More

Today’s guest blog comes from Eli Uzan, CEO of Screenz. Screenz joined forces with Google to revolutionize the TV viewing experience. Their Real Time Platform is the state-of-the-art technology powering the upcoming ABC television show Rising Star.



Our goal is to revolutionize television viewing by shifting ordinary programming to live interactive events. Our Real Time Platform, using Google Cloud Platform, enables broadcasters to do just that. The first show to use our Real Time Platform is ‘Rising Star’ – Keshet International’s hit TV show that will be launching in more than 25 countries including on ABC in the US on June 22nd. When the ‘Rising Star’ format was first developed with Keshet, our joint objective was to create a live show that dramatizes each performance and keeps viewers engaged each and every moment of the show.



‘Rising Star’ marks a new era in home entertainment. The show enables real-time voting by viewers via a mobile app that is fully integrated into the television program. It is the first talent show where the viewer is the judge from start to finish, and they can see the results of their vote in real time. The mobile screen element enables audiences to have instant gratification, which is paramount to the format’s appeal.



The Real Time Platform also has many benefits for TV broadcasters. It increases the number of viewers engaging with the show, the social buzz, and pushes up audience numbers at time of the live broadcast. It also enables sophisticated audience targeting, allowing broadcasters and brands to effectively identify viewer demographics and interests.



Our collaboration with Google Cloud Platform successfully delivered the scale to process the huge numbers of votes per second required for Rising Star. Our system uses various Cloud Platform components, including Compute Engine, Cloud Storage and BigQuery. We chose Google because we needed massive scale. In testing, we were able to process over 100 million interactions in one minute. Cloud Platform scales exceptionally well across all layers of the service, and allows us to provide a truly global platform.



By working together with Google Cloud Platform, Screenz has enabled broadcasters and producers to throw out the rulebook and combine compelling live content with real time contribution by viewers.



-Contributed by Eli Uzan, CEO, Screenz

Today’s guest blog is by Autism Speaks Chief Science Officer Robert Ring. As the world’s largest autism science and advocacy organization, Autism Speaks has committed more than $500 million to its mission, the majority in science and medical research. ...Read More

Today’s guest blog is by Autism Speaks Chief Science Officer Robert Ring. As the world’s largest autism science and advocacy organization, Autism Speaks has committed more than $500 million to its mission, the majority in science and medical research.



An estimated 1 in 68 children in the U.S. is on the autism spectrum. Caused by a combination of genetic and environmental influences, autism is characterized, in varying degrees, by deficits in social communication and interaction, along with the presence of repetitive patterns of behavior, interests or activities. Many individuals with autism also face a lifetime of associated medical conditions (e.g. anxiety, sleep problems, seizures and/or GI symptoms) that frequently contribute to poor outcomes.



With the participation of our amazing autism community, Autism Speaks has worked for 15 years to assembled the largest open-access collections of DNA samples from families affected by autism. Our Autism Genetic Resource Exchange (AGRE) holds the DNA of 12,000 individuals affected by autism and their parents and siblings as well as information on the autism symptoms and autism-related medical conditions of these individuals.



Building on AGRE, Autism Speaks launched the AUT10K program in collaboration with the University of Toronto’s Hospital for Sick Children’s Centre for Applied Genomics. AUT10K has already completed the sequencing of 1,000 cases, and currently has close to 2,000 additional samples nearing completion.



From the beginning, we realized that the amount of data collected by AUT10K would create many challenges. We needed to find a way to store and analyze massive data sets, while allowing remote access to this unprecedented resource for autism researchers around the world.



In the beginning, we shared genomic information by shipping hard drives around the world. Downloading even one individual’s whole genome in a conventional manner can take hours – the equivalent of downloading a hundred feature films. And by the time AUT10K achieves its milestone of 10,000 genomes, we knew we’d have a database on the petabyte scale.



Now, Autism Speaks is using Google Cloud Platform to store its data and enable real-time, collaborative access among researchers around the world. We are in the process of uploading 100 terabytes of data to Google Cloud Storage, and from there, we can import it into Google Genomics. Google Genomics will allow scientists to access the data via the Genomics API, explore it interactively using Google BigQuery, and perform custom analysis using Google Compute Engine.



Researchers will spend less time moving data around and more time analyzing data and collaborating with colleagues. We hope this will enable us to make discoveries and drive innovation faster than ever.



The insight and expertise the Google team has already brought to the table has been unmatched. Our work with them has been a game-changer for AUT10k. Together, we hold the capability of accelerating breakthroughs in understanding the causes and subtypes of autism in ways that can advance diagnosis and treatment as never before.



- Contributed By Robert Ring, Chief Science Officer, Autism Speaks

Everything at Google, from Search to Gmail, is packaged and run in a Linux container. Each week we launch more than 2 billion container instances across our global data centers, and the power of containers has enabled both more reliable services and higher, more-efficient scalability. Now we’re taking another step toward making those capabilities available to developers everywhere.Read More

Everything at Google, from Search to Gmail, is packaged and run in a Linux container. Each week we launch more than 2 billion container instances across our global data centers, and the power of containers has enabled both more reliable services and higher, more-efficient scalability. Now we’re taking another step toward making those capabilities available to developers everywhere.



Support for Docker images in Google App Engine

Last month we released improved Docker image support in Compute Engine. Today, we’re building on that work and adding a set of extensions that allow App Engine developers to build and deploy Docker images in Managed VMs. Developers can use these extensions to easily access the large and growing library of Docker images, and the Docker community can easily deploy containers into a completely managed environment with access to services such as Cloud Datastore. If you want to try it, sign up via this form.



Kubernetes—an open source container manager

Based on our experience running Linux containers within Google, we know how important it is to be able to efficiently schedule containers at Internet scale. We use Omega within Google, but many developers have more modest needs. To that end, we’re announcing Kubernetes, a lean yet powerful open-source container manager that deploys containers into a fleet of machines, provides health management and replication capabilities, and makes it easy for containers to connect to one another and the outside world. (For the curious, Kubernetes (koo-ber-nay'-tace) is Greek for “helmsman” of a ship.)



Kubernetes was developed from the outset to be an extensible, community-supported project. Take a look at the source and documentation on GitHub and let us know what you think via our mailing list. We’ll continue to build out the feature set, while collaborating with the Docker community to incorporate the best ideas from Kubernetes into Docker.



Container stack improvements

We’ve released an open-source tool called cAdvisor that enables fine-grain statistics on resource usage for containers. It tracks both instantaneous and historical stats for a wide variety of resources, handles nested containers, and supports both LMCTFY and Docker’s libcontainer. It’s written in Go with the hope that we can move some of these tools into libcontainer directly if people find them useful (as we have).



A commitment to open container standards

Finally, I'm happy that I've been nominated to Docker's Governance Committee to continue working with the Docker community toward better open container standards. Containers have been a great building block for Google and by working together we can make them the key building block for “cloud native” applications.



-Posted by Eric Brewer, VP of Infrastructure

Cloud usage monitoring and analysis has served an important role in helping users optimize their cloud services. However, many users have expressed a need to access more granular information about their ...Read More

Cloud usage monitoring and analysis has served an important role in helping users optimize their cloud services. However, many users have expressed a need to access more granular information about their Google Compute Engine usage. Today, we are making this possible with Compute Engine Usage Export, which enables you to easily export detailed reports about your usage data Usage Export makes it easy to obtain more granular insight into your Compute Engine usage at a resource level. For example, you can monitor exactly how long a virtual machine has been running or how much storage space a persistent disk uses on a daily basis.



Similar to Billing Export, Compute Engine Usage Export allows you to export a CSV file with the detailed usage data to a Google Cloud Storage bucket you specify. This CSV file can then be accessed through the Cloud Storage API, CLI tool, or Console.



The Usage Export feature will provide both daily reports that include usage data from the latest 24 hour period and monthly rollup reports that include monthly usage data up to the most current day.



This feature complements Billing Export to provide you with basic tools for monitoring, analyzing, and optimizing cost. You can link items between the two exports to easily identify charges for a particular resource.



Compute Engine Usage Export can be enabled from the Console as shown below.



The CSV files will be placed in a Cloud Storage bucket as shown in the Cloud Console example below. In this example, you can see one monthly rollup and three daily reports for May 2014.



The CSV file will contain line items for each resource during the time frame covered. In this example, you can see line items representing two different instances that have both run for the entire day in a daily report for May 1, 2014.

Report Date,MeasurementId,Quantity,Unit,Resource URI,ResourceId,Location
2014-05-01,com.google.cloud/services/compute-engine/VmimageN1Standard_1,86400,seconds,https://www.googleapis.com/compute/v1/projects/123456789/zones/us-central1-a/instances/testinstance1,13495613777116200000,us-central1-a
2014-05-01,com.google.cloud/services/compute-engine/VmimageN1Standard_1,86400,seconds,https://www.googleapis.com/compute/v1/projects/123456789/zones/us-central1-b/instances/testinstance2,17932867728944100000,us-central1-b

For those of you who’d like to see your usage data for May, we will automatically generate your full monthly usage report for May 2014 in the Cloud Storage bucket you have specified if you enable Compute Engine Usage Export by June 9, 2014.



To find out more about how to use the Compute Engine Usage Export, please check out the Usage Export documentation.



-Posted by Ken Sim, Product Manager