cloud-test: June 2011

Google Cloud Platform Blog

Highlights from our Google I/O Presentations

Wednesday, June 22, 2011

Now that things have settled down from last month’s Google I/O, we thought it would be a good time to highlight some of the technical talks given by the App Engine team. At this year’s I/O, we emphasized some of the big themes we’ve been focusing on this past year: availability and productionization of our platform, removing limits while maintaining performance at scale, and developing new APIs.Availability and Productionization of our PlatformSince App Engine automatically handles all of the sharding and distribution of your application across datacenters, we thought it would be important to talk about the strategies we use to run and maintain our platform. These talks also give you an opportunity to meet some of the engineers who carry pagers so you don’t have to.

More 9s Please: Under the Covers of the High Replication Datastore - We were excited to announce the HR Datastore in January, since it provides a high level of availability, even during unplanned outages, and even if an entire datacenter goes offline. HRD uses the paxos algorithm to distribute data across multiple datacenters. Alfred Fuller and Matt Wilder talk about the high level distribution strategy used and how this helps keep your app serving.

Life in App Engine Production - Let’s face it - everyone has some great war stories about running live applications. And the stories of Google’s SRE team probably rank with the best of them. Michael Handler and Alan Green talk about what goes right and what can go wrong running in Production.

Removing Limits while maintaining performance at scaleHow can you design your application so you can transition from serving 1 user to 1 million users with the minimum amount of work? How do you make sure your user experience stays consistent for those users?

App Engine Backends - At I/O this year we launched App Engine Backends, which allow you to write long running processes on App Engine. Crunch numbers, process data, and build large in-memory models! Justin Haugh and Greg Darke give an overview of this new feature and cover best practices.

Scaling App Engine Applications - We always get lots of question on what it means and how it works to scale an app using App Engine (hint: low latency is the key!). Guido van Rossum and Justin Haugh explain how instances work and cover best practices for scaling your application.

Developing New APIs

Full Text Search - It took 8 days after App Engine’s initial launch for this feature request to be filed - and now work on the Full Text Search API is nearing completion. Bo Majewski and Ged Ellis give a preview of the upcoming App Engine Full Text Search API.

App Engine MapReduce- So you just want to analyze 5TB of data? You’ll probably need MapReduce to do so! Mike Aizatsky talks about the progress being made on the App Engine MapReduce framework and how you can use it to generate reports for your data.

Large-scale Data Analysis using the App Engine Pipeline API - Once you’ve written one MapReduce, you’ll probably want to write more, and then you’ll need a way to manage your application’s analysis pipeline. In this talk, Brett Slatkin discusses a lightweight API for managing MapReduce workflows.

Our team always enjoys participating in Google I/O because our engineers get to share their vision and discuss their work with a large audience. If you are interested in what some of our customers had to say at I/O, check out the Google Enterprise Blog. Of course, with App Engine featured in 23 talks for Google I/O, we couldn’t cover them all here. See the complete session list for I/O and discover all the talks on App Engine and more!Posted by The App Engine Team

Google

App Engine 1.5.1 Release

Monday, June 20, 2011

It’s been about a month since Google I/O 2011, so we thought it was time for another release. This month we are releasing ProtoRPC as an official Python API, making the High Replication Datastore (HRD) available in the SDK to help developers better understand the consistency model, adding presence to the Channel API, plus a few more goodies. Check them out below:Serving Changes

Geolocation Header: App Engine will now include a header with every client request that represents a best-effort attempt to identify the country from which the request originated. This header, “X-AppEngine-country,” can allow you to customize content based on the origin of the user. We hope this means no more “Select your location” drop downs for users!

Updated Java and Python APIs

Channel API (Presence Support): You can now configure an inbound service for your application to detect user presence for the Channel API. This lets you take an action when a user connects or disconnects from the Channel.

Images API (WebP Support): The Images API now supports WebP images. If you haven’t checked it out, WebP is a new image format that Google open sourced earlier this year which provides lossy compression for images and can be up to 39% smaller than JPEG images of a similar quality!

New Python APIs

ProtoRPC: ProtoRPC is an open source framework for creating a well defined and easy to use web-based RPC service. While ProtoRPC is similar to using Google Protocol Buffers to define services, the goal of ProtoRPC is to make it easy for developers to start defining web based services, while still allowing those services to scale and evolve over time.

Datastore

High Replication in the SDK: Since releasing the High Replication Datastore, we’ve wanted to provide tools that help developers understand and test the new consistency model while developing their applications. The 1.5.1 SDKs for Java and Python can now emulate the HRD consistency model. This means that now, by setting the appropriate SDK config options, queries across entity groups will occasionally return results that don’t reflect the most recent data written. This should allow you to develop your application to be more resilient to this consistency model.

As with all releases, 1.5.1 contains some additional small features and many bugfixes, the full list of which is available in our release notes for Python and Java. Hopefully you’ll find this release useful. Either way feel free let us know what you think by posting in the App Engine group.Thanks!The App Engine Team