Google Cloud Platform Blog
Help us improve Google Cloud Platform by participating in a Build Day Study
Monday, July 15, 2013
We are constantly looking to improve our products by gathering real customer feedback from you. Often times it's through
G+ comments
, 1:1 interactions,
Twitter
, conferences, and more. '
Are you interested in learning more about Google Cloud Platform development as well as helping improve our products directly? You can by participating in a Google Cloud Platform Build Day Study! Selected participants will be invited to our Mountain View or Seattle campuses to work with Google engineers on a day-long build as a part of a User Experience Research study.
Please fill out this questionnaire if you’re interested
, and we’ll be in touch if you’re a good fit!
Thanks in advance for your interest and help.
- Posted by Andrew Macvean, UX Researcher
Get up and Running with Cassandra on Google Compute Engine
Wednesday, July 10, 2013
Recently, Eric Johnson released a
guide to setting up a Cassandra cluster
on
Google Compute Engine
. Cassandra is a
NoSQL database
that is designed around distributed principles. By distributing data across multiple nodes, your cluster becomes resilient to individual node failure, and scaling up your cluster is as trivial as adding new nodes.
The guide walks you through creating your nodes (instances), setting up Java, and creating and configuring a firewall. Included in the guide are several scripts that make the configuration and setup easy to understand and execute. Once you are finished with your cluster, a simple call to a teardown script cleans up your project’s environment.
Depending on your system requirements, you will want to make some adjustments to the setup in the guide. For instance, you should modify the global configuration file to meet the needs of your system. Cassandra runs best with plenty of CPU and memory, so you will likely want to choose one of our higher power
instance types
. You should also adjust the number of overall nodes and nodes per zone to match your requirements. Lastly, best practices highly recommend that you use
persistent disks
for your cluster.
Many of the core features of Google Compute Engine match up well with the requirements of a distributed database like Cassandra. Distributing instances across
zones
protects against individual node and zone failures. Using the metadata server means that your nodes can configure themselves, and a change to the configuration file can propagate easily to existing nodes. Consistent, fast disk I/O means that you can rely upon quick queries and reliable write throughput.
For more information about Cassandra, to download, or to contribute, visit the
database’s site
. Hear from experts about different approaches to distributed databases by watching the
Google I/O industry panel
.
- Posted by Julia Ferraioli, Developer Advocate
Lock-in, what lock-in?
Tuesday, July 9, 2013
A frequent theme we have heard from customers and read in the community lately has been about lock-in. What does it mean to choose a cloud provider? What trade-offs are reasonable but also proprietary? Is there a model that works or can work for you? Peter Magnusson, the engineering director responsible for
Google App Engine
,
takes on the topic recently
on G+ and it is
worth a read
. But he’s not the only
pundit
discussing the
topic
and I’m sure he won’t be the
last
.
Take a look at
Peter’s thoughts
and let us know what you think.
- Posted by Brian Goldfarb, Head of Marketing
Development in the Cloud with Codenvy and Google Cloud Platform
Monday, July 8, 2013
Today’s post is from Tyler Jewell, CEO at Codenvy. In this post, Tyler looks at how you can leverage cloud based development tools to build applications for Google Cloud Platform.
Codenvy
is a cloud development environment for coding, building, and testing applications for
Google Cloud Platform
. In a recent LinkedIn survey, 1200 engineers indicated that they spend nearly ⅓ of their week administering their desktop. This includes configuring the IDE, build system, runtime, and plug-ins. By providing a cloud IDE that is pre-integrated with
Google App Engine
, we can change the developer workflow dramatically by automatically provisioning a cloud workbench that allows a developer to be immediately productive with fewer errors, and have a higher confidence that their application will deploy correctly to App Engine when pushed.
Each Codenvy workspace is comprised of an IDE, a code assistant service, a build system and a debugging runtime. These four components are integrated and decoupled to scale independently based on usage. This allows us to eliminate configuration needs while reducing compilation time and deployment time. Projects in Codenvy can start through a project creation wizard, or can be imported from a git repository. After coding operations are complete, code can be deployed directly to App Engine through using continuous integration post-commit hooks on git, or through a jClouds-based direct deployment connection to App Engine.
The main challenge is setting up the development environment to match the production environment. Each language, framework and PaaS has its nuances that must be addressed. To reproduce the App Engine environment in Codenvy, we embed the Google App Engine SDK into the IDE (enabling compilation and auto-completion), and in the debugging runtime so that applications can be functionally tested in a cloud-local environment before being pushed to App Engine directly.
The App Engine SDK is provided automatically by Codenvy whenever a App Engine application is configured in the project space. By providing this SDK in a cloud local environment, you save time by being able to do a high number of iterative changes to test functionality before pushing artifacts onto a App Engine instance. Since the deployment process of your artifacts onto App Engine has an uptake time, developers who need to make many changes would wait longer than making use of the cloud local environment that is low latency.
Other Google Services Used by Codenvy
Google technology has been instrumental in building our product and growing our user base:
First, we enable multi-cursor collaborative editing which is incredibly useful for pair programming, code reviews, or classroom teaching. This is powered by Google Web Toolkit, a framework to write optimized Ajax applications, and Collide, an open source collaboration system published by Google.
Second, oAuth makes it possible to register with a Google account and start coding in seconds.
Third, we support Chromebook development and have certified on Pixel tablets.
Finally, we are preparing Android support in Codenvy. Check out this sample demo of an Android app built in Codenvy and deployed to
Manymo’s
web-based Android emulator:
http://vimeo.com/66157251
.
Later this year, we will release production Android development support. In addition to editing, building and packaging Android applications, it will be possible to run them in the browser with a tenanted emulator. We’ll also be shipping an SDK that will allow the community to create programming language, deployment target, and framework extensions so that we can work to extend Codenvy to support PHP and Go in App Engine.
Please visit Codenvy today and get started building your App Engine application. Full documentation and tutorials are at
docs.codenvy.com
. And you can vote for the features that
you want here
. And finally, do not hesitate to contact support with any questions at support@codenvy.com.
- Contributed by Tyler Jewell, CEO, Codenvy
See how to test drive Virtual Machines on Google Compute Engine in 10 minutes
Wednesday, July 3, 2013
A single US cent stretches quite far when experimenting with Google Compute Engine. Watch this video to see how affordable and quick it is. In this video, we walk through the steps required to create a Google Cloud Platform project and start up a virtual machine. Next, we install a web server (Apache) on the virtual machine and fetch a web page to confirm a successful installation. Finally we tear down the virtual machine to wrap up the exercise - all in less than 10 minutes, and under a cent!
Check out the video to see how easy it is to get started with Google Compute Engine and the Cloud Platform. For a self-paced guide on getting started with Compute Engine,
visit the Hello World tutorial
.
- Posted by Jonathan Simon, Developer Relations
IT Operations Management for Google Compute Engine based Apps with Boundary
Tuesday, July 2, 2013
The following post was contributed by Gary Read CEO of Boundary, a modern IT operations management platform provider and Google Cloud Platform partner. With the Boundary service, operations teams can get early warnings about potential problems, before the dominoes start to fall, so that they can prevent application outages. Get a
free, full functional version of Boundary
just for Google Compute Engine customers.
Cloud services like
Google Compute Engine
have fundamentally altered the IT landscape and to effectively monitor this landscape, a new approach is required. To understand and manage performance, operations teams need a monitoring service architected for today’s realities:
Rapid rate of change
. With today’s DevOps and Agile approaches, organizations are rolling out applications daily, if not hourly. With shifting workloads and resources, static reports and even hourly updates won’t suffice.
Highly distributed
. As organizations take advantage of cloud services and new technologies like Hadoop, and Cassandra, application environments continue to get more distributed.
It’s with these basic realities in mind, that Boundary developed an entirely new SaaS-based monitoring service. Our platform supports dynamic environments by streaming data from every system,
every second
, and applying real-time analytics to that information. In distributed environments, we analyze the flows between every application and infrastructure component to give a comprehensive view on all the inter-dependencies and how the distributed system is performing as a whole.
Once this data is collected, Boundary applies big data analytics to the task of IT operations management, and delivers insights through graphical views that are visually exciting and easy to understand. In other words, Boundary makes it practical for IT teams to quickly understand and optimize their modern IT environments.
Delivering Optimized Support for Google Compute Engine
We chose software, rather than appliances to collect application flow data, as this enables us to provide deep visibility without access to the network. Streaming per-second views of performance data, deep visibility into latency, and support for cloud environments, makes Boundary uniquely equipped for Google Compute Engine monitoring.
Agile/DevOps support
Increasing responsive to changing technical and business demands, often means using multiple tools to manage environments and service levels. Boundary can combine real-time streaming data with alerts from third-party platforms, including Google Compute Engine partners Opscode and Puppet Labs, to support fast-changing development environments. This means as customers push out new configurations and automations in Chef and Puppet Enterprise, they can instantly see those changes correlated against the performance impact.
The Boundary Architecture
The graphic below provides an overview of Boundary’s architecture. Lightweight meters automatically stream data from Google Compute Engine instances to the Boundary analytics engine. We analyze this streaming “application chatter” in real-time to automatically build application topology maps,
establish a baseline of normal behavior, and generate alerts when there is a deviation from that normal behavior
. Events from third parties like Chef and Puppet are correlated against this streaming data, so the operation team has a consolidated view and can quickly do root cause analysis.
Please
sign up to check out the free, full-function version of Boundary
just for Google Compute Engine users. Take a look and let us know what you think.
- Contributed by Gary Read, CEO, Boundary
Google App Engine takes the pain out of sending iOS push notifications
Monday, July 1, 2013
Delivering scalable, reliable mobile push notifications when hundreds of thousands of users have installed your app on their phones can be a major headache. Fortunately, Google App Engine’s support for sockets and accessible but powerful queues makes it easy to quickly build a mobile backend that can reliably scale to huge numbers of devices.
Get the code!
We’ve created a simple push notification application to help you get started in our Github repository that uses all of the techniques described below.
Download or fork the source
code to get started.
Push notifications are the little pings your phone gives you to let you know that you’ve got a new message, your friend is waiting for you to take your turn on the latest game or that band you like has just announced a concert in your town. As a developer, push notifications give you a new dimension to engage with your users in real time, any time, regardless as whether they have your app open or even if they have their phone in their hand.
On iOS devices, like iPhones and iPads, push notifications are handled by Apple’s Push Notification Service (APNS). APNS is hosted by Apple, and acts as a bridge between your server and your mobile clients. In brief, here’s how it works:
Your mobile application securely registers itself with Apple to be able to receive push notifications, usually when the app is being launched the first time. It receives a device token, which the mobile application passes to your mobile backend.
Your server opens a secure connection to APNS, and when an event occurs that requires a push notification - your server sends a short message including the device token of the device that should receive the message to APNS. APNS will then handle the ‘last-mile delivery’ of the notification to the device.
Although this seems relatively trivial, there are a few important things to consider when implementing push notifications in your application.
Connection pooling with backend instances and pull queues
If you have a popular application you can quickly end up generating a large number of push notifications - even after a single event.
For both performance reasons, and should avoid opening a large number of secure connections to APNS, but rather simply hold a few connections open and funnel any push notifications your applications generate through those. This approach is commonly called Connection Pooling.
Fortunately, App Engine provides the building blocks for scalable connection pooling. Resident
backend instances
are long running App Engine containers that can be used to as workers to hold open APNS notifications for sending notifications. These workers can then
monitor a pull queue
that can signal to the workers when a notification should be sent. When an event occurs in another component of your application that should trigger a push notification (say an action triggered by your mobile API in a frontend instance), other components of your application can simply enqueue a task on the pull queue.
Each worker can then periodically read from a pull queue to see if any notifications need to be sent by the application, and if there are, lease a block of them, send them via the previously established APNS connection, and delete them.
As well as saving on opening many connections to APNS, this approach also improves the reliability of the app. If a worker is unable to deliver a message to APNS for some reason (e.g., because the TCP connection was severed), App Engine’s pull queues will release the lease on the task and allow another worker to retry it. You can also scale the solution simply by adding additional workers that read off the same pull queue.
Sending bulk notifications with push queues and cursors
You may find a need to send a push notification to a large number of devices at once. This requires a query to your database/datastore to find the list of relevant device tokens and then enqueuing a request onto the pull queue described above that includes the message you want to send along with the relevant device token.
If you were to attempt this in a single request, you could quickly run into problems as your list of device IDs becomes large. A simple but elegant solution is to use push queues and (if you’re storing device IDs in the App Engine datastore) query cursors.
A
query cursor
is a token that can be used to iterate over a given a given query result set in small batches. A query cursor is an opaque string marking the index position of the last result retrieved. The application can then use the cursor as the starting point for a subsequent retrieval operation, to obtain the next batch of results from the point where the previous retrieval ended
Query cursors can be combined with App Engine push queues. A
push queue
handler is written to take a query and an optional cursor. The push queue handler then executes the query with a small result limit (say 100 entities), and for each result adds a task to the pull queue described above. If the result of the query also includes a cursor then this indicates there are still unretrieved entities in the query. Once the task handler has cycled through the results it has retrieved, if it has a new cursor, then it can initiate a new push task with that cursor’s value.
Connecting to APNS
While you can use App Engine’s outbound sockets support to talk to APNS from
Java
or
Python
directly, popular 3rd party libraries such as
JavaPNS
also work well, and often provide a cleaner higher level interface for sending notifications.
Putting it all together
Although this sounds like a lot, putting all of this together on App Engine is remarkably straightforward, requiring only a simple batch query queue handler and notification worker. Everything else is taken care of by App Engine’s robust queueing and datatore APIs.
If you’re feeling ready to add Push Notifications to your app, we’ve got some great resources to help you get started.
Download the source
to our (Java) iOS push notification example on Github
Read our in depth Cloud solutions paper
on Orchestrating iOS Push Notifications, which covers this architecture (and more) in greater detail
Read more about push queues (
Python
,
Java
), pull queues (
Python
,
Java
) and outbound sockets (
Python
,
Java
)
- Posted by Grzegorz Gogolowicz, Solutions Architect
Don't Miss Next '17
Use promo code NEXT1720 to save $300 off general admission
REGISTER NOW
Free Trial
GCP Blogs
Big Data & Machine Learning
Kubernetes
GCP Japan Blog
Labels
Announcements
56
Big Data & Machine Learning
91
Compute
156
Containers & Kubernetes
36
CRE
7
Customers
90
Developer Tools & Insights
80
Events
34
Infrastructure
24
Management Tools
39
Networking
18
Open Source
105
Partners
63
Pricing
24
Security & Identity
23
Solutions
16
Stackdriver
19
Storage & Databases
111
Weekly Roundups
16
Archive
2017
Feb
Jan
2016
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2015
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2014
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2013
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2012
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2011
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2010
Dec
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2009
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Feed
Subscribe by email
Technical questions? Check us out on
Stack Overflow
.
Subscribe to
our monthly newsletter
.
Google
on
Follow @googlecloud
Follow
Follow