cloud-test: Google Cloud Platform and Leanplum help app developers conduct on-the-fly A/B tests

Google Cloud Platform Blog

Google Cloud Platform and Leanplum help app developers conduct on-the-fly A/B tests

Friday, March 21, 2014

Editor’s note: Today’s guest blog comes from Andrew First, co-founder and CTO at Leanplum, a platform for optimizing the mission-critical metrics of mobile apps. Leanplum processes billions of events per day to provide a faster and more effective way for developers to optimize mobile applications. LeanplumGoogle Cloud PlatformGoogle App EngineGoogle BigQueryGoogle Cloud DatastoreGoogle Cloud Storage Google Compute Engine

By default, App Engine indexes every field in your datastore entities. By indexing only the fields that you need, you can reduce datastore costs by an order of magnitude.

Run a separate version of your application as a staging instance, so you can test out new features. You can run multiple versions of your app simultaneously, so that every engineer has their own application running at the same time to test new features or to show demos.

Use AppStats to find bottlenecks in your application. Using AppStats, we were able to reduce the latency of our API by an order of magnitude.

Use Traffic Splitting to roll out risky changes. Fittingly, we are an A/B testing company that uses A/B testing on our own backend. When we roll out substantial backend changes, we can test them out on a small percentage of traffic to make sure it works well, and to measure the impact on latency and number of instances used.

Use BigQuery to analyze your logs. We wrote a custom mapreduce to collect our logs and dump them into BigQuery. This gives us the ability to gather statistics on latencies for different RPC calls, frequencies and types of errors, and to resolve customer issues by being able to query over our logs in seconds.