Overview of App Engine Features

This page summarizes App Engine's features. A feature may be available in every runtime language, or only in a subset of languages. The functionality of a feature is usually the same in all the runtimes where it's available, but there can be exceptions.

Every App Engine feature is classified according to its status and availability:

• General Availability (GA) features are publicly available and are covered by App Engine's SLA and deprecation policy. The implementation of a GA feature is stable; any changes made to it will be backwards-compatible. Unless otherwise noted, the App Engine features described on this page are all in GA.
• Beta features are publicly available. They may or may not become GA features in a future App Engine release, and their implementation may change in backward-incompatible ways.
• Alpha features may or may not become Beta or GA features in a future App Engine release, and their implementation may change in backward-incompatible ways. Some Alpha features are publicly available, while others require that you request access in order to use them.

Some features described here are provided by third-party vendors, others are open source projects. These will be marked third-party or Open Source.

The feature descriptions on this page are grouped according to the general functions they serve:

1. Languages and Runtimes
2. Data storage, retrieval, and search
3. Communications
4. Process Management
5. Computation
6. App configuration and management

Index of features

Languages and Runtimes

Java Runtime

App Engine runs your Java web application using a Java 7 JVM in a safe sandboxed environment. App Engine invokes your app's servlet classes to handle requests and prepare responses in this environment.

App Engine uses the Java Servlet standard for web applications. You provide your app's servlet classes, JavaServer Pages (JSPs), static files and data files, along with the deployment descriptor (the web.xml file) and other configuration files, in a standard WAR directory structure. App Engine serves requests by invoking servlets according to the deployment descriptor.

The secured sandbox environment isolates your application for service and security. It ensures that apps can only perform actions that do not interfere with the performance and scalability of other apps. For instance, an app cannot spawn threads in some ways, write data to the local file system or make arbitrary network connections. An app also cannot use JNI or other native code. The JVM can execute any Java bytecode that operates within the sandbox restrictions.

The Google Plugin for Eclipse adds new project wizards and debug configurations to your Eclipse IDE for App Engine projects. App Engine for Java makes it especially easy to develop and deploy world-class web applications using Google Web Toolkit (GWT). The Eclipse plugin comes bundled with the App Engine and GWT SDKs. Third-party plugins are available for NetBeans and IntelliJ.

Java Developers Guide   [Back to top]

Python Runtime

App Engine executes your Python application code using a pre-loaded Python interpreter in a safe sandboxed environment. Your app receives web requests, performs work, and sends responses by interacting with this environment.

A Python web app interacts with the App Engine web server using the WSGI protocol, so apps can use any WSGI-compatible web application framework. App Engine includes a simple web application framework, called webapp2, to make it easy to get started. For larger applications, mature third-party frameworks, such as Django, work well with App Engine.

The Python interpreter can run any Python code, including Python modules you include with your application, as well as the Python standard library. The interpreter cannot load Python modules with C code; it is a "pure" Python environment.

The secured sandbox environment isolates your application for service and security. It ensures that apps can only perform actions that do not interfere with the performance and scalability of other apps. For instance, an app cannot write data to the local file system or make arbitrary network connections. Instead, apps use scalable services provided by App Engine to store data and communicate over the Internet. The Python interpreter raises an exception when an app attempts to import a module from the standard library known to not work within the sandbox restrictions.

Python Developers Guide  [Back to top]

PHP Runtime ^Beta

The PHP runtime executes your application code in a sandboxed PHP 5.4 environment. Your app receives web requests, performs work, and sends responses by interacting with this environment. The runtime supports many of the standard PHP extensions. You cannot upload extensions written in C.

App Engine runs its own web server, which can be configured using an app.yaml file that's uploaded with your code. This file specifies how incoming HTTP requests to your application are directed to PHP scripts.

The sandbox isolates your application for reliability, scalability and security. For this reason, a small number of PHP functions are not available on App Engine, and others will raise an exception if used incorrectly. For instance, an app cannot write data to the local file system. Instead, apps can use scalable services provided by Google to store and process data and communicate over the Internet.

The PHP runtime provides a built-in Google Cloud Storage stream wrapper that allows you to use many of the standard PHP filesystem functions to access Google Cloud Storage.

PHP Developers Guide  [Back to top]

Go Runtime ^Beta

App Engine runs Go version 1.4. The SDK provides an interface similar to the standard Go http package; writing Go App Engine apps is akin to writing stand-alone Go web servers.

The SDK includes the Go compiler and standard library; it has no additional dependencies. Because the Go runtime executes in a sandboxed environment, some of the standard library functions will return errors and should not be used. For example, an os.ErrPermission will occur if an app tries to write to the local file system or make arbitrary network connections. You must use App Engine's scalable services to store data and communicate over the Internet.

You never need to invoke the Go compiler yourself. Your app is automatically re-built on the server side whenever you upload new code, and if you are running the local development server the SDK automatically recompiles sources on-the-fly when you change them.

The Go runtime environment for App Engine supports goroutines, but they are scheduled on a single thread. Goroutines can run concurrently but not in parallel. An application instance can handle multiple requests. For example, if one request is waiting for a datastore API call, the instance can process another request in the meantime.

Go Developers Guide  [Back to top]

Managed VMs ^Beta

The Managed VM hosting environment lets you run App Engine applications on configurable Compute Engine Virtual Machines (VMs). This VM-based hosting environment offers more flexibility and provides more CPU and memory options. Applications that run on Managed VMs, are not subject to Java and Python runtime restrictions and they have access to all the Compute Engine machine types. You can also add third-party libraries and frameworks to your app.

Managed VMs Overview   [Back to top]

Data storage, retrieval, and search

Datastore

App Engine Datastore is a schemaless NoSQL datastore providing robust, scalable storage for your web application, with the following features:

• No planned downtime
• Atomic transactions
• High availability of reads and writes
• Strong consistency for reads and ancestor queries
• Eventual consistency for all other queries

The Datastore holds data objects known as entities. An entity has one or more properties, named values of one of several supported data types: string, integer, or a reference to another entity. Each entity is identified by its kind, which categorizes the entity for the purpose of queries, and a key that uniquely identifies it within its kind. The Datastore can execute multiple operations in a single transaction. By definition, a transaction cannot succeed unless every one of its operations succeeds; if any of the operations fails, the transaction is automatically rolled back. This is especially useful for distributed web applications, where multiple users may be accessing or manipulating the same data at the same time.

Available in Python | Java | Go   [Back to top]

Datastore Backup/Restore ^Beta

You can use the Datastore Admin tab of the Admin Console to backup selected kinds of entities, or to restore entities from a backup. You can backup using either Blobstore or Google Cloud Storage.

Available in Python | Java | Go   [Back to top]

HRD Migration Tool

Use the migration tools found in the Application Settings tab of the Administration Console to migrate your application data from the deprecated Master/Slave Datastore to the High Replication Datastore (HRD). You must duplicate your application and then migrate the data. You may need to make some changes to your application to ensure an optimal migration.

Available in Python | Java | Go   [Back to top]

Google Cloud SQL

Google Cloud SQL is a MySQL database that lives in Google's cloud. It has all the capabilities and functionality of MySQL, with a few additional features and a few unsupported features as listed below. Google Cloud SQL is easy to use, doesn't require any software installation or maintenance and is ideal for small to medium-sized applications.

Google Cloud SQL provides these features:

• Ability to host your MySQL databases in the cloud
• All data replicated in multiple locations for great availability and durability
• Choice of billing options:
• Per use option means you only pay for the time you access your data
• Package option allows you to control your costs for more frequent access
• Google Cloud SQL instances can have up to 16GB of RAM and 500GB data storage
• Create and manage instances in the Google Developers Console
• Data stored in datacenters in the EU or the US
• Cloud SQL customer data is encrypted when on Google’s internal networks and when stored in database tables and temporary files (with encryption of backups coming soon)
• Synchronous or asynchronous geographic replication
• Import or export databases using mysqldump
• Java and Python compatibility
• Support for MySQL wire protocol and MySQL connectors
• Support for connecting with the Secure Sockets Layer (SSL) protocol

Available in Python | Java | PHP | Go   [Back to top]

Blobstore

The Blobstore API allows your application to serve data objects, called blobs, that are much larger than the size allowed for objects in the Datastore service. Blobs are useful for serving large files, such as video or image files, and for allowing users to upload large data files. Blobs are created by uploading a file through an HTTP request. Typically, your applications will do this by presenting a form with a file upload field to the user. When the form is submitted, the Blobstore creates a blob from the file's contents and returns an opaque reference to the blob, called a blob key, which you can later use to serve the blob. The application can serve the complete blob value in response to a user request, or it can read the value directly using a streaming file-like interface.

Available in Python | Java | Go   [Back to top]

Google Cloud Storage Client Library

Google Cloud Storage is useful for storing and serving large files. Additionally, Cloud Storage offers the use of access control lists (ACLs), and the ability to resume upload operations if they're interrupted, and many other features. (The Google Cloud Storage client library makes use of this resume capability automatically for your app, providing you with a robust way to stream data into Google Cloud Storage.)

The Google Cloud Storage client library lets your application read files from and write files to buckets in Google Cloud Storage. This library supports reading and writing large amounts of data to Google Cloud Storage, with internal error handling and retries, so you don't have to write your own code to do this. Moreover, it provides read buffering with prefetch so your app can be more efficient.

Available in Python | Java | PHP | Go   [Back to top]

Search

The Search API provides a model for indexing documents that contain structured data (text and HTML strings, atoms, dates, geopoints, numbers). Documents and indexes are saved in a separate persistent store optimized for search operations. You can search an index, and organize and present search results. The API supports partial text matching on string fields. The Search API can index any number of documents, however, a single search can return no more than 10,000 matching documents. The App Engine Datastore may be more appropriate for applications that need to retrieve very large result sets.

Available in Python | Java | Go   [Back to top]

Prospective Search ^Alpha

Prospective search is a querying service that allows your application to match search queries against real-time data streams. For every document presented, prospective search returns the ID of every registered query that matches the document.

Prospective search allows you to register a large set of queries and simultaneously match the queries against a single document. It is particularly useful for applications that process streaming data, for example:

Applications that match against all the updates on a social networking service, or against high-frequency comments in a chat room. Applications that process data sources that provide notification, monitoring, or filtering services. To understand prospective search, it's helpful to compare it to the conventional retrospective search model. In a retrospective search application, such as Google search, the application must build, or have access to, an index of the data to be searched. Needing to pre-index the data makes it difficult and expensive to create real-time applications, because each query must be executed separately against a potentially large index.

In a prospective search application, such as Google Alerts, you register search queries and match them against new documents in real time, as the documents are inserted into your application. This allows you to create applications that efficiently monitor incoming live data. You are not limited to using existing, indexed data.

Applications often use both retrospective and prospective search capabilities to get the best of both worlds. For example, an application can use retrospective search to find matching documents indexed in the past while using prospective search to find matching documents as soon as they arrive.

Available in Python | Java   [Back to top]

Memcache

App Engine supports two classes of the memcache service:

• Shared memcache is the free default for App Engine applications. It provides cache capacity on a best-effort basis and is subject to the overall demand of all applications served by App Engine.
• Dedicated memcache provides a fixed cache capacity assigned exclusively to your application. It's billed by the GB-hour of cache size. Having control over cache size means your app can perform more predictably and with fewer accesses to more costly durable storage. (Note that dedicated memcache is only available to HRD apps.)

Whether shared or dedicated, memcache is not a durable storage. Keys may be evicted when the cache fills up, according to the cache's LRU policy. Changes in the cache configuration or datacenter maintenance events may also flush some or all of the cache.

Available in Python | Java | PHP | Go   [Back to top]

Dedicated Memcache

Dedicated memcache provides a fixed cache capacity assigned exclusively to your application. It's billed by the GB-hour of cache size. Having control over cache size means your app can perform more predictably and with fewer accesses to more costly durable storage.

Available in Python | Java | PHP | Go   [Back to top]

Logs

App Engine maintains two kinds of logs:

• Application logs contain arbitrary messages with a timestamp and log level.
• Request logs contain entries for each request handled by your app, with information such as the app ID, HTTP version, and so forth. Each request log includes a list of application logs associated with that request.

The Logs API provides access to your application's logs. You can also access the logs from the Admin Console.

Available in Python | Java | PHP | Go   [Back to top]

Communications

Channel

The Channel API creates a persistent connection between an application and its users, allowing the application to send real time messages without the use of polling.

To use the API, you must add a JavaScript client in your web pages. The client performs these tasks:

• Connecting to the channel once it receives the channel’s unique token from the server
• Listening on the channel for updates regarding other clients and making appropriate use of the data (e.g. updating the interface, etc.)
• Sending update messages to the server so they may be passed on to remote clients

Your application, acting as server, is responsible for:

• Creating a unique channel for individual JavaScript clients
• Creating and sending a unique token to each JavaScript client so they may connect and listen to their channel
• Receiving update messages from clients via HTTP requests
• Sending update messages to clients via their channels
• Optionally, managing client connection state.

Available in Python | Java | Go   [Back to top]

Google Cloud Endpoints

Google Cloud Endpoints consists of tools, libraries and capabilities that allow you to generate APIs and client libraries from an App Engine application, referred to as an API backend, to simplify client access to data from other applications. Endpoints makes it easier to create a web backend for web clients and mobile clients such as Android or Apple's iOS.

For mobile developers, Endpoints provides a simple way to develop a shared web backend and also provides critical infrastructures, such as OAuth 2.0 authentication, eliminating a great deal of work that would otherwise be needed. Furthermore, because the API backend is an App Engine app, the mobile developer can use all of the services and features available in App Engine, such as Datastore, Google Cloud Storage, Mail, Url Fetch, Task Queues, and so forth. And finally, by using App Engine for the backend, developers are freed from system admin work, load balancing, scaling, and server maintenance.

It is possible to create mobile clients for App Engine backends without Endpoints. However, using Endpoints makes this process easier because it frees you from having to write wrappers to handle communication with App Engine. The client libraries generated by Endpoints allow you to simply make direct API calls.

Available in Python | Java   [Back to top]

Mail

App Engine applications can send email messages on behalf of the app's administrators, and on behalf of users with Google Accounts. Apps can receive email at various addresses. Apps send messages using the Mail service and receive messages in the form of HTTP requests initiated by App Engine and posted to the app.

Available in Python | Java | PHP | Go   [Back to top]

SendGrid ^Third-party

You can use SendGrid to power your emails on Google App Engine. Using SendGrid can improve your deliverability and provide transparency into what actually happens to those emails your app sends. You can see statistics on opens, clicks, unsubscribes, spam reports and more through either the SendGrid interface or its API.

Available in Python | Java | PHP   [Back to top]

Sockets ^Beta

App Engine supports regular outbound sockets in all runtimes, without requiring you to import any special App Engine libraries or add any special App Engine code. However, there are certain limitations and behaviors you need to be aware of when using sockets. The details vary depending on the runtime. Read the runtime documentation for more information.

Available in Python | Java | PHP | Go   [Back to top]

Twilio ^Third-party

Twilio is powering the future of business communications, enabling developers to embed voice, VoIP, and messaging into applications. They virtualize all infrastructure needed in a cloud-based, global environment, exposing it through the Twilio communications API platform. Applications are simple to build and scalable. Enjoy flexibility with pay-as-you go pricing, and benefit from cloud reliability.

Twilio Voice enables your application to make and receive phone calls. Twilio SMS enables your application to send and receive text messages. Twilio Client allows you to make VoIP calls from any phone, tablet, or browser and supports WebRTC.

Available in Python | Java | PHP   [Back to top]

URL Fetch

App Engine applications can communicate with other applications or access other resources on the web by fetching URLs. An app can use the URL Fetch service to issue HTTP and HTTPS requests and receive responses. The URL Fetch service uses Google's network infrastructure for efficiency and scaling purposes.

Available in Python | Java | PHP | Go   [Back to top]

XMPP

An App Engine application can send and receive chat messages to and from any XMPP-compatible chat messaging service, such as Google Talk. An app can send and receive chat messages, send chat invites, request a user's chat presence and status, and provide a chat status. Incoming XMPP messages are handled by request handlers, similar to web requests.

Some possible uses of chat messages include automated chat participants ("chat bots"), chat notifications, and chat interfaces to services. A rich client with a connection to an XMPP server (such as Google Talk) can use XMPP to interact with an App Engine application in real time, including receiving messages initiated by the app. (Note that such a client using Google Talk must use the user's password to make an XMPP connection, and cannot use a Google Accounts cookie.)

Currently, an app cannot participate in group chats. An app can only receive messages of types "chat" and "normal". An app can send messages of any type defined in RFC 3921.

Available in Python | Java | Go   [Back to top]

Process management

Task Queue

With the Task Queue API, applications can perform work outside of a user request, initiated by a user request. If an app needs to execute some background work, it can use the Task Queue API to organize that work into small, discrete units, called tasks. The app adds tasks to task queues to be executed later.

App Engine provides two different queue configurations:

• Push queues process tasks based on the processing rate configured in the queue definition. App Engine automatically scales processing capacity to match your queue configuration and processing volume, and also deletes tasks after processing. Push queues are the default.
• Pull queues allow a task consumer (either your application or code external to your application) to lease tasks at a specific time for processing within a specific timeframe. Pull queues give you more control over when tasks are processed, and also allow you to integrate your application with non-App-Engine code using the Task Queue REST API. When using pull queues, your application needs to handle scaling of instances based on processing volume, and also needs to delete tasks after processing.

Available in Python | Java | PHP | Go   [Back to top]

Task Queue REST API ^Alpha

Provides a REST interface so any web client app can manage your application's task queues and the tasks in them.

Available in Python | Java | PHP and Go also implement the API as defined for Python   [Back to top]

Task Queue Tagging ^Beta

The tagging API lets pull queue consumers lease a specified number of tasks with the same tag from a pull queue.

Available in Python | Java | Go   [Back to top]

Scheduled Tasks

The App Engine Cron Service allows you to configure regularly scheduled tasks that operate at defined times or regular intervals. These tasks are commonly known as cron jobs. These cron jobs are automatically triggered by the App Engine Cron Service. For instance, you might use this to send out a report email on a daily basis, to update some cached data every 10 minutes, or to update some summary information once an hour.

A cron job will invoke a URL, using an HTTP GET request, at a given time of day. An HTTP request invoked by cron can run for up to 10 minutes, but is subject to the same limits as other HTTP requests.

Free applications can have up to 20 scheduled tasks. Paid applications can have up to 100 scheduled tasks.

Available in Python | Java | PHP | Go   [Back to top]

Computation

Images

The Images API provides the ability to describe and manipulate image data. The API provides information about an image, such as its format, width, height, and a histogram of color values. It can resize, rotate, flip, and crop images. It also has the ability to composite multiple images into one image, enhance images automatically, and convert images between several formats

The Images service can accept image data directly from the app, or it can use data retreived from Blobstore or Google Cloud Storage.

Available in Python | Java | Go   [Back to top]

MapReduce ^{Open Source}

App Engine MapReduce is an open source library that is built on top of App Engine services, including Datastore and Task Queues. You must include the library with your application, it provides:

• A programming model for large-scale distributed data processing
• Automatic parallelization and distribution within your existing codebase
• Access to Google-scale data storage
• I/O scheduling
• Fault-tolerance, handling of exceptions
• User tunable settings to optimize for speed/cost
• Tools for monitoring status

Available on Github for Java and Python   [Back to top]

App configuration and management

App Identity

The Application Identity service provides the ability for an application to identify itself, by retrieving its App ID or the hostname part of its URL. The app's identity may be used to generate a URL or email address, or to make some run-time decision.

Many Google APIs support OAuth assertions to identify the source of a request. The App Identity API can create tokens that can be used to assert that the source of a request is the application itself. This token can be included in the HTTP headers of a call to identify the calling application. The OAuth token only works against Google systems. However you can use the underlying signing technology in the API to assert the identity of your application to other systems.

Available in Python | Java | PHP | Go   [Back to top]

OAuth ^Alpha

: Using Google Accounts and the OAuth API, any App Engine application can be an OAuth service provider. Google Accounts handles the details of the OAuth protocol. The app accesses the consumer authentication information using the OAuth API during a service request, just as it would with the Users API if the user were accessing the app directly.

Available in Python | Java | PHP | Go   [Back to top]

OpenID ^Alpha

OpenID is an open technology used for authenticating users across various web services. A user creates an account on a service that acts as an OpenID provider. When logging in to a provider, the user receives an OpenID identifier, which can be used to sign in to other services, called OpenID relying parties.

An App Engine app can be configured as an OpenID relying party, and use OpenID for sign in. App Engine apps cannot be Open ID providers.

Available in Python | Java | PHP | Go   [Back to top]

Capabilities

You can use the Capabilities API to detect the availability of services (like the Images service) or specific capabilities of a service (such as datastore reads and writes). You can reduce downtime in your application by testing for the availability of a capability before trying to use it. The services that support this API vary depending which runtime you are using.

Available in Python | Java | Go   [Back to top]

Modules

Modules are used to factor large applications into logical components that can share stateful services and communicate in a secure fashion. An app that handles customer requests might include separate modules to handle other tasks:

• API requests from mobile devices
• Internal, admin-like requests
• Backend processing such as billing pipelines and data analysis

Modules can have different versions, performance levels, and authorization. While running, a particular module will have one or more instances, which may be managed statically or dynamically. Incoming requests are routed to an existing or new instance of the appropriate module.

Scaling types control the creation of instances. There are three scaling types. Each type offers a variety of instance classes, with different amounts of CPU and Memory:

• A module with manual scaling runs continuously, allowing you to perform complex initialization and rely on the state of its memory over time.
• A module with basic scaling will create an instance when the application receives a request. The instance will be turned down when the app becomes idle. Basic scaling is ideal for work that is intermittent or driven by user activity.
• Automatic scaling is the scaling policy that App Engine has used since its inception. It is based on request rate, response latencies, and other application metrics. Previously users could use the Admin Console to configure the automatic scaling parameters (instance class, idle instances and pending latency) for an application's frontend versions only. These settings now apply to every version of every module that has automatic scaling.

Available in Python | Java | PHP | Go   [Back to top]

Multitenancy

Multitenancy is a software architecture in which one instance of an application, running on a remote server, serves many client organizations, or tenants.

Google App Engine provides the Namespaces API, which includes a namespace manager. The Namespaces API is also incorporated in other namespace-enabled APIs. When you set the namespace in the namespace manager, all the namespace-enabled APIs use that namespace by default. This lets you partition data across tenants by specifying a unique namespace for each tenant. For instance, using namespaces with the Datastore API, all users can share the same data schema, but different users see different content.

The Namespaces API is integrated with Google Apps, allowing you to use your Google Apps domain as the current namespace. Because Google Apps lets you deploy your app to any domain that you own, you can easily set unique namespaces for all domains linked to your Google Apps account.

Available in Python | Java | Go   [Back to top]

Remote

Lets external applications transparently access App Engine services. For example, you can use Remote API to access a production datastore from an app running on your local machine.

Available in Python | Java | Go   [Back to top]

SSL for Custom Domains

Allows applications to be served via both HTTPS and HTTP via a custom domain instead of an appspot.com address.

Available in Python | Java | PHP | Go   [Back to top]

Traffic Splitting

Allows you to roll out features for your app slowly over a period of time and do A/B testing. Traffic Splitting works by splitting incoming requests to different versions of your app.

Available in Python | Java | PHP | Go   [Back to top]

Users

App Engine applications can authenticate users using any one of three methods: Google Accounts, accounts on your own Google Apps domains, or OpenID identifiers. (Note that the support for OpenID is in Beta.) An application can detect whether the current user has signed in, and can redirect the user to the appropriate sign-in page to sign in or, if your app uses Google Accounts authentication, create a new account. While a user is signed in to the application, the app can access the user's email address (or OpenID identifier if your app is using OpenID). The app can also detect whether the current user is an administrator, making it easy to implement admin-only areas of the app.

Available in Python | Java | PHP | Go   [Back to top]