Datastore reference

Types and Property Classes

The App Engine datastore supports a fixed set of value types for properties on data entities. Property classes can define new types that are converted to and from the underlying value types, and the value types can be used directly with Expando dynamic properties and ListProperty aggregate property models.

The following table describes the Property classes whose values correspond directly with the underlying data types. Any of these value types can be used in an Expando dynamic property or ListProperty aggregate type.

Property class	Value type	Sort order
StringProperty	str unicode	Unicode (str is treated as ASCII)
ByteStringProperty	ByteString	byte order
BooleanProperty	bool	False < True
IntegerProperty	int long (64 bits)	Numeric
FloatProperty	float	Numeric
DateTimeProperty DateProperty TimeProperty	datetime.datetime	Chronological
ListProperty StringListProperty	list of a supported type	If ascending, by least element; if descending, by greatest element
ReferenceProperty SelfReferenceProperty	db.Key	By path elements (kind, ID or name, kind, ID or name...)
blobstore.BlobReferenceProperty	blobstore.BlobKey	byte order
UserProperty	users.User	By email address (Unicode)
BlobProperty	db.Blob	(not orderable)
TextProperty	db.Text	(not orderable)
CategoryProperty	db.Category	Unicode
LinkProperty	db.Link	Unicode
EmailProperty	db.Email	Unicode
GeoPtProperty	db.GeoPt	By latitude, then longitude
IMProperty	db.IM	Unicode
PhoneNumberProperty	db.PhoneNumber	Unicode
PostalAddressProperty	db.PostalAddress	Unicode
RatingProperty	db.Rating	Numeric

Datastore Value Types

Datastore entity property values can be of one of the following types. See above for a list of corresponding Property classes to use with Model definitions.

Other than the Python standard types and users.User, all classes described in this section are provided by thegoogle.appengine.ext.db module.

str or unicode

A short string value, less than 500 characters in length.

A str value is assumed to be text encoded with the ascii codec, and is converted to a unicode value before being stored. The value is returned by the datastore as a unicode value. For short strings using other codecs, use a unicode value.

Short strings are indexed by the datastore, and can be used in filters and sort orders. For text strings longer than 500 characters (which are not indexed), use a Text instance. For unencoded byte strings longer than 500 bytes (also not indexed), use a Blobinstance. For non-textual unencoded byte strings up to 500 bytes (not characters) that should be indexed, use a ByteStringinstance.

Model property: StringProperty

bool

A Boolean value, True or False.

Model property: BooleanProperty

int or long

An integer value, up to 64 bits.

Python int values are converted to Python long values prior to storage. A value stored as an int will be returned as a long.

If a long larger than 64 bits is assigned, only the least significant 64 bits are stored.

Model property: IntegerProperty

float

A floating point value.

Model property: FloatProperty

datetime.datetime

A date and time. See the datetime module documentation.

If the datetime value has a tzinfo attribute, it will be converted to the UTC time zone for storage. Values come back from the datastore as UTC, with a tzinfo of None. An application that needs date and time values to be in a particular time zone must set tzinfo correctly when updating the value, and convert values to the timezone when accessing the value.

Some libraries use the TZ environment variable to control the time zone applied to date-time values. App Engine sets this environment variable to "UTC". Note that changing this variable in an application will not change the behavior of some datetime functions, because changes to environment variables are not visible outside of the Python code.

If you only convert values to and from a particular time zone, you can implement a custom datetime.tzinfo to convert values from the datastore:

class Pacific_tzinfo(datetime_module.tzinfo):
    """Implementation of the Pacific timezone."""
    def utcoffset(self, dt):
        return datetime_module.timedelta(hours=-8) + self.dst(dt)

    def _FirstSunday(self, dt):
        """First Sunday on or after dt."""
        return dt + datetime_module.timedelta(days=(6-dt.weekday()))

    def dst(self, dt):
        # 2 am on the second Sunday in March
        dst_start = self._FirstSunday(datetime_module.datetime(dt.year, 3, 8, 2))
        # 1 am on the first Sunday in November
        dst_end = self._FirstSunday(datetime_module.datetime(dt.year, 11, 1, 1))

        if dst_start <= dt.replace(tzinfo=None) < dst_end:
            return datetime_module.timedelta(hours=1)
        else:
            return datetime_module.timedelta(hours=0)
    def tzname(self, dt):
        if self.dst(dt) == datetime_module.timedelta(hours=0):
            return "PST"
        else:
            return "PDT"

pacific_time = utc_time.astimezone(Pacific_tzinfo())

See the datetime module documentation (including datetime.tzinfo). See also the third-party module pytz, though note that the pytz distribution has many files.

The DateTimeProperty model property class includes features such as the ability to automatically use the date and time a model instance is stored. These are features of the model, and are not available on the raw datastore value (such as in an Expandodynamic property).

Model properties: DateTimeProperty, DateProperty, TimeProperty

list

A list of values, each of which is of one of the supported data types.

When a list is used as the value of an Expando dynamic property, it cannot be an empty list. This is due to how list values are stored: When a list property has no items, it has no representation in the datastore. You can use a static property and theListProperty class to represent an empty list value for a property.

Model property: ListProperty

db.Key

The key for another datastore entity.

m = Employee(name="Susan", key_name="susan5")
m.put()
e = Employee(name="Bob", manager=m.key())
e.put()

m_key = db.Key.from_path("Employee", "susan5")
e = Employee(name="Jennifer", manager=m_key)

Model properties: ReferenceProperty, SelfReferenceProperty

blobstore.BlobKey

The key for a Blobstore value, generated by the Blobstore when the value is uploaded.

Model properties: blobstore.BlobReferenceProperty

users.User

A user with a Google account.

A User value in the datastore does not get updated if the user changes her email address. This may be remedied in a future release. Until then, you can use the User value's user_id() as the user's stable unique identifier.

Model property: UserProperty

class Blob(arg=None)

Binary data, as a byte string. This is a subclass of the built-in str type.

Blob properties are not indexed, and cannot be used in filters or sort orders.

Blob is for binary data, such as images. It takes a str value, but this value is stored as a byte string and is not encoded as text. Use a Text instance for large text data.

Model property: BlobProperty

class MyModel(db.Model):
    blob = db.BlobProperty()

m = MyModel()
m.blob = db.Blob(open("image.png", "rb").read())

In XML, blobs are base-64 encoded whether or not they contain binary data.

class ByteString(arg)

A short blob value (a "byte string"), less than 500 bytes in length. ByteString is a subclass of str, and takes an unencoded strvalue as an argument to its constructor.

ByteStrings are indexed by the datastore, and can be used in filters and sort orders. For byte strings longer than 500 bytes (which are not indexed), use a Blob instance. For encoded text data, use str (short, indexed) or Text (long, not indexed).

Model property: ByteStringProperty

class Text(arg=None, encoding=None)

A long string. This is a subclass of the built-in unicode type.

arg a unicode or str value. If arg is a str, then it is parsed with the encoding specified by encoding, or ascii if no encoding is specified. See the list of standard encodings for possible values for encoding.

Unlike an entity property whose value is a simple str or unicode, a Text property can be more than 500 characters long. However, Text properties are not indexed, and cannot be used in filters or sort orders.

Model property: TextProperty

class MyModel(db.Model):
    text = db.TextProperty()

m = MyModel()
m.text = db.Text(u"kittens")

m.text = db.Text("kittens", encoding="latin-1")

class Category(tag)

A category or "tag". This is a subclass of the built-in unicode type.

Model property: CategoryProperty

class MyModel(db.Model):
    category = db.CategoryProperty()

m = MyModel()
m.category = db.Category("kittens")

In XML, this is an Atom category element. See the Atom specification.

class Email(email)

An email address. This is a subclass of the built-in unicode type.

Neither the property class nor the value class perform validation of email addresses, they just store the value.

Model property: EmailProperty

class MyModel(db.Model):
    email_address = db.EmailProperty()

m = MyModel()
m.email_address = db.Email("larry@example.com")

In XML, this is a gd:email element. See the GData API reference.

class GeoPt(lat, lon=None)

A geographical point represented by floating-point latitude and longitude coordinates.

Model property: GeoPtProperty

In XML, this is a georss:point element. See georss.org.

class IM(protocol, address=None)

An instant messaging handle.

protocol is the canonical URL of the instant messaging service. Some possible values:

Protocol	Description
sip	SIP/SIMPLE
xmpp	XMPP/Jabber
http://aim.com/	AIM
http://icq.com/	ICQ
http://talk.google.com/	Google Talk
http://messenger.msn.com/	MSN Messenger
http://messenger.yahoo.com/	Yahoo Messenger
http://sametime.com/	Lotus Sametime
http://gadu-gadu.pl/	Gadu-Gadu
unknown	Unknown or unspecified

address is the handle's address.

Model property: IMProperty

class MyModel(db.Model):
    im = db.IMProperty()

m = MyModel()
m.im = db.IM("http://example.com/", "Larry97")

In XML, this is a gd:im element. See the GData API reference.

class Link(link)

A fully qualified URL. This is a subclass of the built-in unicode type.

Model property: LinkProperty

class MyModel(db.Model):
    link = db.LinkProperty()

m = MyModel()
m.link = db.Link("http://www.google.com/")

In XML, this is an Atom link element. See the Atom specification.

class PhoneNumber(phone)

A human-readable telephone number. This is a subclass of the built-in unicode type.

Model property: PhoneNumberProperty

class MyModel(db.Model):
    phone = db.PhoneNumberProperty()

m = MyModel()
m.phone = db.PhoneNumber("1 (206) 555-1212")

In XML, this is a gd.phoneNumber element. See the GData API reference.

class PostalAddress(address)

A postal address. This is a subclass of the built-in unicode type.

Model property: PostalAddressProperty

class MyModel(db.Model):
    address = db.PostalAddressProperty()

m = MyModel()
m.address = db.PostalAddress("1600 Ampitheater Pkwy., Mountain View, CA")

In XML, this is a gd:postalAddress element. See the GData API reference.

class Rating(rating)

A user-provided rating for a piece of content, as an integer between 0 and 100. This is a subclass of the built-in long type. The class validates that the value is an integer between 0 and 100, and raises a BadValueError if the value is invalid.

Model property: RatingProperty

class MyModel(db.Model):
    rating = db.RatingProperty()

m = MyModel()
m.rating = db.Rating(97)

In XML, this is a gd:rating element. See the GData API reference.

Property Classes

All model property classes provided by google.appengine.ext.db are subclasses of the Property base class, and support all of the base constructor's arguments. See the base class documentation for information about those arguments.

The google.appengine.ext.db package provides the following model property classes:

class BlobProperty(...)

A binary data property.

Blob data is a byte string. For text data, which may involve encoding, use TextProperty.

Value type: Blob

class BooleanProperty(...)

A Boolean property.

Value type: bool

class ByteStringProperty(verbose_name=None, ...)

A short blob property (a "byte string"). Takes a ByteString value of 500 bytes or less.

ByteStringProperty property values are indexed, and can be used in filters and sort orders.

Like StringProperty, except the value is not encoded in any way. The bytes are stored literally.

If the ByteStringProperty is required, the value cannot be an empty string.

Value type: ByteString

class CategoryProperty(...)

A category or "tag," a descriptive word or phrase.

Value type: Category

class DateProperty(verbose_name=None, auto_now=False, auto_now_add=False, ...)

A date property, without a time of day. See DateTimeProperty for more information.

Value type: datetime.date. This is converted to a datetime.datetime internally.

class DateTimeProperty(verbose_name=None, auto_now=False, auto_now_add=False, ...)

A date and time property.

If auto_now is True, the property value is set to the current time whenever the model instance is stored in the datastore, overwriting the property's previous value. This is useful for tracking a "last modified" date and time for a model instance.

If auto_now_add is True, the property value is set to the current time the first time the model instance is stored in the datastore, unless the property has already been assigned a value. This is useful for storing a "created" date and time for a model instance.

Date-time values are stored as and returned using the UTC time zone. See datetime.datetime for a discussion of how to manage time zones.

Value type: datetime.datetime

class EmailProperty(...)

An email address.

Neither the property class nor the value class perform validation of email addresses, they just store the value.

Value type: Email

class FloatProperty(...)

A floating point number property.

Value type: float

class GeoPtProperty(...)

A geographical point represented by floating-point latitude and longitude coordinates.

Value type: GeoPt

class IMProperty(...)

An instant messaging handle.

Value type: IM

class IntegerProperty(...)

An integer property, up to 64 bits.

Python int values are converted to Python long values prior to storage. A value stored as an int will be returned as a long.

If a long larger than 64 bits is assigned, only the least significant 64 bits are stored.

Value type: int or long

class LinkProperty(...)

A fully qualified URL.

Value type: Link

class ListProperty(item_type, verbose_name=None, default=None, ...)

A list of values of the type given as item_type.

In a query, comparing a list property to a value performs the test against the list members: list_property = value tests if the value appears anywhere in the list, list_property < value tests if any of the members of the list are less than the given value, and so forth.

A query cannot compare two list values. There is no way to test two lists for equality without testing each element for membership separately.

item_type is the type of the items in the list, as a Python type or class. All items in the list value must be of the given type.item_type must be one of the datastore value types, and cannot be list. See Datastore Value Types, above.

The value of a ListProperty cannot be None. It can, however, be an empty list. When None is specified for the default argument (or when the default argument is not specified), the default value of the property is the empty list.

Tip: Because ListProperty aggregate types do not use the Property classes, Property class features such as automatic values and validation are not applied automatically to members of the list value. If you want to validate a member value using a Property class, you can instantiate the class and call its validate() method on the value.

default is the default value for the list property. If None, the default is an empty list. A list property can define a custom validator to disallow the empty list.

See Entities and Models for more information on ListProperty and list values.

Value type: a Python list of zero or more values, where each value is of the configured type

class PhoneNumberProperty(...)

A human-readable telephone number.

Value type: PhoneNumber

class PostalAddressProperty(...)

A postal address.

Value type: PostalAddress

class RatingProperty()

A user-provided rating for a piece of content, as an integer between 0 and 100.

Value type: Rating

class ReferenceProperty(reference_class=None, verbose_name=None, collection_name=None, ...)

A reference to another model instance. For example, a reference may indicate a many-to-one relationship between the model with the property and the model referenced by the property.

reference_class is the model class of the model instance being referenced. If specified, only model instances of the class can be assigned to this property. If None, any model instance can be the value of this property.

collection_name is the name of the property to give to the referenced model class. The value of the property is a Query for all entities that reference the entity. If no collection_name is set, then modelname_set (with the name of the referenced model in lowercase letters and "_set" added) is used.

Note: collection_name must be set if there are multiple properties within the same model referencing the same model class. Otherwise, a DuplicatePropertyError will be raised when the default names are generated.

ReferenceProperty automatically references and dereferences model instances as property values: A model instance can be assigned to a ReferenceProperty directly, and its key will be used. The ReferenceProperty value can be used as if it were a model instance, and the datastore entity will be fetched and the model instance created when it is first used in this way. Untouched reference properties do not query for unneeded data.

class Author(db.Model):
    name = db.StringProperty()
class Story(db.Model):
    author = db.ReferenceProperty(Author)

story = db.get(story_key)
author_name = story.author.name

author = db.get(author_key)
stories_by_author = author.story_set.get()

As with a Key value, it is possible for a reference property value to refer to a data entity that does not exist. If a referenced entity is deleted from the datastore, references to the entity are not updated. Accessing an entity that does not exist raises aReferencePropertyResolveError.

Deleting an entity does not delete entities referred to by a ReferenceProperty.

Value type: db.Key (see above)

class SelfReferenceProperty(verbose_name=None, collection_name=None, ...)

A reference to another model instance of the same class. See ReferenceProperty.

Value type: db.Key (see above)

class blobstore.BlobReferenceProperty(verbose_name=None, ...)

A reference to the BlobInfo entity that corresponds with a Blobstore value. See ReferenceProperty.

The value stored in the reference property is the Blobstore key. When the property is de-referenced, the corresponding BlobInfoobject is fetched and behaves as the value of the property. As with a ReferenceProperty and model instance values, a BlobReferenceProperty must be assigned a BlobInfo object as its value.

Value type: blobstore.BlobKey

class StringListProperty(verbose_name=None, default=None, ...)

Similar to a ListProperty of Python str or unicode (basestring) values. See ListProperty.

Value type: a Python list of str or unicode values

class StringProperty(verbose_name=None, multiline=False, ...)

A short string property. Takes a Python str or unicode (basestring) value of 500 characters or less.

StringProperty property values are indexed, and can be used in filters and sort orders.

If multiline is False, then the value cannot include linefeed characters. The djangoforms library uses this to enforce a difference between text fields and textarea fields in the data model, and others can use it for a similar purpose.

If the StringProperty is required, the value cannot be an empty string.

Value type: str or unicode

class TextProperty()

A long string.

Unlike StringProperty, a TextProperty value can be more than 500 characters long. However, TextProperty values are not indexed, and cannot be used in filters or sort orders.

TextProperty values store text with a text encoding. For binary data, use BlobProperty.

If the TextProperty is required, the value cannot be an empty string.

Value type: Text

class TimeProperty(verbose_name=None, auto_now=False, auto_now_add=False, ...)

A time property, without a date. Takes a Python standard library datetime.time value. See DateTimeProperty for more information.

Value type: datetime.time. This is converted to a datetime.datetime internally.

class UserProperty(verbose_name=None, auto_current_user=False, auto_current_user_add=False, ...)

A user with a Google account.

If auto_current_user is True, the property value is set to the currently signed-in user whenever the model instance is stored in the datastore, overwriting the property's previous value. This is useful for tracking which user modifies a model instance.

If auto_current_user_add is True, the property value is set to the currently signed-in user the first time the model instance is stored in the datastore, unless the property has already been assigned a value. This is useful for tracking which user creates a model instance, which may not be the same user that modifies it later.

UserProperty does not accept a default value. Default values are set when the model class is first imported, and with import caching may not be the currently signed-in user.

Value type: users.User (see above)

GQL Reference

GQL is a SQL-like language for retrieving entities or keys from the App Engine scalable datastore. While GQL's features are different from those of a query language for a traditional relational database, the GQL syntax is similar to that of SQL.

The GQL syntax can be summarized as follows:

SELECT [* | __key__] FROM <kind>
    [WHERE <condition> [AND <condition> ...]]
    [ORDER BY <property> [ASC | DESC] [, <property> [ASC | DESC] ...]]
    [LIMIT [<offset>,]<count>]
    [OFFSET <offset>]

  <condition> := <property> {< | <= | > | >= | = | != } <value>
  <condition> := <property> IN <list>
  <condition> := ANCESTOR IS <entity or key>

As with SQL, GQL keywords are case insensitive. Kind and property names are case sensitive.

A GQL query returns zero or more entities or Keys of the requested kind. Every GQL query always begins with either SELECT * FROM or SELECT __key__ FROM, followed by the name of the kind. (A GQL query cannot perform a SQL-like "join" query.)

Tip: SELECT __key__ queries are faster and cost less CPU than SELECT * queries.

The optional WHERE clause filters the result set to those entities that meet one or more conditions. Each condition compares a property of the entity with a value using a comparison operator. If multiple conditions are given with the AND keyword, then an entity must meet all of the conditions to be returned by the query. GQL does not have an OR operator. However, it does have an INoperator, which provides a limited form of OR.

The IN operator compares value of a property to each item in a list. The IN operator is equivalent to many = queries, one for each value, that are ORed together. An entity whose value for the given property equals any of the values in the list can be returned for the query.

Note: The IN and != operators use multiple queries behind the scenes. For example, the IN operator executes a separate underlying datastore query for every item in the list. The entities returned are a result of the cross-product of all the underlying datastore queries and are de-duplicated. A maximum of 30 datastore queries are allowed for any single GQL query.

A condition can also test whether an entity has a given entity as an ancestor, using the ANCESTOR IS operator. The value is a model instance or Key for the ancestor entity. For more information on ancestors, see Keys and Entity Groups.

The left-hand side of a comparison is always a property name. Property names are typically comprised of alphanumeric characters optionally mixed with underscores. In other words, they match the regular expression [a-zA-Z0-9_]. Property names containing other printable characters must be quoted with double-quotes. For example: "first.name". Spaces or non-printable characters in property names are not supported.

The right-hand side of a comparison can be one of the following (as appropriate for the property's data type):

• a str literal, as a single-quoted string. Single-quote characters in the string must be escaped as ''. For example: 'Joe''s Diner'
• an integer or floating point number literal. For example: 42.7
• a Boolean literal, as TRUE or FALSE.
• the NULL literal, which represents the null value (None in Python).
• a datetime, date, or time literal, with either numeric values or a string representation, in the following forms:
  • DATETIME(year, month, day, hour, minute, second)
  • DATETIME('YYYY-MM-DD HH:MM:SS')
  • DATE(year, month, day)
  • DATE('YYYY-MM-DD')
  • TIME(hour, minute, second)
  • TIME('HH:MM:SS')
• an entity key literal, with either a string-encoded key or a complete path of kinds and key names/IDs:
  • KEY('encoded key')
  • KEY('kind', 'name'/ID [, 'kind', 'name'/ID...])
• a User object literal, with the user's email address:
  USER('email-address')
• a GeoPt literal, with the latitude and longitude as floating point values:
  GEOPT(lat, long)
• a bound parameter value. In the query string, positional parameters are referenced by number: title = :1 Keyword parameters are referenced by name: title = :mytitle

Note: conditions of the form property = NULL (which are equivalent) check to see whether a null value is explicitly stored in the datastore for that property. This is not the same as checking to see if the entity lacks any value for the property! Datastore queries which refer to a property never return entities which don't have some value for that property.

Bound parameters can be bound as positional arguments or keyword arguments passed to the GqlQuery constructor or a Model class's gql() method. Property data types that do not have corresponding value literal syntax must be specified using parameter binding, including the list data type. Parameter bindings can be re-bound with new values during the lifetime of the GqlQuery instance (such as to efficiently reuse a query) using the bind() method.

The optional ORDER BY clause indicates that results should be returned sorted by the given properties, in either ascending (ASC) or descending (DESC) order. The ORDER BY clause can specify multiple sort orders as a comma-delimited list, evaluated from left to right. If the direction is not specified, it defaults to ASC. If no ORDER BY clause is specified, the order of the results is undefined and may change over time.

An optional LIMIT clause causes the query to stop returning results after the first <count> entities. The LIMIT clause can also include an <offset>, to skip that many results to find the first result to return. An optional OFFSET clause can specify an<offset>, if no LIMIT clause is present.

Note: Like the offset parameter for the fetch() method, an OFFSET in a GQL query string does not reduce the number of entities fetched from the datastore. It only affects which results are returned by the fetch() method. A query with an offset has performance characteristics that correspond linearly with the offset size plus the limit size.

For information on executing GQL queries, binding parameters, and accessing results, see the GqlQuery class, and the Model.gql()class method.

The Model Class

The Model class is the superclass for data model definitions.

Model is provided by the google.appengine.ext.db module.

• Introduction
• Model()
• Class methods:
  • Model.get()
  • Model.get_by_id()
  • Model.get_by_key_name()
  • Model.get_or_insert()
  • Model.all()
  • Model.gql()
  • Model.kind()
  • Model.properties()
• Instance methods:
  • key()
  • put()
  • delete()
  • is_saved()
  • parent()
  • parent_key()
  • to_xml()
• Disallowed Property Names

Introduction

An application defines a data model by defining a class that subclasses Model. Properties of the model are defined using class attributes and Property class instances. For example:

class Story(db.Model):
    title = db.StringProperty()
    body = db.TextProperty()
    created = db.DateTimeProperty(auto_now_add=True)

An application creates a new data entity by instantiating a subclass of the Model class. Properties of an entity can be assigned using attributes of the instance, or as keyword arguments to the constructor.

s = Story()
s.title = "The Three Little Pigs"

s = Story(title="The Three Little Pigs")

The name of the model sub-class is used as the name of the datastore entity kind. The names of the attributes are used as the names of the corresponding properties on an entity. Model instance attributes whose names begin with an underscore (_) are ignored, so your application can use such attributes to store data on a model instance that isn't saved to the datastore.

The datastore and the model class API impose several restrictions on property names and model instance attributes. SeeDisallowed Property Names for a complete description.

A data entity can have an optional parent entity. Parent-child relationships form entity groups, which are used to control transactionality and data locality in the datastore. An application creates a parent-child relationship between two entities by passing the parent entity to the child entity's constructor, as the parent argument. For more information about parents and ancestors, see Keys and Entity Groups.

Every entity has a key, a unique identifier that represents the entity. An entity can have an optional key name, a string unique across entities of the given kind. The entity's kind and name can be used with the Key.from_path() and Model.get_by_key_name() methods to retrieve the entity. For more information about keys, see Keys and Entity Groups.

The method Model.get_or_insert() can be used to retrieve an entity that may not exist, creating it in the datastore if necessary:

keyname = "some_key"
s = Story.get_or_insert(keyname, title="The Three Little Pigs")

Note: A Model instance does not have a corresponding entity in the datastore until it is put() for the first time, either explicitly so or via Model.get_or_insert().

The Model class is provided by the google.appengine.ext.db package.

Constructor

The constructor of the Model class is defined as follows:

class Model(parent=None, key_name=None, **kwds)

The superclass for data model definitions.

Arguments:

parent

The Model instance or Key instance for the entity that is the new entity's parent.

key_name

The name for the entity. The name becomes part of the primary key. If None, a system-generated ID is used for the key.

The value for key_name must not be of the form __*__.

A key_name is stored as a Unicode string, with str values converted as ASCII text.

Calling put() on this object will overwrite any existing datastore entity with the same key.

**kwds

Initial values for the instance's properties, as keyword arguments. Each name corresponds with an attribute defined on the Model class.

Additional keyword arguments:

key

The explicit Key instance for the entity. Cannot be used with key_name or parent. If None, falls back on the behavior for key_name and parent. Useful when using allocate_ids() to reserve numeric IDs for new entities.

The value for key must be a valid Key instance.

Calling put() on this object will overwrite any existing datastore entity with the same key.

Class Methods

The Model class provides the following class methods:

Model.get(keys)

Gets the model instance (or instances) for the given Key objects. The keys must represent entities of the model's kind. If a provided key is not of the correct kind, a KindError is raised.

This method is similar to the db.get() function, with additional type checking.

Arguments:

keys

A Key object or a list of Key objects. Can also be a string version of a Key object, or list of strings.

config

A configuration object for the API call.

Model.get_by_id(ids, parent=None)

Gets the model instance (or instances) for the given numeric ID (or IDs).

Arguments:

ids

A numeric entity ID, or a list of numeric entity IDs.

parent

The parent entity for the requested entities, as a Model instance or Key instance, or None (the default) if the requested entities do not have a parent. Multiple entities requested by one call must all have the same parent.

config

A configuration object for the API call.

If ids is a string representing one name, then the method returns the model instance for the name, or None if the entity doesn't exist. If ids is a list, the method returns a list of model instances, with a None value when no entity exists for a corresponding Key.

Model.get_by_key_name(key_names, parent=None)

Gets the model instance (or instances) for the given key name (or names).

Arguments:

key_names

A key name, or a list of key names.

parent

The parent entity for the requested entities, as a Model instance or Key instance, or None (the default) if the requested entities do not have a parent. Multiple entities requested by one call must all have the same parent.

config

A configuration object for the API call.

If key_names is a string representing one name, then the method returns the model instance for the name, or None if the entity doesn't exist. If key_names is a list, the method returns a list of model instances, with a None value when no entity exists for a corresponding Key.

Model.get_or_insert(key_name, **kwds)

Attempts to get the entity of the model's kind with the given key name. If it exists, get_or_insert() simply returns it. If it doesn't exist, a new entity with the given kind, name, and parameters in kwds is created, stored, and returned.

The get and subsequent (possible) put are wrapped in a transaction to ensure atomicity. Ths means that get_or_insert()will never overwrite an existing entity, and will insert a new entity if and only if no entity with the given kind and name exists.

In other words, get_or_insert() is equivalent to this Python code:

def txn():
  entity = MyModel.get_by_key_name(key_name, parent=kwds.get('parent'))
  if entity is None:
    entity = MyModel(key_name=key_name, **kwds)
    entity.put()
  return entity
return db.run_in_transaction(txn)

Arguments:

key_name

The name for the key of the entity

**kwds

Keyword arguments to pass to the model class's constructor if an instance with the specified key name doesn't exist. Theparent argument is required if the desired entity has a parent.

Note: get_or_insert() does not accept an configuration object.

The method returns an instance of the model class that represents the requested entity, whether it existed or was created by the method. As with all datastore operations, this method can raise a TransactionFailedError if the transaction could not be completed.

Model.all(keys_only=False)

Returns a Query object that represents all entities for the kind corresponding to this model. Methods on the Query object can apply filters and sort orders to the query before it is executed. See Query for more information

Arguments:

keys_only

Whether the query should return full entities or just keys. Queries that return keys are faster and cost less CPU than queries that return full entities.

Model.gql(query_string, *args, **kwds)

Performs a GQL query over instances of this model.

Arguments:

query_string

The part of the GQL query following SELECT * FROM model (which is implied by using this class method).

*args

Positional parameter bindings, similar to the GqlQuery constructor.

**kwds

Keyword parameter bindings, similar to the GqlQuery constructor.

s = Story.gql("WHERE title = :1", "Little Red Riding Hood")

s = Story.gql("WHERE title = :title", title="Little Red Riding Hood")

The return value is a GqlQuery object, which can be used to access the results.

Model.kind()

Returns the kind of the model, usually the name of the Model subclass.

Model.properties()

Returns a dictionary of all of the properties defined for this model class.

Instance Methods

Model instances have the following methods:

key()

Returns the datastore Key for this model instance.

A model instance does not have a key until it is put() in the datastore. Calling key() before the instance has a key raises aNotSavedError.

put()

Stores the model instance in the datastore. If the model instance is newly created and has never been stored, this method creates a new data entity in the datastore. Otherwise, it updates the data entity with the current property values.

The method returns the Key of the stored entity.

Arguments:

config

A configuration object for the API call.

delete()

Deletes the model instance from the datastore. If the instance has never been put(), the delete raises a NotSavedError.

Arguments:

config

A configuration object for the API call.

is_saved()

Returns True if the model instance has been put() into the datastore at least once.

This method only checks that the instance has been stored at least once since it was created. It does not check if the instance's properties have been updated since the last time it was put().

dynamic_properties()

Returns a list of the names of all of the dynamic properties defined for this model instance. This only applies to instances ofExpando classes. For non-Expando model instances, this returns an empty list.

parent()

Returns a model instance for the parent entity of this instance, or None if this instance does not have a parent.

parent_key()

Returns the Key of the parent entity of this instance, or None if this instance does not have a parent.

to_xml()

Returns an XML representation of the model instance.

Property values conform to the Atom and GData specifications.

Disallowed Property Names

The datastore and its API impose several restrictions on names for entity properties and model instance attributes.

The datastore reserves all property names that begin and end with two underscores (__*__). A datastore entity cannot have a property with such a name.

The Python model API ignores all attributes on a Model or Expando that begin with an underscore (_). Your application can use these attributes to associate data with the model objects that is not saved to the datastore.

Lastly, the Python model API uses object attributes to define properties of a model, and by default the datastore entity properties are named after the attributes. Because the Model class has several properties and methods for other purposes, those attributes cannot be used for properties in the Python API. For example, a Model cannot have a property accessed with the attribute key.

However, a property can specify a different name for the datastore than the attribute name by giving a name argument to the property constructor. This allows the datastore entity to have a property name similar to a reserved attribute in the Model class, and use a different attribute name in the class.

class MyModel(db.Model):
    obj_key = db.StringProperty(name="key")

The following attribute names are reserved by the Model class in the Python API:

all app copy delete entity entity_type fields from_entity get gql instance_properties

is_saved key key_name kind parent parent_key properties put setdefault to_xml update

The Expando Class

The Expando class is a superclass for data model definitions whose properties are determined dynamically. An Expando model can have a combination of fixed properties similar to Model and dynamic properties assigned to an entity at run-time.

Expando is provided by the google.appengine.ext.db module.

• Introduction
• Expando()

Expando is a subclass of Model, and inherits its class and instance methods from that class. The Expando class does not define or override any methods.

Introduction

An Expando model can have fixed properties and dynamic properties. Fixed properties behave similar to properties of a Model, and are similarly defined in the Expando model class using class attributes. Dynamic properties are created when they are assigned values on the instance. Two instances of the same Expando class can have different sets of dynamic properties, and can even have dynamic properties with the same name but different types. Dynamic properties are always optional, and have no default value: They don't exist until they are assigned a value.

Dynamic properties cannot use Property instances to perform validation, set defaults or apply automatic logic to values. Dynamic properties simply store values of the supported datastore types. See Types and Property Classes.

Also unlike fixed properties, dynamic properties cannot use a different name for the class attribute and the datastore property name. See Disallowed Property Names.

Tip: If you want to validate a dynamic property value using a Property class, you can instantiate the Property class and call itsvalidate() method on the value.

An Expando subclass can define fixed properties similar to a Model class. Fixed properties of an Expando behave similarly to properties of a Model. An Expando instance can have both fixed and dynamic properties.

import datetime
class Song(db.Expando):
    title = db.StringProperty()

crazy = Song(title='Crazy like a diamond',
             author='Lucy Sky',
             publish_date='yesterday',
             rating=5.0)

hoboken = Song(title='The man from Hoboken',
               author=['Anthony', 'Lou'],
               publish_date=datetime.datetime(1977, 5, 3))

crazy.last_minute_note=db.Text('Get a train to the station.')

An Expando instance's dynamic (non-fixed) properties can be deleted. To delete a dynamic property, an application deletes the instance's attribute:

del myobj.myprop

Constructor

The constructor of the Expando class is defined as follows:

class Expando(parent=None, key_name=None, **kwds)

A model class whose properties do not need to be defined in the class before use. Like Model, the Expando class must be subclassed to define the kind of the data entities.

Expando is a subclass of Model, and inherits or overrides its methods.

Arguments:

parent

The Model instance or Key instance for the entity that is the new entity's parent.

key_name

The name for the new entity. The name becomes part of the primary key. If None, a system-generated ID is used for the key.

The value for key_name must not start with a number, and must not be of the form __*__. If your application uses user-submitted data as datastore entity key names (such as an email address), the application should sanitize the value first, such as by prefixing it with a known string like "key:", to meet these requirements.

A key_name is stored as a Unicode string, with str values converted as ASCII text.

**kwds

Initial values for the instance's properties, as keyword arguments. Each name corresponds with an attribute of the new instance, and may either correspond with fixed properties defined in the Expando class, or be dynamic properties.

The PolyModel Class

The PolyModel class is the superclass for data model definitions that can themselves be superclasses for other data model definitions. A query produced from a PolyModel class can have results that are instances of the class or any of its subclasses.

PolyModel is provided by the google.appengine.ext.db.polymodel module.

• Introduction
• PolyModel()
• Class methods:
  • PolyModel.class_key()
  • PolyModel.class_name()

PolyModel is a subclass of Model, and inherits its class and instance methods from that class. The PolyModel class overrides several of Model's methods, but does not introduce any new interface elements.

Introduction

It is often useful to define data models as a classification hierarchy, much like how an object database can define one class of objects as a sub-class of another. Such a database can perform queries on objects of the parent class, and include objects of the sub-class in the results. The App Engine datastore does not support this kind of query natively, but you can implement it using a mechanism included with the Python SDK, the PolyModel class.

A model class derived from PolyModel can be the base class for other model classes. Queries created for these classes using theall() and gql() methods know to include instances of subclasses in the results.

Subclasses can define new properties not present on parent classes. However, subclasses cannot override property definitions of parent classes. (Doing so results in a DuplicateProperty error.)

For reference, here is the simple example from Entities and Models. Notice that the PolyModel class is provided by the packagegoogle.appengine.ext.db.polymodel.

from google.appengine.ext import dbfrom google.appengine.ext.db import polymodel
class Contact(polymodel.PolyModel):
    phone_number = db.PhoneNumberProperty()
    address = db.PostalAddressProperty()
class Person(Contact):
    first_name = db.StringProperty()
    last_name = db.StringProperty()
    mobile_number = db.PhoneNumberProperty()
class Company(Contact):
    name = db.StringProperty()
    fax_number = db.PhoneNumberProperty()

p = Person(phone_number='1-206-555-9234',
           address='123 First Ave., Seattle, WA, 98101',
           first_name='Alfred',
           last_name='Smith',
           mobile_number='1-206-555-0117')
p.put()

c = Company(phone_number='1-503-555-9123',
            address='P.O. Box 98765, Salem, OR, 97301',
            name='Data Solutions, LLC',
            fax_number='1-503-555-6622')
c.put()
for contact in Contact.all():
  # Returns both p and c.
  # ...
for person in Person.all():
  # Returns only p.
  # ...

Polymorphism is not a native feature of the datastore. Instead, polymorphism is implemented in the PolyModel class itself. All entities created from PolyModel subclasses are stored in the datastore with the same kind, which is the name of the root class (e.g. Animal). Each object stores its class hierarchy as a multi-valued property of the entity named 'class'. When the app creates a query using a PolyModel class's all() or gql() method, the query includes a filter on the 'class' property that limits the results to entities created from the class or any subclass.

Because PolyModel uses a property of the entity to store class information, indexes for polymorphic queries must accommodate the 'class' property. The implied filter is an equality filter, and can be combined with other equality filters and inequality filters on other properties.

Note: PolyModel uses just the names of the classes in the 'class' property, not full paths. It's possible to create class hierarchies with multiple nodes of the same name, such as A → B and A → C → B. A query for one will return entities of both. Similarly, queries for A → B → C and A → C → B are functionally identical. It's best to avoid creating a single class hierarchy with multiple nodes of the same name.

PolyModel does not support overriding property model definitions in subclasses. If a subclass tries to redefine a property that is defined on a superclass, the class definition raises a DuplicatePropertyError.

PolyModel supports multiple inheritance, including inheriting from multiple classes that share a superclass ("diamond" inheritance). A class cannot inherit from two classes that each define a property model definition for the same property (this would raise aDuplicatePropertyError). However, a class can inherit from two classes that inherit the same property model definition from the same superclass.

PolyModel does not support dynamic properties, like Expando does. There is not an equivalent of PolyModel for Expando.

Constructor

The constructor of the PolyModel class is defined as follows:

class PolyModel(parent=None, key_name=None, **kwds)

A model class that can be a superclass to other model classes, and whose queries can include instances of subclasses as results. Like Model, the PolyModel class must be subclassed to define the kind of the data entities.

PolyModel is a subclass of Model, and inherits or overrides its methods.

Arguments:

parent

The Model instance or Key instance for the entity that is the new entity's parent.

key_name

The name for the new entity. The name becomes part of the primary key. If None, a system-generated ID is used for the key.

The value for key_name must not start with a number, and must not be of the form __*__. If your application uses user-submitted data as datastore entity key names (such as an email address), the application should sanitize the value first, such as by prefixing it with a known string like "key:", to meet these requirements.

A key_name is stored as a Unicode string, with str values converted as ASCII text.

**kwds

Initial values for the instance's properties, as keyword arguments. Each name corresponds with an attribute of the new instance, and must correspond with fixed properties defined in the PolyModel class.

Class Methods

In addition to the class methods defined by the Model class, the PolyModel class provides the following class methods:

PolyModel.class_key()

Returns the name of the class and the names of all parent classes for the class, as a tuple.

PolyModel.class_name()

Returns the name of the class. A class can override this method if the name of the Python class changes, but entities should continue using the original class name.

The Property Class

The Property class is the superclass of property definitions for data models. A Property class defines the type of a property's value, how values are validated, and how values are stored in the datastore.

Property is provided by the google.appengine.ext.db module.

• Introduction
• Property()
• Class attributes:
  • Property.data_type
• Instance methods:
  • default_value()
  • validate()
  • empty()
  • get_value_for_datastore()
  • make_value_from_datastore()

Introduction

A property class describes the value type, default value, validation logic and other features of a property of a Model. Each property class is a subclass of the Property class. The datastore API includes property classes for each of the datastore value types, and several others that provide additional features on top of the datastore types. See Types and Property Classes.

A property class can accept configuration from arguments passed to the constructor. The base class constructor supports several arguments that are typically supported in all property classes, including all those provided in the datastore API. Such configuration can include a default value, whether or not an explicit value is required, a list of acceptable values, and custom validation logic. See the documentation for a specific property type for more information on configuring the property.

A property class defines the model for a datastore property. It does not contain the property value for a model instance. Instances of the Property class belong to the Model class, not instances of the class. In Python terms, property class instances are "descriptors" that customize how attributes of Model instances behave. See the Python documentation for more information about descriptors.

Constructor

The constructor of the Property base class is defined as follows:

class Property(verbose_name=None, name=None, default=None, required=False, validator=None, choices=None, indexed=True)

The superclass of model property definitions.

Arguments:

verbose_name

A user-friendly name of the property. This must always be the first argument to a property constructor. The djangoformslibrary uses this to make labels for form fields, and others can use it for a similar purpose.

name

The storage name for the property, used in queries. This defaults to the attribute name used for the property. Because model classes have attributes other than properties (which cannot be used for properties), a property can use name to use a reserved attribute name as the property name in the datastore, and use a different name for the property attribute. SeeDisallowed Property Names for more information.

default

A default value for the property. If the property value is never given a value, or is given a value of None, then the value is considered to be the default value.

Note: Model class definitions are cached along with the rest of the application code. This includes caching default values for properties. Do not set a default in the model definition with data specific to the request (such asusers.get_current_user()). Instead, define an __init__() method for the Model class that initializes the property values.

required

If True, the property cannot have a value of None. A model instance must initialize all required properties in its constructor so that the instance is not created with missing values. An attempt to create an instance without initializing a required property, or an attempt to assign None to a required property, raises a BadValueError.

A property that is both required and has a default value uses the default value if one is not given in the constructor. However, the property cannot be assigned a value of None, and there is no automatic way to restore the default value after another value has been assigned. You can always access the property's default attribute to get this value and assign it explicitly.

validator

A function that should be called to validate the property's value when the value is assigned. The function takes the value as its only argument, and raises an exception if the value is invalid. The given validator is called after other validation has taken place, such as the check that a required property has a value. When a non-required property is not given a value, the validator is called with argument None.

choices

A list of acceptable values for the property. If set, the property cannot be assigned a value not in the list. As with required and other validation, a model instance must initialize all properties with choices so that the instance is not created with invalid values. If choices is None, then all values that otherwise pass validation are acceptable.

indexed

Whether this property should be included in the built-in and developer-defined indexes. If False, entities written to the datastore will never be returned by queries that sort or filter on this property, similar to Blob and Text properties.

Note: Every indexed property adds a small amount of overhead, CPU cost, and latency to put() and delete() calls. If you'll never need to filter or sort on a property, consider using indexed=False to avoid that overhead. Be careful, though! If you decide later that you want the property indexed after all, changing it back to indexed=True will only affect writes from that point onward. Entities that were originally written with indexed=False will not be re-indexed.

Class Attributes

Subclasses of the Property class define the following class attribute:

data_type

The Python data type or class the property accepts as a Python-native value.

Instance Methods

Instances of Property classes have the following methods:

default_value()

Returns the default value for the property. The base implementation uses the value of the default argument passed to the constructor. A property class could override this to provide special default value behavior, such as DateTimeProperty's auto-now feature.

validate(value)

The complete validation routine for the property. If value is valid, it returns the value, either unchanged or adapted to the required type. Otherwise it raises an appropriate exception.

The base implementation checks that value is not None if required (the required argument to the base Property constructor), the value is one of the valid choices if the property was configured with choices (the choices argument), and the value passes the custom validator if any (the validator argument).

The validation routine is called when a model using the property type is instantiated (with default or initialized values), and when a property of the type is assigned a value. The routine should not have side effects.

empty(value)

Returns True if value is considered an empty value for this property type. The base implementation is equivalent to not value, which is sufficient for most types. Other types, like a Boolean type, can override this method with a more appropriate test.

get_value_for_datastore(model_instance)

Returns the value that ought to be stored in the datastore for this property in the given model instance. The base implementation simply returns the Python-native value of the property in the model instance. A property class can override this to use a different data type for the datastore than for the model instance, or to perform other data conversion just prior to storing the model instance.

make_value_from_datastore(value)

Returns the Python-native representation for the given value from the datastore. The base implementation simply returns the value. A property class can override this to use a different data type for the model instance than for the datastore.

The Query Class

The Query class is a datastore query interface that uses objects and methods to prepare queries.

Query is provided by the google.appengine.ext.db module.

• Introduction
• Query()
• Instance methods:
  • filter()
  • order()
  • ancestor()
  • get()
  • fetch()
  • count()
  • cursor()
  • with_cursor()

Introduction

An application creates a Query object by calling either the Model class's all() class method or the Query constructor.

class Song(db.Model):
    title = db.StringProperty()
    composer = db.StringProperty()
    date = db.DateTimeProperty()

query = Song.all()

query = db.Query(Song)

ancestor = Song.get_by_key_name('my_song')
query = db.Query(ancestor)

Without modification, the object represents a query for all entities with a specific kind or ancestor (for "kindless" queries). Method calls customize the query with property conditions (filter()), ancestor conditions (ancestor()), and ordering (order()). For convenience, these methods return self so that they can combined in a single statement.

query.filter('title =', 'Imagine')
query.order('-date')
query.ancestor(key)

query.filter('title =', 'Imagine').order('-date').ancestor(key)

The application executes the query in one of two ways:

by calling the fetch() method. This performs a single call to the datastore to fetch results, up to the specified number of results. The Query object does not cache results, so calling fetch() a second time re-executes the query.

results = query.fetch(limit=5)
for song in results:
    print song.title

by treating the Query object as an iterable. The iterator retrieves results from the datastore in small batches, allowing for the app to stop iterating on results to avoid fetching more than is needed. Iteration stops when all of the results that match the query have been retrieved. As with fetch(), the iterator interface does not cache results, so creating a new iterator from the Query object re-executes the query.

for song in query:
    print song.title

See also GqlQuery, a query class that uses a SQL-like query language.

Note: The index-based data structures and algorithms that power datastore queries do not support some kinds of queries. SeeQueries and Indexes: Restrictions on Queries for more information.

Constructor

The constructor of the Query class is defined as follows:

class Query(model_class, keys_only=False)

A datastore query interface that uses objects and methods to prepare queries.

The Query instance returned by the constructor represents a query for all entities of the kind. The instance methods filter(),order() and ancestor() apply criteria to the query to filter or order the results.

Arguments:

model_class

The class of the Model (or Expando) that represents the datastore entity kind for the query.

keys_only

Whether the query should return full entities or just keys. Queries that return keys are faster and cost less CPU than queries that return full entities.

Instance Methods

Instances of the Query class have the following methods:

filter(property_operator, value)

Adds a property condition filter to the query. The query returns only entities with properties that meet all of the conditions.

Arguments:

property_operator

A string containing the property name, and an optional comparison operator. The name and the operator must be separated by a space, as in: age >. The following comparison operators are supported: < <= = >= > != IN. If the operator is omitted from the string (the argument is just the property name), the filter uses the = operator.

value

The value to use in the comparison on the right-hand side of the expression. Its type should be the value data type for the property being compared. See Types and Property Classes.

query.filter('height >', 42).filter('city = ', 'Seattle')

query.filter('user = ', users.get_current_user())

order(property)

Adds an ordering for the results. Results are ordered starting with the first order added.

Arguments:

property

A string, the name of the property to order. To specify that the order ought to be in descending order, precede the name with a hyphen (-). Without a hyphen, the order is ascending.

# Order by last name, alphabetical:
query.order('last_name')
# Order tallest to shortest:
query.order('-height')

ancestor(ancestor)

Adds an ancestor condition filter to the query. The query returns only entities with the given entity as an ancestor (anywhere in its path).

Arguments:

ancestor

A Model instance or Key instance representing the ancestor.

get()

Executes the query, then returns the first result, or None if the query returned no results.

get() fetches a "limit" of one result, at most.

Arguments:

config

A configuration object for the API call.

fetch(limit, offset=0)

Executes the query, then returns the results.

The limit and offset arguments control how many results are fetched from the datastore, and how many are returned by the fetch() method:

• The datastore fetches offset + limit results.
• The first offset results are skipped by the datastore and not sent to the application.
• The fetch() method then returns the rest (limit results).

Note: The query has performance characteristics that correspond linearly with the offset amount plus the limit amount.

Arguments:

limit

The number of results to return. Fewer than limit results may be returned if not enough results are available that meet the criteria.

limit is a required argument. To get every result from a query when the number of results is unknown, use the Query object as an iterable instead of using the fetch() method.

offset

The number of results to skip.

config

A configuration object for the API call.

The return value is a list of model instances or keys, possibly an empty list.

run(config=None)

Runs the query, and returns an iterable of the results. This allows you to use an configuration object for the call, and access results with an iterable interface. If you are not using an configuration object, you can just use the Query (or GqlQuery) object as the iterable to execute the query.

Arguments:

config

A configuration object for the API call.

count(limit)

Returns the number of results this query fetches.

count() is somewhat faster than retrieving all of the data by a constant factor, but the running time still grows with the size of the result set. It's best to only use count() in cases where the count is expected to be small, or specify a limit.

count() has no maximum limit. If you don't specify a limit, the datastore continues counting until it finishes counting ortimes out.

Arguments:

limit

The maximum number of results to count.

config

A configuration object for the API call.

cursor()

Returns an encoded cursor that represents the location in the result set after the last result fetched. A future invocation of the same query can provide this cursor using with_cursor() to start fetching results from this location.

You must fetch results (using either fetch() or the iterator interface) before you can get a cursor.

The cursor is a base64-encoded string. It is safe to use in HTTP GET and POST parameters, and can also be stored in the datastore or memcache.

Note: Not all queries are compatible with cursors. See Query Cursors for more information.

with_cursor(start_cursor, end_cursor=None)

Tells the datastore to start returning results from the location associated with the given encoded start cursor. If present, stops returning results at the end cursor. An app gets the cursor for a location in a list of results by calling cursor() after fetching results.

The query must be identical to the query that generated the cursor, including the kind, property filters, ancestor filters, and sort orders.

The app must call this method before fetching any results (i.e. before executing the query).

Arguments:

start_cursor

The base64-encoded cursor returned by the cursor() method in a previous invocation of the query. Indicates where to start the query.

end_cursor

The base64-encoded cursor returned by the cursor() method in a previous invocation of the query. Indicates where to end the query.

The GqlQuery Class

The GqlQuery class is a datastore query interface that uses the App Engine query language GQL.

GqlQuery is provided by the google.appengine.ext.db module.

• Introduction
• GqlQuery()
• Instance methods:
  • bind()
  • count()
  • cursor()
  • fetch()
  • get()
  • with_cursor()

Introduction

GQL is a SQL-like query language suitable for querying the App Engine datastore. For a complete discussion of the GQL syntax and features, see the GQL Reference.

The GqlQuery constructor takes as an argument a complete GQL statement beginning with SELECT ... FROM model-name. Values in WHERE clauses can be string or number literals, or can use parameter binding for values. Bound parameters can be bound initially using positional or keyword arguments to the constructor.

query = GqlQuery("SELECT * FROM Song WHERE composer = 'Lennon, John'")

query = GqlQuery("SELECT __key__ FROM Song WHERE composer = :1", "Lennon, John")

query = GqlQuery("SELECT * FROM Song WHERE composer = :composer", composer="Lennon, John")

For convenience, Model and Expando classes have a gql() method that returns a GqlQuery instance. This method takes a GQL query string without the SELECT ... FROM model-name, which is implied.

query = Song.gql("WHERE composer = 'Lennon, John'")

As with the Query class, the application executes the query and accesses results either by calling the fetch() method, or by treating the GqlQuery object as an iterable. See the Query documentation for more information.

There is one difference between how Query and GqlQuery access results: If the GQL query includes a LIMIT clause or an OFFSETclause, results are retrieved as with the equivalent fetch() method, even if the iterator interface is used to access the results. When a GqlQuery containing LIMIT or OFFSET is used as an iterable, one call is made to the datastore to fetch all of the results, and the iterator returns each of the results from memory.

for song in q:
    print song.title

See also Query, a query class that uses objects and methods to prepare queries instead of GQL.

Note: The index-based data structures and algorithms that power datastore queries do not support some kinds of queries. SeeQueries and Indexes: Restrictions on Queries for more information.

Constructor

The constructor of the GqlQuery class is defined as follows:

class GqlQuery(query_string, *args, **kwds)

A query object using the App Engine query language GQL.

Arguments:

query_string

A complete GQL statement.

*args

Positional parameter bindings.

**kwds

Keyword parameter bindings.

Instance Methods

A GqlQuery instance has the following methods:

bind(*args, **kwds)

Re-binds parameters for the query. The new query is executed the first time results are accessed after parameters have been re-bound.

Re-using the GqlQuery object with new parameters is faster than building a new GqlQuery object because re-binding does not require that the query string be parsed again.

Arguments:

*args

The new positional parameter bindings.

**kwds

The new keyword parameter bindings.

count(limit=1000)

Returns the number of results this query fetches.

count() is somewhat faster than retrieving all of the data by a constant factor, but the running time still grows with the size of the result set. It's best to only use count() in cases where the count is expected to be small, or specify a limit.

Arguments:

limit

The maximum number of results to count, with a default of 1,000. The default value does not override the LIMIT clause in the GQL query statement: If limit is not specified in count(), but the GQL does specify a limit, then count() counts up to the GQL limit.

If limit is specified, then count() counts up to that limit.

If limit is specified as None, then count() continues until it finishes counting or times out.

cursor()

Returns an encoded cursor that represents the location in the result set after the last result fetched. A future invocation of the same query can provide this cursor using with_cursor() to start fetching results from this location.

You must fetch results (using either fetch() or the iterator interface) before you can get a cursor.

The cursor is a base64-encoded string. It is safe to use in HTTP GET and POST parameters, and can also be stored in the datastore or memcache.

Note: Not all queries are compatible with cursors. See Query Cursors for more information.

fetch(limit, offset=0)

Executes the query, then returns the results.

The limit and offset arguments control how many results are fetched from the datastore, and how many are returned by the fetch() method:

• The datastore fetches offset + limit results.
• The first offset results are skipped by the datastore and not sent to the application.
• The fetch() method then returns the rest (limit results).

Note: The query has performance characteristics that correspond linearly with the offset amount plus the limit amount.

Arguments:

limit

The number of results to return. This value overrides the LIMIT clause in the GQL query statement, if any. Fewer than limitresults may be returned if not enough results are available that meet the criteria.

limit is a required argument. To get every result from a query when the number of results is unknown, use the GqlQuery object as an iterable instead of using the fetch() method.

offset

The number of results to skip. This value overrides the OFFSET clause (or the offset in a LIMIT clause) in the GQL query statement, if any.

The return value is a list of model instances, possibly an empty list.

get()

Executes the query, then returns the first result, or None if the query returned no results.

get() implies a "limit" of 1, and overrides the LIMIT clause of the GQL query, if any. At most 1 result is fetched from the datastore.

with_cursor(start_cursor, end_cursor=None)

Tells the datastore to start returning results from the location associated with the given encoded start cursor. If present, stops returning results at the end cursor. An app gets the cursor for a location in a list of results by calling cursor() after fetching results.

The query must be identical to the query that generated the cursor, including the kind, property filters, ancestor filters, and sort orders.

The app must call this method before fetching any results (i.e. before executing the query).

Arguments:

start_cursor

The base64-encoded cursor returned by the cursor() method in a previous invocation of the query. Indicates where to start the query.

end_cursor

The base64-encoded cursor returned by the cursor() method in a previous invocation of the query. Indicates where to end the query.

Metadata Queries Python Reference

Content and Format of Returned Entities

__namespace__

For every namespace N of the application, returns an entity with a key of type __namespace__ and name N (if N is not '') or id 1 (if N is '')

__kind__

For every kind K of the application in the current namespace, returns an entity with a key of type __kind__ and name K.

__property__

For every property P of kind K in the current namespace, returns an entity with the following:

A key of type __property__ and name P, with a parent of type __kind__ and name K

A property named property_representation containing a list of strings, where the list contains representation R if some entity of type K has a property P with representation R (read more about how property classes map to representations).

All three metadata kinds support filters by __key__, and are implicitly ordered by ascending __key__ (order by ascending__key__ can also be explicitly requested).

The __property__ kind supports ancestor queries, where the ancestor is a __kind__ or __property__ key.

The google.appengine.ext.db.metadata package defines Namespace, Kind and Property models for the metadata query kinds.

Class Reference

class Namespace()

The Namespace class allows you to gather metadata across namespaces.

Properties

namespace_name

Returns the name of the namespace specified by this entity's key.

Methods

key_for_namespace(namespace)

Returns the __namespace__ key for the specified namespace.

Arguments:

namespace

The namespace whose key is requested.

key_to_namespace(key)

Returns the namespace specified by a given __namespace__ key.

Arguments:

key

The key whose name is requested.

class Kind()

Returns entities whose key is the name of the kind, and with no properties.

Properties

kind_name

Returns the kind name specified by this entity's key.

Methods

key_for_kind(kind)

Returns the __kind__ key for the specified kind.

Arguments:

kind

The kind whose key is requested.

key_to_kind(key)

Returns the kind specified by a given __kind__ key.

Arguments:

key

The key whose kind is requested.

class Property()

Returns the properties of kinds and keys for this entity.

Properties

kind_name

Returns the property name specified by this entity's key.

property_name

Returns the property name specified by this entity's key.

Methods

key_for_kind(kind)

Returns the parent key for __property__ keys of the specified kind.

Arguments:

kind

The kind whose parent key is requested.

key_for_property(kind, property)

Returns the __property__ key for the specified property and kind.

Arguments:

kind

The kind whose key is requested.

property

The property whose key is requested.

key_to_kind(key)

Returns the kind specified by a given __property__ key.

Arguments:

key

The key whose kind name is requested.

key_to_property(key)

Returns the property specified by a given __property__ key, or None if the key specified only a kind.

Arguments:

key

The key whose property name is requested.

The Key Class

An instance of the Key class represents a unique key for a datastore entity.

Key is provided by the google.appengine.ext.db module.

• Introduction
• Key()
• Class methods:
  • Key.from_path()
• Instance methods:
  • app()
  • has_id_or_name()
  • id()
  • id_or_name()
  • kind()
  • name()
  • namespace()
  • parent()

Introduction

Every model instance that has been stored in the datastore has a unique key that represents the object. The key() method of a model instance returns the Key object for the instance. If the instance has never been put() in the datastore, key() raises a NotSavedError.

An application can retrieve a model instance for a given Key using the get() function.

Key instances can be values for datastore entity properties, including Expando dynamic properties and ListProperty members. TheReferenceProperty model provides features for Key property values such as automatic dereferencing.

Constructor

class Key(encoded=None)

A unique key for a datastore object.

A key can be encoded to a string by passing the Key object to str() (or calling the object's __str__() method). A string-encoded key is an opaque value using characters safe for including in URLs. The string-encoded key can be converted back to a Key object by passing it to the Key constructor (the encoded argument).

Note: A string-encoded key can be converted back to the raw key data. This makes it easy to guess other keys when one is known. While string-encoded key values are safe to include in URLs, an application should only do so if key guessability is not an issue.

encoded

The str form of a Key instance to convert back into a Key.

Class Methods

The Key class provides the following class method:

Key.from_path(kind, id_or_name, parent=none, namespace=None, **kwds)

Builds a new Key object from an ancestor path of one or more entity keys.

A path represents the hierarchy of parent-child relationships for an entity. Each entity in the path is represented the entity's kind, and either its numeric ID or its key name. The full path represents the entity that appears last in the path, with its ancestors (parents) as preceding entities.

For example, the following call creates a key for an entity of kind Address with the numeric ID 9876 whose parent is an entity of kind User with the named key 'Boris':

k = Key.from_path('User', 'Boris', 'Address', 9876)

For more information about paths, see Keys and Entity Groups.

Arguments:

kind

The entity kind, represented as a string or a Unicode string.

id_or_name

The id, specified as a string or long. It cannot be the number 0.

namespace=None

The namespace to set for this Key only. If you supply a namespace here, it overrides the current namespace set in thenamespace_manager. If None, the API uses the current namespace from namespace_manager.get_namespace.

parent=None

Optional parent key. If not supplied, defaults to None.

**kwds

Keyword arguments. The method supports one keyword argument, parent, which specifies a parent entity to prepend to the given path. Its value is the parent's Key.

Instance Methods

Key instances have the following methods:

app()

Returns the name of the application that stored the data entity.

has_id_or_name()

Returns True if the entity has either a name or a numeric ID.

id()

Returns the numeric ID of the data entity, as an integer, or None if the entity does not have a numeric ID.

id_or_name()

Returns the name or numeric ID of the data entity, whichever it has, or None if the entity has neither a name nor a numeric ID.

kind()

Returns the kind of the data entity, as a string.

name()

Returns the name of the data entity, or None if the entity does not have a name.

namespace()

Returns the namespace of the data entity. If the entity does not have a current namespace, this method returns the current namespace set in the namespace_manager.

parent()

Returns the Key of the data entity's parent entity, or None if the entity has no parent.

Functions

The google.appengine.ext.db package provides the following functions:

allocate_ids(model, count)

Allocates a batch of IDs in the datastore for a datastore kind and parent combination.

IDs allocated in this manner will not be used by the datastore's automatic ID sequence generator and may be used in Keyswithout conflict..

Arguments:

model

The model key for which to allocate an ID batch. This is a regular Key but only the parent and kind of the key are necessary to determine which ID sequence to use.

count

The number of IDs to allocate.

Returns a tuple of the first and last IDs that it allocates. For example, if you allocated 10 IDs using this function you would get a return in the format (1, 10), not a full list of created IDs.

Example of allocating and using IDs:

# allocate for MyModel without an instance
handmade_key = db.Key.from_path('MyModel', 1)
first_batch = db.allocate_ids(handmade_key, 10)
first_range = range(first_batch[0], first_batch[1] + 1)
# or allocate using an existing key
model_instance = MyModel.all().get()
second_batch = db.allocate_ids(model_instance.key(), 10)
second_range = range(second_batch[0], second_batch[1] + 1)
# and then use them! woo!
my_id = second_range.pop(0)
new_key = db.Key.from_path('MyModel', my_id)
new_instance = MyModel(key=new_key)
new_instance.put()
assert new_instance.key().id() == my_id
# the datastore will not assign ids in first_batch or second_batch
another_instance = MyModel()
another_instance.put()
assert another_instance.key().id() not in first_rangeassert another_instance.key().id() not in second_range
    

allocate_ids_async(model, count)

Asynchronously allocates a batch of IDs in the datastore for a datastore kind and parent combination. Identical to allocate_ids(), but returns an asynchronous object. You can call get_result() on the return value to block on the call and return the result.

Arguments:

model

A db.Model instance, Key, or string to serve as a template specifying the ID sequence in which to allocate IDs. Returned ids should only be used in entities with the same parent (if any) and kind as this key.

count

The number of IDs to allocate.

Returns a tuple of the first and last IDs that it allocates. For example, if you allocated 10 IDs using this function you would get a return value in the format (1, 10), not a full list of created IDs.

allocate_id_range(model, start, end, **kwargs)

Allocates a range of IDs with specific endpoints. Once these IDs have been allocated, you can manually assign them to newly created entities.

The datastore's automatic ID allocator never assigns a key belonging to an existing entity to a new entity. As a result, entities written to the given key range will never be overwritten. However, writing entities with manually assigned keys in this range may overwrite existing entities (or new entities written by a separate request), depending on the key range state returned.

Use this method only if you have an existing numeric id range that you want to reserve (for example, bulk loading entities that already have IDs). If you don't care about which IDs you receive, use allocate_ids() instead.

Arguments:

model

A db.Model instance, Key, or string to serve as a template specifying the ID sequence in which to allocate IDs. Returned ids should only be used in entities with the same parent (if any) and kind as this key.

start

The first ID to allocate, a number.

end

The last ID to allocate, a number.

Returns one of (KEY_RANGE_EMPTY, KEY_RANGE_CONTENTION, KEY_RANGE_COLLISION). If not KEY_RANGE_EMPTY, this represents a potential issue with using the allocated key range.

create_config(deadline=None, on_completion=None, read_policy=STRONG_CONSISTENCY)

Creates a configuration object for setting the read policy and datastore call deadline for API calls. You can pass this configuration to most datastore calls using the config=... argument, for example:

# Faster get but may retrive stale data.
entity = db.get(key, config=db.create_config(read_policy=db.EVENTUAL_CONSISTENCY))

The datastore call deadline specifies an amount of time the application runtime environment will wait for the datastore to return a result before aborting with an error. By default, the runtime environment waits until the request handler deadline has elapsed. You can specify a shorter time to wait so your app can return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue.

The read policy determines whether read operations use strong consistency (the default) or eventual consistency. Note that thehigh replication datastore (HRD) cannot deliver strong consistency for queries across entity groups. For example, queries across entity groups may return stale results. In order to return strongly consistent query results in the HRD, use ancestor queries.

A read operation with eventual consistency may return sooner than one with strong consistency in the case of failure, and may be appropriate for some uses. However, if you use eventual consistency, recent writes may not immediately appear in query results.

See the function descriptions in this API reference, as well as methods of The Model Class and The Query Class, for information on which functions accept the config argument. See also Queries and Indexes: Setting the Read Policy and Datastore Call Deadline. Note that, in most cases, the config argument is accepted as a keyword argument only, and does not appear explicitly in the function signature.

Arguments:

deadline=None

An optional datastore call deadline, specified as a number of seconds. Accepts a float. If None, the call uses no deadline, and is only interrupted by the request handler deadline.

on_completion=None

An optional callback function. This method defaults to None, but if specified, it will be called with a UserRPC object as an argument when an RPC completes.

read_policy

The read policy, either db.STRONG_CONSISTENCY or db.EVENTUAL_CONSISTENCY. The default isdb.STRONG_CONSISTENCY, and strong consistency is used if a read operation is not passed an RPC object.

create_transaction_options(xg=false, retries=None)

Creates a TransactionOptions instance, to set options on transaction execution.Notice that for cross-group (XG) transaction, the xg param must be set to true. You supply the object returned by this function as the first argument to therun_in_transaction_options() function.

Arguments:

xg

Boolean specifying whether XG transactions are allowed. Must be true to run XG transactions. (Note that XG transactions are supported only for apps using HRD.) Raises a BadArgumentError if a non Boolean is used.

retries

Integer specifying the number of retries to attempt upon a failure from a transaction commit. If no default is supplied, the default datastore retries will be used; currently this is set to 3 retries.

Returns a TransactionOptions instance.

Example of creating the options for a subsequent cross-group transaction:

  from google.appengine.ext import db

  xg_on = db.create_transaction_options(xg=True)

  def my_txn():
    x = MyModel(a=3)
    x.put()
    y = MyModel(a=7)
    y.put()

  db.run_in_transaction_options(xg_on, my_txn)
    

delete(models)

Deletes one or more model instances from the datastore.

Arguments:

models

A model instance, a Key for an entity, or a list (or other iterable) of model instances or keys of entities to delete.

config

datastore_rpc.Configuration to use for this request, specified as a keyword argument.

As with Model.put(), if multiple keys are given, they may be in more than one entity group. See Keys and Entity Groups.

An exception will always be raised if any error occurs during the operation, even if some of the entities actually were deleted. If the call returns without raising an exception, then all of the entities were deleted successfully.

Note: Entities belonging to a single entity group may not be deleted in a single transaction, unless the delete is performed inside a datastore transaction.

delete_async(models)

Asynchronously deletes one or more model instances from the datastore. This method is identical to db.delete(), except it returns an asynchronous object. Call get_result() in the return value to block on the call.

Arguments:

models

A model instance, a Key for an entity, or a list (or other iterable) of model instances or keys of entities to delete.

config

datastore_rpc.Configuration to use for this request, specified as a keyword argument.

As with Model.put(), if multiple keys are given, they may be in more than one entity group. See Keys and Entity Groups.

This method returns an object that lets you block on the result of the call.

An exception will always be raised if any error occurs during the operation, even if some of the entities actually were deleted. If the call returns without raising an exception, then all of the entities were deleted successfully.

Note: Entities belonging to a single entity group may not be deleted in a single transaction, unless the delete_async is performed inside a datastore transaction.

get(keys)

Fetch the specific Model instance with the given key from the datastore. We support Key objects and string keys (we convert them to Key objects automatically).

Arguments:

keys

Key you wish to find within the datastore entity collection; or a string key; or a list of Keys or string keys.

config

The datastore_rpc.Configuration to use for this request.

If a single key was given, this method returns a Model instance associated with the key if the key exists in the datastore, otherwise None. If a list of keys was given, this method returns a list whose items are either a Model instance or None.

See also Model.get().

get_async(keys)

Asynchronously fetches the specified Model instance(s) from the datastore. Identical to db.get(), except it returns an asynchronous object. You can call get_result() on the return value to block and the call and get the results.

Arguments:

keys

A Key object or a list of Key objects.

config

The datastore_rpc.Configuration to use for this request.

If one Key is provided, the return value is an instance of the appropriate Model class, or None if no entity exists with the given Key. If a list of Keys is provided, the return value is a corresponding list of model instances, with None values when no entity exists for a corresponding Key.

See also Model.get().

get_indexes()

Returns a list of composite indexes belonging to the calling application.

Example of getting and using the indexes:

def get_index_state_as_string(index_state):
  return {db.Index.BUILDING:'BUILDING', db.Index.SERVING:'SERVING',
          db.Index.DELETING:'DELETING', db.Index.ERROR:'ERROR'} [index_state]
def get_sort_direction_as_string(sort_direction):
  return {db.Index.ASCENDING:'ASCENDING',
          db.Index.DESCENDING:'DESCENDING'}[sort_direction]

def dump_indexes():
  for index, state in db.get_indexes():
    print "Kind: %s" % index.kind()
    print "State: %s" % get_index_state_as_string(state)
    print "Is ancestor: %s" % index.has_ancestor()
    for property_name, sort_direction in index.properties():
      print "  %s:%s" % (property_name, get_sort_direction_as_string(sort_direction)) 

get_indexes_async()

Asynchronously returns a list of composite indexes belonging to the calling application.

is_in_transaction()

Returns a boolean indicating whether the current scope is executing in a transaction.

model_to_protobuf(model_instance)

Creates the "protocol buffer" serialization of a Model instance. A protocol buffer is Google's serialization format used for remote procedure calls, and can be useful for serializing datastore objects for backup and restore purposes.

Note: This method uses a different (older) format for protobuffers than the open sourced protocol buffer format. It is not compatible with the open source implementation.

Arguments:

model_instance

The instance of the Model class (or a subclass) to serialize.

Returns the protobuffer serialization of the object, as a byte string.

model_from_protobuf(pb)

Creates a Model instance based on a "protocol buffer" serialization, as returned by model_to_protobuf(). See that method for more information about protocol buffers.

Arguments:

pb

The protobuffer serialization, as returned by model_to_protobuf().

Returns an object of the appropriate kind class. If the kind class does not exist, raises a db.KindError. If the object is not valid according to the model, raises a db.BadValueError.

You can save the new object to the datastore just like any other Model instance, such as by calling its put() method. The object retains the key it had when the protocol buffer was created. If an object with that key already exists in the datastore, saving the deserialized object overwrites the existing object.

Note: If the object's key uses a system-assigned ID and that ID has not already been allocated for the given path and kind, the save will succeed, but the ID is not reserved. An object created in the future may be assigned that ID, and would overwrite the earlier object. For safety, only restore objects in the same application where they existed when they were serialized.

put(models)

Puts one or more model instances into the datastore.

Arguments:

models

A model instance or a list of model instances to store.

config

The datastore_rpc.Configuration to use for this request, specified as a keyword argument.

If multiple model instances are given, they may be in more than one entity group. See Keys and Entity Groups for more information.

An exception will always be raised if any error occurs during the operation, even if some of the entities actually were written. If the call returns without raising an exception, then all of the entities were written successfully.

Returns the Key object (if one model instance is given) or a list of Key objects (if a list of instances is given) that correspond with the stored model instances.

Note: Entities belonging to a single entity group may not be written in a single transaction, unless the put is performed inside a datastore transaction.

put_async(models)

Puts one or more model instances into the datastore. Identical to db.put(), except it returns an asynchronous object. You can callget_result() on the return value to block on the call and get the results.

Arguments:

models

A model instance or a list of model instances to store.

config

The datastore_rpc.Configuration to use for this request, specified as a keyword argument.

If multiple model instances are given, they may be in more than one entity group. See Keys and Entity Groups.

An exception will always be raised if any error occurs during the operation, even if some of the entities actually were written. If the call returns without raising an exception, then all of the entities were written successfully.

This method returns an asynchronous object containing the result of calling get_result().

Note: Entities belonging to a single entity group may not be written in a single transaction, unless the put_async is performed inside a datastore transaction.

query_descendants(model_instance)

Returns a query for all the descendants of a model instance.

Arguments:

model_instance

The model instance whose descendants you want to find.

run_in_transaction(function, *args, **kwargs)

Runs a function containing datastore updates in a single transaction. If any code raises an exception during the transaction, all datastore updates made in the transaction are rolled back.

Arguments:

function

The function to run in a datastore transaction.

*args

Positional arguments to pass to the function.

**kwargs

Keyword arguments to pass to the function.

If the function returns a value, run_in_transaction() returns the value to the caller.

If the function raises an exception, the transaction is rolled back. If the function raises a Rollback exception, the exception is not re-raised. For any other exception, the exception is re-raised to the caller.

The datastore uses optimistic locking and retries for transactions. If the transaction prepared by the function cannot be committed, run_in_transaction() calls the function again, retrying the transaction up to 3 times. (To use a different number of retries, use db.run_in_transaction_custom_retries().) Because the transaction function may be called more than once for a single transaction, the function should not have side effects, including modifications to arguments.

If the transaction cannot be committed, such as due to a high rate of contention, a TransactionFailedError is raised.

For more information about transactions, see Transactions.

def decrement(key, amount=1):
    counter = db.get(key)
    counter.count -= amount
    if counter.count < 0:    # don't let the counter go negative
        raise db.Rollback()
    db.put(counter)

q = db.GqlQuery("SELECT * FROM Counter WHERE name = :1", "foo")
counter = q.get()
db.run_in_transaction(decrement, counter.key(), amount=5)

run_in_transaction_custom_retries(retries, function, *args, **kwargs)

Runs a function containing datastore updates in a single transaction, retrying the transaction the given number of times in the event of contention. If any code raises an exception during the transaction, all datastore updates made in the transaction are rolled back.

Arguments:

retries

The maximum number of times to call the transaction function in the event of contention in the entity group (more than one user attempting to modify the group simultaneously).

function

The function to run in a datastore transaction.

*args

Positional arguments to pass to the function.

**kwargs

Keyword arguments to pass to the function.

Other than the ability to specify the number of retries, this function behaves identically to db.run_in_transaction().

run_in_transaction_options(options, function, *args, **kwargs)

Runs a function containing datastore updates in a single transaction using the passed-in TransactionOptions object. For cross-group (XG) transactions you must specify a TransactionOptions object with its XG parameter set to true, and your app must use the High Replication Datastore (HRD). If any code raises an exception during the transaction, all datastore updates made in the transaction are rolled back.

Arguments:

options

The TransactionOptions object containing the settings used by this transaction. To enable XG transactions, its xg parameter must be set to true.

function

The function to run in a datastore transaction.

*args

Positional arguments to pass to the function.

**kwargs

Keyword arguments to pass to the function.

If the function returns a value, run_in_transaction_options() returns the value to the caller.

If the function raises an exception, the transaction is rolled back. If the function raises a Rollback exception, the exception is not re-raised. For any other exception, the exception is re-raised to the caller.

The datastore uses optimistic locking and retries for transactions. If the transaction prepared by the function cannot be committed, run_in_transaction_options() calls the function again, retrying the transaction up to up to the number of retries specified by the TransactionOptions object. Because the transaction function may be called more than once for a single transaction, the function should not have side effects, including modifications to arguments. If the transaction cannot be committed, such as due to a high rate of contention, a TransactionFailedError is raised.

For more information about transactions, see Transactions.

This snippet shows how to use this function to run a cross-group transaction.

  from google.appengine.ext import db

  xg_on = db.create_transaction_options(xg=True)

  def my_txn():
    x = MyModel(a=3)
    x.put()
    y = MyModel(a=7)
    y.put()

  db.run_in_transaction_options(xg_on, my_txn)
    

Exceptions

The google.appengine.ext.db package provides the following exception classes:

exception Error()

This is the base class for all exceptions in this package.

exception BadArgumentError()

A bad argument was given to a query method.

exception BadFilterError()

A filter string in the query is invalid.

exception BadKeyError()

The provided key string is not a valid key.

exception BadPropertyError()

The property could not be created because its name is not a string.

exception BadQueryError()

The query string is not a valid query.

exception BadRequestError()

The request to the datastore service has one or more invalid properties. This is possible if a subclass of Model overrides some methods (such as kind()) with an incorrect implementation.

exception BadValueError()

The property could not be assigned a value because the value is invalid for the property type.

exception ConfigurationError()

A property is not configured correctly.

exception DuplicatePropertyError()

A model definition has more than one property with the same name.

exception InternalError()

There was an error internal to the datastore service. This exception does not necessarily mean that the operation failed.

exception KindError()

The application attempted to use a data entity with a model class that does not match the entity.

exception NeedIndexError()

Raised when the SDK does not find a matching index for a query that requires one. Check the Administration Console to manage your indexes and your index.yaml file.

exception NotSavedError()

An action was performed that requires the object to have been saved (put) to the datastore, but the object is not saved.

exception PropertyError()

The referenced model property does not exist on the data object.

exception ReferencePropertyResolveError()

The model referenced by a ReferenceProperty does not exist. See References.

exception ReservedWordError()

A model defines a property whose name is disallowed. See Disallowed Property Names.

exception Rollback()

Indicates that a function in a transaction wants to roll back the transaction instead of committing it. Any uncaught exception in a transaction will cause the transaction to roll back. This exception class is for convenience, when a function wants to roll back and no other exception applies.

exception Timeout()

Raised when the datastore operation exceeds the maximum amount of time allowed for datastore operations. This exception does not necessarily mean that the operation failed.

exception TransactionFailedError()

The transaction or datastore operation could not be committed, even after retrying. This is usually caused by a high rate of contention: The data is being updated by many other application instances simultaneously, and this instance could not commit its transaction within a fixed number of retries. See Transactions.

The google.appengine.runtime.apiproxy_errors package provides the following exception classes:

exception CapabilityDisabledError()

Indicates that a datastore api call was not performed as that particular datastore functionality is not available.