Understanding Basic Data Storage Options on Google App Engine

The default storage option available on the App Engine today is the Data Store, a proprietary Google database. In this post we will talk about the Google Data Store and some of the basic aspects of storage, features available and the key benefits and challenges in working with the Data Store.

What is the Google Data Store ?

Google Data Store is a scalable data storage solution for the web applicable, primarily optimized for read and query performance. The key aspect of the Data Store is that it is Entity based storage which means all data is stored and retrieved as entities and attributes. Here are the key features of the Data Store:

• Google Data Store refers to all data in the form of Entities
• Each entity has one more named properties
• A property can reference another entity
• Supports multiple operations in a single transaction
• It is not a relational database
• It transparently handles scalability, performance, replication, load balancing while all these aspects are abstracted from the application developer by usage of APIs

Creating Entities

Creating entities involves definining a model of the entity as a class in either Python/Java or other App Engine languages and creating the entity of the class in the constructor by calling the put() method of the API in the class.

Here is an example of a Employee entity in Python

  class Employee(db.Model):  
  name = db.StringProperty(required=True) 
  role = db.StringProperty(required=True, 
  choices=set(["executive", "manager","producer"]))
   hire_date = db.DateProperty() 
   new_hire_training_completed = db.BooleanProperty() 
    account = db.UserProperty()

    e = Employee(name="",    
         role="manager",      
       account=users.get_current_user())
    e.hire_date = datetime.datetime.now().date()
    e.put()

Entity Groups allow related entities to be grouped together with a parent child relationship, allowing entities to be updated as part of a single transaction.

Querying Data in the Data Store

Querying Entities from the Data Store can be done through two interfaces: the Query interface and the GqlQuery interface

• The Query interface prepares a query using instance methods. Here is a usage example for the filter() method retrieving filtered data of an entity
  

  q = Person.all()q.filter("last_name =", "Smith")
  q.filter("height <", 72)q.order("-height")
  
  results = q.fetch(5)

• The GqlQuery interface is an SQL like query interface allowing the application to write queries to operate on the entities.
  

       q = db.GqlQuery("SELECT * FROM Person " +       
         "WHERE last_name = :1 AND height < :2 " +       
         "ORDER BY height DESC",  
              "Smith", 72)
  
       results = q.fetch(5)

Indexes

The process of generating and using indexes in the Google Data Store is transparent to the application developer. Here are some of the key aspects of the Indexing process:

• An App Engine datastore maintains and uses an Index for every query that an application uses.
• An Index is a table which contains the result for each query in the desired order
• While the App Engine application can define its indexes in a configuration file the web server automatically changes this file based on execution patterns
• The indexes are continuously updated as the queries are executed

The above aspects are sufficient to get started with the Google Datastore and start utilizing its key features. For understanding more details on how the Data Store works and some of the more detailed options refer the Google Documentation

From http://www.thetechtrendz.com/2010/07/understanding-basic-data-storage.html