Powering Event Management Systems with Google Calendar

A frequent need for many community and groupware websites is an event calendar.  Though there are many calendar solutions available in commercial offerings, custom built and smaller sites often find it difficult to find a solution that is both cost-effective and feature rich.  This white paper examines how to use the existing Google Calendar infrastructure, by way of the published API, as the foundation for a fully featured calendar and event management system.

Background

Google Calendar was introduced by Google in 2006 as a web based, standards compliant calendar application.  In addition to the client interface, Google also published the full Application Programming Interface (API) so that third party developers could interact with the underlying data stored in the Google Calendar system.  The API is based on Atom RSS, an XML based publishing protocol operating over HTTP.  Because it is based on XML and Atom, the API is easily used by almost any programming language available, as long as there is a generic HTTP and XML parsing library available.

In addition to the API, Google also provides an easy way to embed Google Calendars in third party websites using HTML <iframe> tags.  This allows websites to use the display capabilities of the Google Calendar client on their own sites.  The embedded calendars can be customized to match the look and feel of a website while retaining the classic Google branding and style.

This document assumes basic familiarity with the Google Calendar service, as well as knowledge of basic programming concepts, including XML and HTTP.

Advantages of Integration with Google Calendar
It is possible to use the Google Calendar service as both the data provider (via the API) and the display system (through the embedded <iframe>) for a calendar, or as either option individually.

Using the Google Calendar system as a basis for a Calendar/Event Management system can provide many benefits:

• Reduced Development Time: Google has implemented a robust, tested data model that can be leveraged for event storage and processing.  The well document API decreases development time by providing an existing data model and framework from which to build from.
• Rich Feature Set: the Google Calendar service implements a range of useful features, from RSS syndication of calendars to Event attendee support.  These features are all standards compliant, ensuring interoperability with major calendaring applications like Microsoft Outlook and Apple iCal.
• Improved Reliability: the Google services are enterprise applications that have an uptime > %99.9.
• Cost Effectiveness: the Google services are free – using the API requires nothing more than a standard Google account, but you gain an enterprise level tool for your website.
• Customization: Google provides many options for customizing the look and feel as well as the functionality available through the Google Calendar API.  You can choose to use only those features that you need, and a display that matches the look-and-feel of your site.

Taken together, the benefits and features of using the Google Calendar system to power an event management/calendar system can be compelling for the small to medium sized website.

Requirements
In our experience, small to medium sized community based sites have a common set of requirements for a calendar/event management system:

• Display a publicly available calendar of events
• Display private calendar events restricted to members/registered users
• Categorize events by type and display in discrete sub-calendars
• Allow users to submit events
• Accept/Reject submitted events
• Manage events, including updating and deletion

System Design

Building a system that interacts with the Google Calendar system has three main components:

• The application server
• The display subsystem
• The event management staging subsystem

Figure 1 details the structure of the system.  The application server is the underlying structure and code upon which the website is based.  This may be any content management system, Ruby on Rails, etc.

The display subsystem includes all of the forms and ‘views’ that provide an interface to the user.  Within the display subsystem, and the main interface to the Google Calendar data is the embedded Google Calendar.  The calendar is embedded using <iframe> tags pointing to a Google hosted calendar resource.  Options are passed in the <iframe> source parameter, instructing Google on how to format the returned HTML calendar.  The display subsystem manages the display of the embedded calendar by dynamically updating the <iframe> and source parameters.

The event management staging subsystem handles events before they are committed to the Google Calendar service.  This is essential if a requirement for the system is to approve/reject event submissions before they are displayed publicly.  The staging subsystem can also provide a mechanism for storing and displaying additional information other than what Google provides in the default event data model (examples of this may be contact information, admission costs, website, etc.)

Figure 1: Diagram of System Design Utilizing Google Calendar

Workflow
The workflow for the system is relatively simple.  First, a user views a form with fields for entering all of the event information, then submits that information to the application server.  Assuming everything is validated, the event information is saved into the Event Staging System.  Before the event is saved to the Google Calendar service, it is reviewed and approved by a site administrator or other designated user.  Once the information is approved, the event staging system saves the event to the Google Calendar system using the Google Calendar API.  At this point, the information is ‘live’ and viewable on the calendar.

Any changes to the event go through the Event Staging System rather than the Google Calendar interface – this way, all changes are tracked and approved in the staging system before being made public.

Figure 2: Event Submission Workflow

Staging System Data Model
The key to synchronizing information between the Staging system and the Google Calendar system are unique IDs.  Google assigns unique IDs to all calendars and events within its system, providing the Staging system with a way to select and track data within the Google Calendar system.  By saving this ID locally with the event information, a representation of the Google event is made. Any change then made to the local representation of the event can be transparently saved to the Google representation via the unique ID.

Interacting with the Google Calendar API
Assuming the Application server is built using a MVC framework, the obvious place for interacting with the Google Calendar service is through the model.  The model will already contain the methods to save and update data, so it is a simple task to add the equivalent code to save the same information to the Google Calendar service.

Data Duplication vs Latency
A key thing to note about the system design described above is that data exists in two places simultaneously; in the staging database and the Google calendar service.  This is a design decision that weighs the latency cost of communicating with the Google Calendar service against the technical hassle of maintaining two separate data sets for the same event.

Depending on location, a typical API request to the Google servers will take anywhere from 200 to 600 ms to complete.  The most basic ’save’ POST operation to the Google calendar service requires two requests, one to authenticate and one to send the data.  Combined, these calls can take anywhere from 1/2 second to 1 second to complete, a very noticeable number for the end user.  And 2 requests to the Google service is the absolute minimum.  Therefore, using the Google service as the sole repository of the data, while possible, brings a sacrifice in perceived usability because of the overhead cost of the API requests.

The alternative, as outlined above, is a hybrid approach.  Latency is reduced by only communicating with the Google Server when necessary, i.e. when updating the displayed data on the Google servers.  For all other operations, such as searching for a particular event or populating a form with event data, the local staging table can be used, providing a faster end user experience.

However, this approach requires special care to reduce potential uses where data can become out of sync.  A simple set of design rules can help eliminate these situations:

1. Information should flow in one direction – i.e. only one data set should be the ‘master’, and all others are updated from that set.
2. If a remote save operation fails and data can not be updated to the remote service, the local operation should fail and no local data should be saved.
3. If in doubt as to whether the remote copy is in sync with the master copy, do a full replace of the record from the master set.

Conclusions

Implemented correctly, integrating Google Calendars on your site can save time, lower development costs and give you cutting edge and standards compliant features, all for free.  The architecture outlined here will allow you to retain the look and feel of your own site, while making use of the features provided by Google through the Calendar API.

Have questions or want to discuss a project?  Feel free to contact me.  Comments and thoughts are always welcome.

New Ruby Google Calendar API Gem: GCal4Ruby

I’ve written a few previous posts about using the Google Calendar API with Ruby and Ruby on Rails, but I’ve wanted to create my own library that includes the functionality I’ve had to monkey patch into others’.  So, I’ve gone ahead and created GCal4Ruby, a full featured Google Calendar API wrapper for Ruby available as a Gem.  You can download the source here or with ’sudo gem install gcal4ruby’.  Some of the features:

• Clean, ActiveRecord style syntax and code structure
• Full querying of calendars and events
• Full calendar creation, editing and deletion
• Setting public/private permissions for calendars
• Event creation/editing/deletion
• Support for full event recurrence
• Support for event reminders
• Support for event attendees

All of the major features have been implemented, and over the next couple of days I hope to complete the smaller outstanding todos.  Any help testing is much appreciated.  Below are some example uses.  For more information, check out the documentation.

Get all calendars for an account

       service = Service.new
       service.authenticate("user@gmail.com", "password")
       calendars = service.calendars

Create a new Calendar

       cal = Calendar.new(service)

Find an existing Calendar

       cal = Calendar.find(service, "New Calendar", {:scope => :first})

Find all calendars containing the search term

       cal = Calendar.find(service, "Soccer Team")

Find a calendar by ID

       cal = Calendar.find(service, id)

Create a new Event

       event = Event.new(calendar)
       event.title = "Soccer Game"
       event.start = Time.parse("12-06-2009 at 12:30 PM")
       event.end = Time.parse("12-06-2009 at 1:30 PM")
       event.where = "Merry Playfields"
       event.save

Find an existing Event

       event = Event.find(cal, "Soccer Game", {:scope => :first})

Find all events containing the search term

       event = Event.find(cal, "Soccer Game")

Create a recurring event for every Saturday

       event = Event.new(calendar)
       event.title = "Baseball Game"
       event.where = "Municipal Stadium"
       event.recurrence = Recurrence.new
       event.recurrence.start = Time.parse("13-06-2009 at 4:30 PM")
       event.recurrence.end = Time.parse("13-06-2009 at 6:30 PM")
       event.recurrence.frequency = {"weekly" => ["SA"]}
       event.save

Create an event with a 15 minute email reminder

       event = Event.new(calendar)
       event.title = "Dinner with Kate"
       event.start = Time.parse("20-06-2009 at 5 pm")
       event.end = Time.parse("20-06-209 at 8 pm")
       event.where = "Luigi's"
       event.reminder = {:minutes => 15, :method => 'email'}
       event.save

Create an event with attendees

       event = Event.new(calendar)
       event.title = "Dinner with Kate"
       event.start = Time.parse("20-06-2009 at 5 pm")
       event.end = Time.parse("20-06-209 at 8 pm")
       event.attendees => {:name => "Kate", :email => "kate@gmail.com"}
       event.save

Creating Public Calendars with Google Calendar API

Update: I’ve got a new Ruby Gem that improves upon the functionality described below.  Check it out here.

In my previous post I covered how to create and manage google calendars using the Google Calendar API and Ruby on Rails.  One feature I really wanted in order to make my project really user friendly was the ability to make a newly created calendar public without having to log in to the Google Calendar website and make the permissions change manually.  Unfortunately, the documentation for how to do this is sparse at best.

Background
The key to making a public calendar lies in the Google Calendar Access Control Lists (ACL).  Like all other parts of the API, this is simply an Atom RSS feed that you can POST or GET from.  Essentially, ACLs allow you to specify the major permissions for each user that is added to a calendar.  The first problem I ran into was trying to figure out the correct url to POST to.  The google documentation only lists how to change the ACL for a specific user (and by default their primary calendar) rather than for a secondary calendar.  With a little sluething, I was able find a forum post that gives the calendar url:

"http://www.google.com/calendar/feeds/#{cal_id}/acl/full"

Where ‘cal_id’ is the unique calendar id.

To change a permission, you post a new rule to the feed above.  For example, to add a user with basic read permissions, you would POST the following code the to ACL feed url:

<entry xmlns='http://www.w3.org/2005/Atom' xmlns:gAcl='http://schemas.google.com/acl/2007'>
  <category scheme='http://schemas.google.com/g/2005#kind'
    term='http://schemas.google.com/acl/2007#accessRule'/>
  <gAcl:scope type='user' value='darcy@gmail.com'></gAcl:scope>
  <gAcl:role
    value='http://schemas.google.com/gCal/2005#read'>
  </gAcl:role>
</entry>

The code above is pretty straightforward, and the google documentation gives a nice overview of what each part of the XML represents. However, I don’t want to add specific users to my calendars – rather I want to make them public to everyone by default.

After looking more at the documentation and at some forum posts, I figured out that instead of specifying a user, you can set the ’scope’ type to ‘default’ and omit the value attribute.  The role value can then be ‘read’ to allow for read only access to the public.  Therefore, the XML to create a public calendar looks like this:

<entry xmlns='http://www.w3.org/2005/Atom' xmlns:gAcl='http://schemas.google.com/acl/2007'>
  <category scheme='http://schemas.google.com/g/2005#kind'
    term='http://schemas.google.com/acl/2007#accessRule'/>
  <gAcl:scope type='default'></gAcl:scope>
  <gAcl:role
    value='http://schemas.google.com/gCal/2005#read'>
  </gAcl:role>
</entry>

Extend GCalAPI
The code for adding this to the GCapAPI code, as I did in my previous post, is as follows:

class GoogleCalendar::ServiceBase
  def make_calendar_public(cal_id)
    #change permissions on the calendar to public
    path = "http://www.google.com/calendar/feeds/#{cal_id}/acl/full"
    request = "<entry xmlns='http://www.w3.org/2005/Atom' xmlns:gAcl='http://schemas.google.com/acl/2007'>
                    <category scheme='http://schemas.google.com/g/2005#kind' term='http://schemas.google.com/acl/2007#accessRule'/>
                     <gAcl:scope type='default'></gAcl:scope>
                     <gAcl:role value='http://schemas.google.com/gCal/2005#read'></gAcl:role>
                  </entry>"
    uri = URI.parse(path)
    res = do_post(uri, {"Content-Type" => "application/atom+xml", "Content-Length" => request.length.to_s}, request)
    if res.code.to_i != 201
      raise "Could not change permissions on calendar: "+res.body
    end
  end
end

You can now add the ‘make_calendar_public’ function call wherever you need, passing the relevant calendar id.

Using Google Calendar API in Ruby on Rails

Update: I’ve got a new Ruby Gem that improves upon the functionality described below.  Check it out here.

Update 2: Rewrote this article to use GCal4Ruby.

A new project of mine involves project managment and milestones.  One feature I was interested in was natively integrating the Rails app with Google Calendar so that the milestone information would be available to anyone who wanted to subscribe to it.  I looked around and found a couple of google calendar integration libraries, but suprisingly there isn’t a fully implemented GData API that allow access to all the features of the Google Calendar API.

• Google Calendar: a pretty simple and full featured google calendar library in ruby, with optional Rails plugin.  A good option for anyone who’s interested in accessing or exporting Google Calendars into HTML or Excel.  Though the Event support is excellent, my application needed finer grained control over multiple calendars under each user, and this library didn’t include the functions to determine calendars programmatically.
• GCalAPI: also a good and simple API for Google Calendar, though it provided a better interface for finding calendars associated with a user account.  Unfortunately, the documentation is a little lacking, and there was no built in way to create a new calendar.  But, I ended up using this library despite the drawbacks.

These are two very good libraries, but haven’t been updated in a couple of years, and were missing some key pieces of functionality that I wanted.  So, I’ve created a new Ruby library (available as a Gem) called GCal4Ruby that implements all of the major features of the GCal API, including recurrence, attendees and reminders.  One of the main improvements in GCal4Ruby over the other two libraries is that you don’t have to remember the ‘feed’ urls for Google; all that is handled transparently for you.  This article describes how to use GCal4Ruby to interact with Google Calendars through Ruby on Rails.  Be sure to read the documentation for all available functions and features.

Setup
Setting up the GCal4Ruby gem is straightforward: just install it using sudo gem install gcal4ruby or by adding gem.config ‘gcal4ruby’ line to your Rails project environment.rb file. Then, all you need to do is run rake gems:install to install the gem through rails.

My application includes a ‘project’ model and a ‘milestone’ model.  Each project has a unique google calendar associated with it, though they all appear under one account.  You can either use a Google Gmail account for this, or you can your own Google Apps account if you have one.  You’ll probably want to add your username and password to the environment.rb file as static constants for reference later.

Creating a Calendar
The one thing that the GCalAPI library can’t do by default is create a calendar programmatically.  GCal4Ruby includes the ability to do this.  Here’s how to create a new public calendar from within a Calendar model in Rails:

#Step 1: Setup feed url and create account service using constants
service = GCal4Ruby::Service.new
service.authenticate(GOOGLE_LOGIN, GOOGLE_PASS)

#Step 2: Create a new calendar
calendar = GCal4Ruby::Calendar.new(service)
calendar.title = self.title # tile = 'A New Calendar'
calendar.summary = self.description
if calendar.save
 return true
else
 return false
end

#Step 3: Make the calendar publicly available
calendar.public = true

Adding an Event
Adding an event is straightforward.  Basically, its a three step process:

1. Authenticate and load your Google Calendar Account
2. Load the specific calendar you’d like to work with
3. Construct and add your event to the selected calendar

I added this code to the milestone model which is within the save method.  Here’s the basic code for accomplishing the three steps outlined above:

#Step 1: Setup feed url and create account service using constants
service = GCal4Ruby::Service.new
service.authenticate(GOOGLE_LOGIN, GOOGLE_PASS)

#Step 2: Load calendar data for the calendar associated with the project
calendar = GCal4Ruby::Calendar.find(service, 'A Test Calendar', :first)

#Step 3: create a new event and save
event = GCal4Ruby::Event.new(calendar)
event.title = self.title
event.content = self.description
event.where = "N/A"
event.start = self.start_date
event.end = self.end_date || self.start_date
event.all_day = self.all_day_event
if !event.save
  return false
end