Powering Event Management Systems with Google Calendar

A frequent need for many community and groupware websites is an event calendar.  Though there are many calendar solutions available in commercial offerings, custom built and smaller sites often find it difficult to find a solution that is both cost-effective and feature rich.  This white paper examines how to use the existing Google Calendar infrastructure, by way of the published API, as the foundation for a fully featured calendar and event management system.

Background

Google Calendar was introduced by Google in 2006 as a web based, standards compliant calendar application.  In addition to the client interface, Google also published the full Application Programming Interface (API) so that third party developers could interact with the underlying data stored in the Google Calendar system.  The API is based on Atom RSS, an XML based publishing protocol operating over HTTP.  Because it is based on XML and Atom, the API is easily used by almost any programming language available, as long as there is a generic HTTP and XML parsing library available.

In addition to the API, Google also provides an easy way to embed Google Calendars in third party websites using HTML <iframe> tags.  This allows websites to use the display capabilities of the Google Calendar client on their own sites.  The embedded calendars can be customized to match the look and feel of a website while retaining the classic Google branding and style.

This document assumes basic familiarity with the Google Calendar service, as well as knowledge of basic programming concepts, including XML and HTTP.

Advantages of Integration with Google Calendar
It is possible to use the Google Calendar service as both the data provider (via the API) and the display system (through the embedded <iframe>) for a calendar, or as either option individually.

Using the Google Calendar system as a basis for a Calendar/Event Management system can provide many benefits:

• Reduced Development Time: Google has implemented a robust, tested data model that can be leveraged for event storage and processing.  The well document API decreases development time by providing an existing data model and framework from which to build from.
• Rich Feature Set: the Google Calendar service implements a range of useful features, from RSS syndication of calendars to Event attendee support.  These features are all standards compliant, ensuring interoperability with major calendaring applications like Microsoft Outlook and Apple iCal.
• Improved Reliability: the Google services are enterprise applications that have an uptime > %99.9.
• Cost Effectiveness: the Google services are free – using the API requires nothing more than a standard Google account, but you gain an enterprise level tool for your website.
• Customization: Google provides many options for customizing the look and feel as well as the functionality available through the Google Calendar API.  You can choose to use only those features that you need, and a display that matches the look-and-feel of your site.

Taken together, the benefits and features of using the Google Calendar system to power an event management/calendar system can be compelling for the small to medium sized website.

Requirements
In our experience, small to medium sized community based sites have a common set of requirements for a calendar/event management system:

• Display a publicly available calendar of events
• Display private calendar events restricted to members/registered users
• Categorize events by type and display in discrete sub-calendars
• Allow users to submit events
• Accept/Reject submitted events
• Manage events, including updating and deletion

System Design

Building a system that interacts with the Google Calendar system has three main components:

• The application server
• The display subsystem
• The event management staging subsystem

Figure 1 details the structure of the system.  The application server is the underlying structure and code upon which the website is based.  This may be any content management system, Ruby on Rails, etc.

The display subsystem includes all of the forms and ‘views’ that provide an interface to the user.  Within the display subsystem, and the main interface to the Google Calendar data is the embedded Google Calendar.  The calendar is embedded using <iframe> tags pointing to a Google hosted calendar resource.  Options are passed in the <iframe> source parameter, instructing Google on how to format the returned HTML calendar.  The display subsystem manages the display of the embedded calendar by dynamically updating the <iframe> and source parameters.

The event management staging subsystem handles events before they are committed to the Google Calendar service.  This is essential if a requirement for the system is to approve/reject event submissions before they are displayed publicly.  The staging subsystem can also provide a mechanism for storing and displaying additional information other than what Google provides in the default event data model (examples of this may be contact information, admission costs, website, etc.)

Figure 1: Diagram of System Design Utilizing Google Calendar

Workflow
The workflow for the system is relatively simple.  First, a user views a form with fields for entering all of the event information, then submits that information to the application server.  Assuming everything is validated, the event information is saved into the Event Staging System.  Before the event is saved to the Google Calendar service, it is reviewed and approved by a site administrator or other designated user.  Once the information is approved, the event staging system saves the event to the Google Calendar system using the Google Calendar API.  At this point, the information is ‘live’ and viewable on the calendar.

Any changes to the event go through the Event Staging System rather than the Google Calendar interface – this way, all changes are tracked and approved in the staging system before being made public.

Figure 2: Event Submission Workflow

Staging System Data Model
The key to synchronizing information between the Staging system and the Google Calendar system are unique IDs.  Google assigns unique IDs to all calendars and events within its system, providing the Staging system with a way to select and track data within the Google Calendar system.  By saving this ID locally with the event information, a representation of the Google event is made. Any change then made to the local representation of the event can be transparently saved to the Google representation via the unique ID.

Interacting with the Google Calendar API
Assuming the Application server is built using a MVC framework, the obvious place for interacting with the Google Calendar service is through the model.  The model will already contain the methods to save and update data, so it is a simple task to add the equivalent code to save the same information to the Google Calendar service.

Data Duplication vs Latency
A key thing to note about the system design described above is that data exists in two places simultaneously; in the staging database and the Google calendar service.  This is a design decision that weighs the latency cost of communicating with the Google Calendar service against the technical hassle of maintaining two separate data sets for the same event.

Depending on location, a typical API request to the Google servers will take anywhere from 200 to 600 ms to complete.  The most basic ’save’ POST operation to the Google calendar service requires two requests, one to authenticate and one to send the data.  Combined, these calls can take anywhere from 1/2 second to 1 second to complete, a very noticeable number for the end user.  And 2 requests to the Google service is the absolute minimum.  Therefore, using the Google service as the sole repository of the data, while possible, brings a sacrifice in perceived usability because of the overhead cost of the API requests.

The alternative, as outlined above, is a hybrid approach.  Latency is reduced by only communicating with the Google Server when necessary, i.e. when updating the displayed data on the Google servers.  For all other operations, such as searching for a particular event or populating a form with event data, the local staging table can be used, providing a faster end user experience.

However, this approach requires special care to reduce potential uses where data can become out of sync.  A simple set of design rules can help eliminate these situations:

1. Information should flow in one direction – i.e. only one data set should be the ‘master’, and all others are updated from that set.
2. If a remote save operation fails and data can not be updated to the remote service, the local operation should fail and no local data should be saved.
3. If in doubt as to whether the remote copy is in sync with the master copy, do a full replace of the record from the master set.

Conclusions

Implemented correctly, integrating Google Calendars on your site can save time, lower development costs and give you cutting edge and standards compliant features, all for free.  The architecture outlined here will allow you to retain the look and feel of your own site, while making use of the features provided by Google through the Calendar API.

Have questions or want to discuss a project?  Feel free to contact me.  Comments and thoughts are always welcome.

2009 MacBook Pro Review

Got home yesterday and decided to buy a new computer – just quit my job on Friday and signed a new contract for Seabourne so I felt like I could use a new toy.  For the last three years I’ve been using a Core 2 Duo MacBook and a Lenovo Thinkpad T60 with Kubuntu.  I liked both computers, but have been feeling hindered by the lack of screen real estate, especially when coding.  About three months ago I almost bought a 15 inch MBP, but decided to wait until I had more money.  And boy am I glad I did!  The whole line of MacBook Pros were updated last week at WWDC 09, and they all just got a lot better, and a lot cheaper.  Here’s my quick review.

Unboxing
As usual, Apple has done a terrific job on packaging.  Even though the computer is a solid 2 inches wider than my MacBook, there is probably 1/3 less packaging total.  Rather than a soft sleeve, the MBP comes wrapped in a clear plastic, similar to what the iPhones now come in.  As usual, a small manual and restore CDs come in a box below the computer, along with the power supply.  However, Apple no longer includes a free Apple Remote with laptops.

Look and Feel
The computer is gorgeous.  No other way to put it.  If I could pick one adjective for the whole thing, it would be ’smooth’.  The aluminum has a beautiful satin finish that feels silky to the touch, but definitely has enough texture to make it grippable.  The screen has a glossy finish – I don’t mind, but I can see how this would bug some people as you really have to be careful not to touch it otherwise the fingerprints are glaringly visible.  The increased screen color gamut is obviously apparent when comparing this version of the MBP with any previous Apple laptop – the colors are much sharper.  I’ve noticed that the screen hinge seems much smoother than the MacBook.  It is easy to move the screen with one finger and have it stay in that position.

The keyboard is great, though the same chiclet style as all the Apple laptops are now (some people hate it).  The keys are much more firm than the MacBook, a big improvement.  I really enjoy the backlit keyboard, and there are some very handy function buttons to dim or brighten the lighting behind the keys.

Size & Screen
One of the major reasons I bought a new computer was to get more screen space.  I’ve enjoyed the size of the 13 inch MacBook, but just felt that I’d be more productive with a 15 inch notebook.  Given the width and weight are about the same for both models now, it turns out to be a gain without much of a loss.

The screen is vibrant and has what seems like twice the brightness of my old MacBook.  Honestly, I feel about twice as productive with the new setup (as you can see in the picture above, that’s probably because I can have two terminal windows open at once!)

This thing is thin!  Though significantly wider than a MacBook (obviously), it is probably 10-20% thinner.  The LCD has benefited from the LED backlights by dropping a few millimeters of width.

Trackpad and Gestures
One of my favorite things about the new laptop is the support for multi-finger gestures.  Three fingers side to side goes back and forward in the browser or the finder, four fingers up and down triggers exposé or shows the desktop.  This took a little getting used to, but now that it’s an ingrained habit, I find that it is really hard to go back to computers without these features.  I had to download the new Release Candidate of Firefox 3.5 to get the three finger gesture support.  The track pad itself is much larger than almost any other laptop, which is really great for scrolling both horizontally and vertically through large code blocks.  I have had no problem with the new integrated button.  Overall I’m pretty impressed with the selective sensitivity of the track pad; I can keep my thumb on the button part of the pad and use my finger for pointing, and everything works smoothly as it would with a dedicated button.

Battery Life
The biggest selling point for me on this laptop was the new lithium-polymer battery.  The downside is that it’s integrated tightly into the unibody enclosure so you can’t remove it yourself or swap out another battery for longer uses.  The upside is that the battery now gets about 5-6 hours of use per charge, more than twice what my old MacBook used to get.  I’ve tested it out for about a week now, and with standard use (wireless on, screen brightness at 50%, bunches of programs open) I can get 5.5 hours pretty consistently.  The battery is also now supposed to last up to 1000 charges, or three years of standard use, and is replaceable at an apple store for $129, the same price as a new battery for older laptops.

Summary
Overall, this is the best MacBook Pro ever, and I think the best Apple Laptop ever.  The 15 inch MBP is a great middle ground between portability and usability.  The new features make this a compelling computer to use, has increased my productivity, but most importantly, is a pure joy to use.

Kubuntu + KDE 4.2 Bluetooth and Wireless Tips

It’s been about a week since I made the switch from XP to KDE 4.2 as my primary desktop.  Overall, everything has been very smooth, however I’ve run into two mildly annoying problems.  Both my bluetooth mouse and keyboard and my wireless connections seems to time out after I put the computer to sleep, never to return until I reboot.  With bluetooth, this is especially annoying as I depend on the external keyboard and mouse with my laptop.  So, after some forum searching, turns out there are two pretty easy fixes.  First off, wireless.

Wireless Sleep Fix
First thing you’ll need to do is figure out which wireless kernel module you’re using, there are various ways to do this – my computer is using a proprietary Atheros module, so I could go to the hardware app in KDE.  Second, add the following line to /etc/default/acpi-support:

SUSPEND_MODULES="$SUSPEND_MODULES ath_pci"

Now, when your laptop goes to sleep, the ath_pci module will be suspended as well. This will allow it to gracefully start up again once your computer awakens.

Bluetooth Sleep Fix
To do the same for bluetooth, edit the same file and find the line ‘MODULES_WHITELIST’, and add an entry for ‘bluetooth’.  You should get the resulting line:

MODULES_WHITELIST="bluetooth"

Conversely, this will keep the bluetooth module from being removed from the kernel on sleep – I think that it was not getting loaded back in correctly.

Anyway, these two things worked for me on an Intrepid Kubuntu machine, so best of luck.

VirtualBox vs. VMWare Server on Linux

For the last year I’ve been trying to get virtualization of a Windows Partition working on Linux.  The goal is to be able to use Linux as my primary desktop OS, while still having Windows available for the MS Apps I need for work (read Outlook).  The two main desktop apps that are freely available for this setup are VirtualBox, an open source solution by Sun, and VMware Server.  I tested both of these on a Core2 Duo 2 GHz system with 3 GB of RAM and an 80 GB hard disk divided into 3 partitions.  Windows XP Pro was installed on the first partition and Ubuntu running KDE 4.2 was installed on the other two.

VirtualBox
VirtualBox is now at version 2.1.2 and is a very solid app.  Sun provides an open source version and a freely available closed source binary for Windows, OS X and Linux.  The GUI is very solid and integrates well with KDE, due to the QT interface implementation.  The closed source binary has some features that are not available in the open source version, though they are getting migrated as the codebase grows (running a virtual machine from a physical partition, called rawdisk mode, used to only be available in the closed source version).

Setting up virtual box is relatively easy as most distributions have package available from the VirtualBox download page.  To get VirtualBox running with an installed version of windows, I found this pretty excellent guide.  The only problem I had was with the windows bootloader: I couldn’t get the open source iso to work, so I had to download a copy from a windows XP cd, as I didn’t have one handy.

Impressions
Using virtual box is very straightforward.  Setting up the machine and all the settings was straightforward.  There are very useful hints at the bottom of each tab in the settings pane which makes choosing options to enable particularly easy.  Once I had the machine booting into Windows, I installed the guest additions and rebooted.  The VirtualBox Guest Additions are a set of drivers and apps that make integration with the host OS seamless – i.e. you can move your mouse between the client and the host desktops.

Once I started using the virtualized OS, I ran into problems.  First, Windows was really slow.  To the point of being unusable.  Startup took about 10 minutes from boot to fully loaded, and opening an app was a time consuming task.  I turned off all the eye candy, which helped opening menus and such, but the overall speed and responsiveness was lacking.  Part of the appeal of VirtualBox is the ability to go into ‘Seamless Mode’ a la Parallels on the Mac.  Unfortunately, this never worked for me (but I also didn’t wait around for 20 minutes to see if it would work eventually).

I like VirtualBox, I really do, but its just too slow to be a functional, every day virtualization solution.  Combine that with some obvious inadequecies, like supporting only one processor and very rudimentary 3D acceleration support, and it doesn’t seem ready for prime time to me.  But, I plan on tracking the progress the Sun team is able to make and am definitely willing to give it another shot in the future.

VMware Server
VMware Server is a desktop/server virtualization solution from VMware.  The current version is 2.0, however this review will look at version 1.0.8 because the 2.0 version does not support rawdisk mode any longer.  Getting the older version working was a little bit of a challenge, given my kernel is newer (2.6.27-11) than the last one supported by the 1.0.8 release.  I followed this guide for getting the older version working on Ubuntu 8.10.  This guide will help to set up VMware to work with a physical disk and existing windows install.

Following the instructions was very simple, and I had everything up and running in about an hour, including download/install time.  The options for setting up a rawdisk VMDK in VMware were much more powerful than in VB, for example one could choose the number of processors to emulate on the client.

Impressions
Overall, VMware server is definitely not as pretty as VirtualBox – definitely not built as a desktop centered application.  However, the app is rock solid and wickedly fast.  Boot time was about half what VB was, and applications were reasonably responsive.

The guest additions were easy to install and gave the same basic features as VirtualBox, except the seamless mode.  I did run into a problem with the client software clock getting way out of sync with the hardware clock, either going to fast or two slow.  An easy fix was turning on the option to keep the two automatically synchronized in the guest additions control panel.

Conclusions
VMware came out ahead for my particular setup, which admittidly is not one most users will have.  As I wasn’t testing the traditional VM setup (having a completely ’soft’ install rather than running an OS of an existing partition), its hard to draw any broad conclusions, but for this use case, VMware is definitely the choice to go with.

Update: Check out my post here on how to get VMWare server 1.0x working on Ubuntu 9.0.

KDE 4.2 = My New Desktop

With the release of KDE 4.2, I decided to switch my primary work desktop from Windows XP to Kubuntu.  And I couldn’t be happier.  Overall, the desktop looks gorgeous (easily comparible to Mac OS X), the apps are great (very few crashes compared to previous 4.x versions), and of course, super powerful.  It has been a great switch. 

I tried using KDE 4.0 and 4.1, but along with everyone else, found that they were just too instable and feature poor for everyday use.  KDE 4.2 has added many features that existed in the old 3.5 branch, but also added a host of new and really innovative features that make this desktop a real competitor.  Here are a few of my favorites:

• KWin: Finally, the devs were able to bring compiz style eye candy to KDE natively.  KWin worked out of the box, autodetected my ATI chipset and turned on 3D effects automatically.  They standards are all there, and have been fleshed out to be comparible to Compiz.  On my 4.1 install, I never really got Compiz to work correctly with KDE – it ended up crashing a lot, and the configuration was in two places (the Compiz control panel and the KDE control panel for themes).  
• Notifications: Even in previous versions of KDE, notifications were always a pain.  They showed up willy nilly on the screen, with no organization which meant I turned them off for most programs.  KDE 4.2 has collected notifications down in the system tray, staking new notifications as they appear.  Any file action also goes into the same notification pile, keeping all the information in one place.  Very handy.
• Themes & Repository Integration: This is a feature that keeps on amazing me.  In the KDE system settings panel, you have the ability to add themes, desktop backgrounds, etc.  Most of these are integrated with the same “Get More…” button you can use to download additional plasmoids.  But these things are in the handiest of places – like the GRUB configuration panel, where you can choose to download new GRUB backgrounds from KDE-look.org using the built in preview/download tool.

At work, we use Outlook and some other Windows only apps that I need.  To use these apps, I needed to run my existing Windows XP install off the hard drive in Linux.  As you can see in the screenshot above this works really well – I’ll post the details in an upcoming post.