I started a post a while back called "Why Dropbox is so damn good", which basically pointed to a post by Michael Wolfe on Quora titled "Why is Dropbox more popular than other programs with similar functionality?". In addition to the Quora link, my post pretty much said "Polling is slow and dumb. Dropbox is asynchronous, uses a persistent connection to it's server, and tells clients right away about updates. This means you get fast updates without stalls, which rocks". The Quora post is a quick and amusing read, which you should check out. I'll wait.

The background to my non-posted post, is that I've been working on "Dropbox*" support for VoodooPad. You'd think this would be simple, but it really isn't. The combination of (non-mergeable) SQLite and SearchKit databases as well as a couple of other factors is requiring a big reworking of VoodooPad's file format to make it friendly with 3rd party background syncing solutions. I put Dropbox in quotes above because this rewrite isn't specific to Dropbox. The rewrite enables VoodooPad documents to be hosted on multiple syncing services: Dropbox, MobileMe iDisk Sync, even SCMs (Subversion, Git, Mercurial), or whatever else might come along.

As part of this rewrite I've also came up with a new syncing solution for VoodooPad, which is similar to it's existing WebDAV syncing support. It's a little project I call SyncDAV, but it isn't specific to VoodooPad at all. SyncDAV is sort of like Dropbox, but over WebDAV. You can set it up to use your iDisk or your own personal WebDAV server for syncing folders (and that's what a VoodooPad document is- a bundle (aka folder) for pages and other supporting files). This is needed because VoodooPad is also used in environments where multiple people will work on the same document, and then sync up using it's existing WebDAV support.

Then WWDC11 happened, as well as iCloud, and that put things in a bit of a spin for me**.

Guy English has a nice post up called "Push" which basically outlines how Apple is using their push notification system as a nice way to handle updates from iCloud. It uses a single persistent connection and files update asynchronously in the background, which means you get fast updates without stalls, which rocks. Just like Dropbox! Though iCloud does so much more than that (read Guy's post).

So I thought iCloud basically made my SyncDAV solution moot, but with WWDC's RDF a bit behind us I'm not so sure anymore. And I'm not sure iCloud is going to really help for syncing between groups of people.

Of course SyncDAV is missing one very important part- and that's the persistent connection to a server for fast updates, instead of polling. Which brings me to the whole point of this post:

What the heck can indie devs use as a persistent server connection, so that their syncing implementations can receive fast updates? I'm not going to host a server for this purpose- that really isn't a core competency of mine, and I know it isn't for many other indie devs either. And as a customer, I wouldn't want to rely on a 3rd party server for this. Servers go down, get raided, or you get kicked off for no known reason. Sometimes companies turn to the dark side. But do we really have another choice?

Here are some ideas:

Setup a Comet server along with the WebDAV server to send out the updates. Downside: it's annoying enough to setup a WebDAV server if you don't know what you're doing, and maybe you're using a hosted WebDAV server which won't let you setup extra bits like that.

Use XMPP, where there are extensions to the protocol which allows applications to talk to each other. Downside: you now have another server to set up in addition to your WebDAV server, as well as another username and password to keep track of.

Go peer to peer. That would be a really fun problem to work on, but you're still going to end up needing some sort of centralized server for devices to find each other.

Piggyback off of iCloud, but don't actually use iCloud to store your data. Instead, just store information about where documents are stored (such as WebDAV URLs) and what has just been updated. Downside: I think this could get messy very quick, and it also requires an iCloud account. (Of course the OS might force an iCloud account on us in the future anyway).

What other ideas are there? If you've got one that I haven't though of, let me know in the comments.

I really think this is something that indie devs are going to need, and we're going to need it soon. Customers want a solution that doesn't rely on Apple or Dropbox for syncing.





* You can put your VoodooPad documents on Dropbox, or in various SCM's. The problem is- you can't have the same document open on multiple devices at the same time and expect everything to work correctly. And VoodooPad certainly won't update an open page on device B when device A changes and saves a page. I'm working towards making such things possible.

** I have one big unanswered question about iCloud. Are developers allowed to use it if their app is not sold via the App Store? I asked this to multiple people at WWDC, and got conflicting answers. My educated guess is "No", but I'd be happy if someone could point me to documentation which confirms it either way.