Syncing strategies

Another problem has (mostly) been solved for me it seems. I've had quite a lot of problems the last few months with calendars and email and contacts being out of sync between workplaces and my private computers/cellphones etc. The problem has been that I've gotten a new contract and hence am relocated to another workplace. My employer has a stupid (ok, maybe not so stupid, but annoying me nonetheless) policy of not allowing any e-mail to internal addresses be forwarded or fetched from external networks. At the same time, my contracting has put a heavy load on the calendar and all of a sudden I understand everyones problems with syncing e-mail/contacts/calendars etc... it's really a must.

Anyways, in short, I started out with 4 calendars (workplace1, workplace2, home1, home2(laptop) and cellphone) needing sync, and using Microsoft Exchange weirdo protocols was not an option (I'm not using Windows or Outlook at home anymore). This has later been extended to sync contacts and my two instances of thunderbird (not yet finished). So, in short:

1. Workplace1 = Windows Vista with bluetooth
2. Workplace2 = Microsoft Exchange server with limited access.
3. Home1 = Ubuntu with thunderbird
4. Home2 = Old Laptop, Windows XP with thunderbird, will likely migrate to Linux soon as well since I barely use it anymore due to the OS on it.
5. Cellphone = LG KC910 with bluetooth and wifi.

First off, finding a sync strategy wasn't easy. First, decide on where your "central repository" is, or rather which will be your main device. My current solution relies on cellphone (LG KC910) being the central repository since it's the only common gadget at all locations. connection at workplace1 is directly over bluetooth to the KC910 using the LG sync application. The application is absolutely horrible, but it does it's job (barely). Unfortunately LG relies on a proprietary bluetooth protocol for syncing so I have yet to find any decent replacement applications.

My big problem was finding a working solution at home, and I think I finally found it in Funambol (https://www.forge.funambol.org/DomainHome.html) which is a SyncML server. Basically, I got a server on my local network running Funambol, when I get home, connect to the local wifi, and sync with funambol (See http://www.mobyko.com/phoneinfo/lg/renoirkc910Info.do, a bit down for instructions). The funambol server then acts as a "central repository" when I'm at home containing all calendars etc. Thunderbird sessions on Home1 and Home2 uses the funambol addon (https://addons.mozilla.org/en-US/thunderbird/addon/8616) to sync with the funambol server.

WARNING! So far I dont trust funambol to run on the public internet, for one part it seems to be sending passwords in cleartext, as well as data. I'd love to figure out a way to get it all encrypted using SSL/https, but I'm a complete newbie to Tomcat (base plattform for funambol) as well as java. As far as possible, try to use a closed/encrypted network for this unless you get https running imho.

A second note on Funambol is that I had some really funny Timezone problems when setting it up, all devices run the correct timezones, but for some reason my calendars winded up being winded 2 hours into the future at home, I got it fixed by setting all timezones in funambol for all devices manually, and then disabling the timezone handling in funambol... don't ask me why it fixed it etc, I hate working with timezones ;) .

All that said, I really think SyncML was a big saviour for me in the end, but I had a hard time finding a single word on it or anyone really recommending it. Bluetooth just needs to be .... well, better support, and everyone needs to agree on standards. Everyone (companies) seems to be running around doing their own thing, which means Linux has very good basic bluetooth support, but none of the higher layer stuff since it's badly implemented or proprietary.

"New" subversion structure using svn:externals

Me and the boss deviced a new structure for the project during the last few weeks, and it's been slowly refining in our heads until yesterday when we finally implemented it. I think we made a rather refined and complex structure, but once we got it into place physically and once we get the general idea into the developers heads (including me), I think it will prove very powerful.

That being said, I don't think this is a new structure, I just think people are very quiet about how they use subversion, and it's a problem. Newcomers do the same old errors over and over again. So, let's get on to try and explain it all.

Most projects uses a single BTT root, where BTT stands for Branches, Tags and Trunk. Ie, they start a project, and then straight in the root put the BTT, and then inside that, they create the project structure. For example:

• project-root/
• Branches
• Tags
• Trunk
• admin
• src
• out
• test

This is a good basic structure for very small projects, containing perhaps 10'ish files, or where the actual implementation is perfectly homogenous and has no need for separated versioning. Every time we want to make a release, we cheap copy the content to Tags as a new tag (called perhaps /Tags/Milestone1-RC1). We now have a release that we can provide to people.

The problem comes if it isn't so homogenous. For example, let's say you are developing a calculator. It has two objects, a numpad and a display. What if you want to make a new version just of the display? You need to make a completely new version, including for the numpad.
Or how about wanting to branch just a small part of the project? Ie, I want to use a branch for the numpad, and then use the trunk for the display. You'd then have to make a cheap copy for the entire tree. Admittedly, it isn't costing too much.

Our "new" structure deals with this on a different level. Basically, the idea is to have multiple BTT roots, and then use svn:externals to connect the correct tags to create
1) a complete releasable project and
2) a complete workarea project.

For the calculator example, you get the following structure:

• calculator/
• Calculator_Modules/
• Display/
• Branches/
• Tags/
• Trunk/
• Numpad/
• Branches/
• Tags/
• Trunk/
• Calculator/
• Branches/
• Tags/
• Trunk/

As you can see, it looks much more complex, and it is, but the possibilities are infinitely much better.

The Calculator/Trunk/ directory contains a svn:externals property linking in the Calculator_Modules/Display/Trunk as Display and Calculator_Modules/Numpad/Trunk as Numpad. This works by linking external resources into the current directory structure, so basically I would get the trunks into my Calculator trunk, but properly renamed, without them actually being there in the repository. This also works on "real externals" by the way, such as linking in a specific version of a library from some repository on the Internet.

To create a Calculator/Tags/MS1 we could either just set a -rXX to the correct subversion revision, or we would create svn:externals to the correct Display and Numpad Tags, not their trunk. This way, we can say that "Calculator 1.0 contains Display 2.0 and Numpad 2.1", not "Calculator 1.0 contains Display revision 439 and Numpad revision 587", or even worse "Calculator 1.0 is revision 587" which completely lacks granularity.

I'm not completely sure it's perfect, and others have probably already tested it, but I think it will be pretty sweet :-).