Synchronizing my Desktop, Tablet and Laptop

Frustrated at the manual work needed to keep multiple machines in synch,  then try Allway Sync,  I use it to keep my Desktop, Laptop and Tablet in synch and it’s working great so far,  what follows is a much longer description that might be worth reading if you like detail!

To give you an idea of scale – 15GB of data in 20,000 files and 1,500 directories.  Full synch takes less than 5 minutes!  Quick synchs take a few seconds.

I use synchronization extensively to keep my Desktop, Tablet and Laptop in synch and to create backups of my main data disks.  For many years I have used a combination of robocopy scripts to achieve this synch, and it has worked well except for one scenario;  if I delete files in any of the three replica’s these deletes are not propagated, in fact what normally happens is that the deleted file is copied back to my machine from one of the other replica’s.  This happens because of a limitation with robocopy which is that it doesn’t maintain a database that records the synch state of three replica’s, without a database it has no way of knowing whether a deleted file is in fact a missing file.  I now have a solution to this, which I will describe in a minute,  however first a few more details on the robocopy method that has worked for so long.

For backup I have a batch file that runs every night,  it does a replication with the /MIR switch from my desktop working disk  \\desktop\d$ to my backup disk \\desktop\e$ and from the backup disk to my server \\server\d$ this allows me to recover quickly from a disk or a system failure.  /MIR ensures that my backups are exact replica’s of the original \\desktop\d$ it’s easy to avoid backing up files I don’t want to keep by using the /xf and /xd switches which exclude files and directories from the backup.  For backups I will continue to use robocopy as it has served me well for thousands of backups without ever failing!

However for synchronization I have three scripts which run from my quick launch toolbar which you can see to the right.  These scripts are called synch, tidy and Maxthon sync.  These work as follows:

• Synch.bat copies the latest versions of all files to all three machines,  ie it copies the latest versions (including new files) from \\tablet\d$ and \\laptop\d$ to \\desktop\d$ which means that \\desktop has the latest copies of everything and then copies the latest versions from \\desktop\d$ back to \\tablet\d$ and \\laptop\d$ this means that all three machines end up being exactly the same, and that they each contain a superset made up of the latest versions of any files that existed on any of the machines.  Normally this is what you want, except that if I have been working on \\tablet for a few hours by the pool and have renamed, moved and deleted a whole load of files then all of these will be copied back to \\tablet from \\desktop and all of the renamed and moved files will be duplicated of \\desktop and \\laptop, not exactly my intent.
• Tidy.bat is designed to get around this,  tidy.bat uses the /MIR switch to make \\laptop and \\tablet exact copies of \\desktop, which is useful but means that I need to make sure that moves, renames and deletes all take place on \\desktop.
• Maxthon sync.bat is essentially the same as synch.bat but just runs on the Maxthon directory and synchs up the Maxthon groups between machines, I run this separately because it is quick.

I have lived with and been annoyed by this for years and tried loads of solutions which all failed for one or more of the following reasons:

• Poor support for auto-discovery of new subdirectories (of which I have loads)
• Poor performance
• Poor support for filtering files and directories I don’t want to synch (again I have quite a few of these)
• Poor handling of the fact that both \\tablet and \\laptop are frequently not connected to the network when the synch tries to run, and that the synch should wait for them to connect and then carry on
• Poor handling of the fact that some directory sub-trees need to be synched at a much greater frequency than others.

I finally have a solution Allway sync is free for personal use and meets all of my needs (it had a few bugs – but within a week of me reporting them they were all fixed, the version that works for me is 3.42).  These are the features that made this just perfect for me:

• It supports multiple jobs, these jobs can each synch different sub-trees of the same tree, or operate on the same tree but at different frequencies.
• Each job can be scheduled to run with a different frequency and with different include and exclude filters at the file or directory level.  These filters are very flexible
• Jobs can replicate multiple devices, in my case 3
• Each job can either do a one way replication, ie makes the targets exact replicas of the source, or can synchronize, synchs can optionally propagate deletes and modifications. In my case I want to be able to make changes on any of the three devices and the changes to be reflected seamlessly on every device, so I synchronize with modifications and deletes enabled
• You can preview the changes it will make,  which is great for designing your jobs and for testing.
• You can copy deleted files to the recycle bin, just in case you deleted a file by accident
• It copes with devices being disconnected from the network
• It only needs installing on one devices, in my case \\desktop
• It is very visual and easy to use,  for example you can select the directories you want to sync by navigating to them, which reduces errors

At the moment I have 4 synch jobs:

• Every 2 minutes synch favourites on all three machines
• Every 2 minutes synch Maxthon (my browser) config and group definitions on all three machines
• Every 3 hours synch all of my dynamic data, which is my full data tree with all “static” data filtered out (for example archive areas)
• Every day synch the whole data tree, but with temp data and un-synchable data (for example databases) filtered out, (database backups are included).

With this config I am finding that I can work pretty well seamlessly on any of my machines and never have to worry about which one I am using,  since each one is configured for different tasks this makes things pretty efficient.

I can run any of these synchs on demand as well with a 3 mouse clicks.  Now that it all seems to be working well I will probably start extending the jobs to refine things a bit more.