posted on Sunday, September 24, 2006 1:13 PM
by
Jonathan Hodgson
Remote File Synchronization
Can you ever have too fast networks?
As more homes in the UK have broadband, we are starting to see more advanced remote synchronization techniques coming to the forefront as people want to share and consume data over more devices.
Unfortunately Microsoft is still unclear on it's synchronization platforms, whether it is
SyncToy for Windows XP,
FolderShare or the
(slimmed-down) Sync Center in Vista.
However in Vista and Longhorn server some new APIs around
Remote Differential Compression (RDC) are available:
Remote Differential Compression (RDC) allows data to be synchronized with a remote source using compression techniques to minimize the amount of data sent across the network.
RDC is different from patching-oriented differencing mechanisms, such as Binary Delta Compression (BDC), that are designed to operate only on known versions of a single file. BDC requires the server to have copies of all versions of the file, and differences between each pair of versions are precomputed so that they can be distributed efficiently from a server to multiple clients.
RDC makes no assumptions about file similarity or versioning. Because differences between files are computed on the fly, RDC is ideally suited for synchronizing files that are different or have been updated independently.
It would be good if someone took the Windows Base sample,
Copying a file over the network using Remote Differential Compression to reduce required bandwidth, and put a managed .Net wrapper over it.
Quite a few commerical products aim to push a combination of differencing and compression, a similar idea to
iOra's Epsilon Technology which they use in a number of their products including network accelerator for SharePoint across WANs.
This is becoming a more inportant technology space in the coming years, if you are still interested read more about
algorithms for delta compression and remote file synchronization (PDF).