[OpenAFS-devel] rsync for OpenAFS

[Peter Somogyi]

--Boundary-00=_LndGC5TiIJf+XzQ
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi,

I've got the task to implement the rsync-algorythm (http://rsync.samba.org/tech_report/)
for OpenAFS (modifying the "vos release" behaviour).
I've made an initial design - see attachment.
We would like to see the implementation in the CVS.
Any comments appreciated.

Peter

--Boundary-00=_LndGC5TiIJf+XzQ
Content-Type: text/plain;
  charset="iso-8859-1";
  name="design_afs_rsync.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename="design_afs_rsync.txt"

Using rsync algorythm in OpenAFS
=====================

Purpose: modify "vos release" synchronization mechanism to support rsync-algorythm, comparing files with 'rolling checksum' and sending only the differences.

In a few words:
- add a new option to volserver "-rsync"
- implement rolling checksum file compare
- implement a new rsync-like protocol
- combining with compression on-the-wire (if the -z [...] option was given)

More details:
- modify the function AFSVolForward[Multi] in volserver to support rsync-like protocol
- add file comparsion algorythm ("rsync algorythm")
- add an rsync-like protocol and design for efficient data transfer and CPU utilization
- important: we _don't_ assume that all RO sites has the same volume data
- a configuration file is needed (there must be a couple of settings which will seriously affect the performance)
- add/modify the RPC 'GetCapabilities'
- there should be a debug-option for volserver which causes to log modifications into VolserLog locally

Protocol design:
- notations:
	'A' means the volserver instance, for which AFSVolForward[Multi] has been invoked (source of up-to-date volume data)
	'Bi' [i=1..n] means the other volservers which are the destinations (where volume data needs to be updated)

- RPC must be used, with the functions rx_Read and rx_Write. A "channel" should mean here an RPC call, where one endpoint calls rx_Write, and the other endpoint calls rx_Read. Exactly one RPC call corresponds to this channel. We assume that such a channel is uni-directional.

There would be 3 channels:
C1) [A->Bi] 'A' sends file block checksums to 'Bi' [i=1..n]
C2) [Bi->A] Bi sends a list of difference query records to A [i=1..n]
C3) [A->Bi] 'A' sends file difference data chunks to Bi [i=1..n]

For each type of these channels a special data format must be used.

Each channel must be alive until all file synchronization has been done for the volume.

Creation of the channels:
- C1) would be the reuse of already existing rx calls
- for C2) and C3), 2 new calls should be created on the volserver interface (for example: "QueryDiff" and "SendDiffData"), or 2 wrappings
(wrapping means: for example calling AFSVolRestore with "magic" parameters instead of creating new RPCs)

Global algorythm:
'A'-side:
-> AFSVolForward[Multi]:
	- create C1)
	- create C3)
	- enumerate vnodes:
		- send block checksums of a vnode through C1), can be multiple sending
	- wait for end C3) and destrory
	- wait for end C1) and destrory
	- END

-> "StartAFSVolQueryDiff" C2):
	- while [rx_Read] "Vnode-diff request list":
		- read Vnode content parts locally on the disk
		- send diff data through C3)
	- END

'Bi'-side:
-> AFSVolRestore C1):
	- if got a special dump version with magic, continue in rsync-like mode (otherwise parse dump normally)
	- while [rx_Read] "Vnode block checksums":
		- generate difference request list (reading the local Vnode content, comparing to the input block checksums) using the rsync-algorythm
		- send difference request list through C2)
	- END

-> "StartAFSVolSendDiffData" C3):
	- create C2)
	- while [rx_Read] "Vnode-diff data":
		- modify Vnode locally on the disk
	- wait for end C2) and destrory
	- END




--Boundary-00=_LndGC5TiIJf+XzQ--