[OpenAFS] recover data from corrupted volume

[Dimitris Z]

Hi,

I have been asked to attempt recovery an old soon to be decommissioned
openafs server. It is running openafs-server-1.4.7-68.2.SL4.x86_64 on
Scientific Linux 4.4.

The RAID10 array (3ware Inc 9550SX)  where vipepX partitions lie went
completely missing while replacing a faulty drive (ECC error). We put
the fauled drive back inside in order to be able to access the array
again and rsycned all the vicepX date elsewhere. This was completed
successfully. Errors logged meanwhile in the kernel logs mentioned
sectors being repaired.

The array was then deleted and remade with new disks and the data was
rsynced back. However some volumes have not come online. Salvaging
them causes them to deleted. These volumes have been off-line before
the disk replacement attemp - so it is likely they have been corrupted
for some time.

bash-3.2$ bos salvage -server afs1 -partition vicepe -volume uk  -showlog
Starting salvage.
bos: waiting for salvage to complete.
bos: salvage completed
SalvageLog:
@(#) OpenAFS 1.4.7 built 2009-04-07 (68.2.SL4@fnal.gov)
01/16/2013 17:14:45 STARTING AFS SALVAGER 2.4 (/usr/afs/bin/salvager
/vicepe 536874966)
01/16/2013 17:14:45 2 nVolumesInInodeFile 64
01/16/2013 17:14:45 CHECKING CLONED VOLUME 0.
01/16/2013 17:14:45 uk (536874966) updated 01/09/2013 17:13
01/16/2013 17:14:45 No header file for volume 0
01/16/2013 17:14:45 totalInodes 6446
-bash-3.2$ bos salvage -server afs1 -partition vicepe -volume uk  -showlog
Starting salvage.
bos: salvage completed
SalvageLog:
@(#) OpenAFS 1.4.7 built 2009-04-07 (68.2.SL4@fnal.gov)
01/16/2013 17:15:22 STARTING AFS SALVAGER 2.4 (/usr/afs/bin/salvager
/vicepe 536874966)
01/16/2013 17:15:22 The volume header file V0536874966.vol is not
associated with any actual data (deleted)
01/16/2013 17:15:22 No applicable vice inodes on vicepe; not salvaged
01/16/2013 17:15:22 0 nVolumesInInodeFile 0
Temporary file /vicepe/salvage.inodes.vicepe.16945 is missing...

/usr/sbin/vos examine uk
Could not fetch the information about volume 536874966 from the server
: No such device
Volume does not exist on server afs1 as indicated by the VLDB

Dump only information from VLDB

uk
    RWrite: 536874966
    number of sites -> 1
       server afs1 partition /vicepe RW Site


My question is, are there any other options to at least partially
recover data from the offline volumes? I do have my last rsync of
vicepe where this volume was. It was probably damaged before it was
backed up but even a partial data recovery would help.

Best regards,