[OpenAFS] Help needed for receovery of data of inode fileserver (Solaris 10 x86)

[John Tang Boyland]

I mentioned a few weeks ago about an OpenAFS 1.4.6 Solaris 10 x86 inode
fileserver that refused to attach any volumes and for which the salavager
just coredumps (without leaving any core files?)  Repeated salvaging does
nothing except remove a few more vnodes.  On the other hand, umounting and 
fsck shows that everything is fine.

Is there any way to get some of the data out of the inode fileserver
partitions?  It's very frustrating because some things weren't
backed up (yes, yes, ...).  The -ForceOnLine option to bos salvage
looked promising but that's only available for MR AFS.

The only thing I can think of is labeling the partition as UFS and
then have the built-in fsck drop everything into lost+found,
but I'm not sure if any of the structure will be recoverable.
Is there any way to just get the data out of the inode partition?
(since fsck is happy with it.)

I even tried reinstalling openafs-1.4.1 to see if some horrible new
bug was introduced, but the same result happened.

John

documentation: 

SalvageLog starts:
@(#) OpenAFS 1.4.6 built  2007-12-17 
04/01/2008 17:15:21 STARTING AFS SALVAGER 2.4 (/usr/afs/bin/salvager -f)
04/01/2008 17:15:21 Starting salvage of file system partition /vicepa
04/01/2008 17:15:21 SALVAGING FILE SYSTEM PARTITION /vicepa (device=c1t1d0s6)
04/01/2008 17:15:21 ***Forced salvage of all volumes on this partition***
04/01/2008 17:15:31 Scanning inodes on device /dev/rdsk/c1t1d0s6...
04/01/2008 17:17:40 242 nVolumesInInodeFile 6776 
04/01/2008 17:17:40 SALVAGING VOLUME 536870912.
04/01/2008 17:17:40 Part of the header (Volume information) is corrupted
04/01/2008 17:17:40 totalInodes 165
04/01/2008 17:17:41 "Salvage volume group" core dumped!
04/01/2008 17:17:41 SALVAGING VOLUME 536870915.
04/01/2008 17:17:41 Part of the header (Volume information) is corrupted
04/01/2008 17:17:41 totalInodes 30
04/01/2008 17:17:41 "Salvage volume group" core dumped!
04/01/2008 17:17:41 SALVAGING 
04/01/2008 17:17:42 totalInodes 526
04/01/2008 17:17:43 "Salvage volume group" core dumped!
04/01/2008 17:17:43 SALVAGING VOLUME 536870927.
04/01/2008 17:17:43 Part of the header (Volume information) is corrupted
04/01/2008 17:17:43 Vnode 1922 (unique 1437): corresponding inode 22233511 is missing; vnode deleted, vnode mod time=Tue Nov 24 08:23:15 1998
04/01/2008 17:17:43 Vnode 1924 (unique 1438): corresponding inode 22233512 is missing; vnode deleted, vnode mod time=Tue Nov 24 08:23:15 1998
04/01/2008 17:17:43 Vnode 1926 (unique 431951): corresponding inode 22233513 is missing; vnode deleted, vnode mod time=Sun Mar 30 22:33:54 2003
04/01/2008 17:17:43 Vnode 1928 (unique 434559): corresponding inode 22233514 is missing; vnode deleted, vnode mod time=Sun Apr  6 23:02:50 2003
04/01/2008 17:17:43 Vnode 1930 (unique 431952): corresponding inode 22233515 is missing; vnode deleted, vnode mod time=Sun Mar 30 22:34:15 2003
...

# fsck /vicepa
----Open AFS (R) openafs 1.4.1 fsck----
** /dev/rdsk/c1t1d0s6
** Last Mounted on /vicepa
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
849922 files, 18516628 used, 39053375 free, 849677 AFS files (89095 frags, 4870535 blocks, 0.2% fragmentation)