[OpenAFS] Volumes that need to be salvaged

Jason Mader jason+openafs@ncac.gwu.edu
Mon, 5 Sep 2005 17:19:53 -0400


On Sep 5, 2005, at 3:21 PM, Esther Filderman wrote:

> On 9/5/05, Jason Mader <jason+openafs@ncac.gwu.edu> wrote:
>
>> @(#) OpenAFS 1.4.0-rc3 built  2005-09-05
>> 09/05/2005 10:49:23 STARTING AFS SALVAGER 2.4 (/usr/local/libexec/
>> openafs/salvager)
>> 09/05/2005 10:49:23 Starting salvage of file system partition /vicepa
>> 09/05/2005 10:49:23 SALVAGING FILE SYSTEM PARTITION /vicepa
>> (device=dks0d2s7)
>> 09/05/2005 10:49:29 Wrong size (2048 instead of 2560) in inode file
>> for <null>
>> 09/05/2005 10:49:29 *** I/O error 1009 when writing a tmp inode  
>> file /
>> vicepa/salvage.inodes.dks0d2s7.303; Not salvaged dks0d2s7 ***
>> Increase space on partition or use '-tmpdir'
>>
>> Mon Sep  5 10:49:30 2005 XFS/EFS File server starting
>> Mon Sep  5 10:49:30 2005 Set thread id 14 for FSYNC_sync
>> Mon Sep  5 10:49:30 2005 Partition /vicepa: attaching volumes
>> Mon Sep  5 10:49:30 2005 VAttachVolume: volume /vicepa/
>> V0536870927.vol needs to be salvaged; not attached.
>
> /vicepa is either full or has a [physical] bad spot on it.  Try moving
> some other volumes from it and then salvaging the volume ["bos salvage
> yourservername a 536870927" will do it without taking the rest of the
> fileserver down].

/vicepa isn't full and isn't reporting physical errors (none are  
reported by the OS, or by a read-only butterfly scan of the disk done  
with fx).  Also, the volumes are teeny and this is a brand new test  
Cell.

     Filesystem             Type  Size   use  avail  %use  Mounted on
     /dev/dsk/dks0d2s7       xfs   17G  836K    17G    1%  /vicepa

So, I did "bos salvage udon a 536870927" and volume 536870927 is now  
gone from the system.

Since this error is about inodes, I used xfs_db to get the options  
set with mkfs_xfs for this filesystem,
     logblocks = 4000
     inodesize = 512
and xfs_size_check had no problem either.

Here's the same done on the next volume that needed salvaging:

% /usr/local/sbin/vos listvol udon
Total number of volumes on server udon partition /vicepa: 6
root.afs.readonly                 536870928 RO          5 K On-line
root.cell.readonly                536870931 RO          3 K On-line
**** Could not attach volume 536870939 ****
**** Could not attach volume 536870936 ****
**** Could not attach volume 536870933 ****
**** Could not attach volume 536870930 ****

Total volumes onLine 2 ; Total volumes offLine 4 ; Total busy 0

% sudo ls -l /vicepa
total 48
-rw-------    1 root     sys           76 Sep  4 14:03 V0536870928.vol
-rw-------    1 root     sys           76 Sep  4 14:00 V0536870930.vol
-rw-------    1 root     sys           76 Sep  4 14:03 V0536870931.vol
-rw-------    1 root     sys           76 Sep  4 14:13 V0536870933.vol
-rw-------    1 root     sys           76 Sep  4 14:15 V0536870936.vol
-rw-------    1 root     sys           76 Sep  4 22:23 V0536870939.vol

% bos salvage udon a 536870930 -showlog
Starting salvage.
bos: salvage completed
SalvageLog:
@(#) OpenAFS 1.4.0-rc3 built  2005-09-05
09/05/2005 17:01:56 STARTING AFS SALVAGER 2.4 (/usr/local/libexec/ 
openafs/salvager /vicepa 536870930)
09/05/2005 17:01:59 The volume header file V0536870931.vol is not  
associated with any actual data (deleted)
09/05/2005 17:01:59 No applicable vice inodes on dks0d2s7; not salvaged
09/05/2005 17:01:59 0 nVolumesInInodeFile 0
Temporary file /vicepa/salvage.inodes.dks0d2s7.19353 is missing...

% /usr/local/sbin/vos listvol udon
Total number of volumes on server udon partition /vicepa: 4
root.afs.readonly                 536870928 RO          5 K On-line
**** Could not attach volume 536870939 ****
**** Could not attach volume 536870936 ****
**** Could not attach volume 536870933 ****

Total volumes onLine 1 ; Total volumes offLine 3 ; Total busy 0

% sudo ls -l /vicepa
total 32
-rw-------    1 root     sys           76 Sep  4 14:03 V0536870928.vol
-rw-------    1 root     sys           76 Sep  4 14:13 V0536870933.vol
-rw-------    1 root     sys           76 Sep  4 14:15 V0536870936.vol
-rw-------    1 root     sys           76 Sep  4 22:23 V0536870939.vol

---Jason Mader, FHWA/NHTSA National Crash Analysis Center,
The George Washington University, VA Campus