[OpenAFS] problem in vos release
Lee Damon
nomad@ssli-mail.ee.washington.edu
Tue, 15 Apr 2003 15:50:45 -0700
> You should vos zap kinetic /vicepf 1443364915; vos addsite, then vos
> release.
...
> Steven
Thanks for the suggestions, but sadly it didn't work. :(
: || stefen [347] ; vos zap kinetic /vicepf 1443364915
Warning: Entry for volume number 1443364915 exists in VLDB (but we're zapping
it anyway!)
Could not start transaction on volume 1443364915
Volume not attached, does not exist, or not on line
Error in vos zap command.
Volume not attached, does not exist, or not on line
: || stefen [348] ; vos addsite kinetic /vicepf v.s8.cadence.2002.ic
Added replication site kinetic /vicepf for volume v.s8.cadence.2002.ic
: || stefen [349] ; vos rel -verbose !$
vos rel -verbose v.s8.cadence.2002.ic
v.s8.cadence.2002.ic
RWrite: 1443364914 ROnly: 1443364915
number of sites -> 3
server potential.ee.washington.edu partition /vicepf RW Site
server potential.ee.washington.edu partition /vicepf RO Site
server kinetic.ee.washington.edu partition /vicepf RO Site -- Not
released
This is a complete release of the volume 1443364914
Recloning RW volume ...
Creating new volume 1443364915 on replication site kinetic.ee.washington.edu:
done.
Starting ForwardMulti from 1443364915 to 1443364915 on
kinetic.ee.washington.edu (full release).
Release failed: VOLSER: Problems encountered in doing the dump !
The volume 1443364914 could not be released to the following 1 sites:
kinetic.ee.washington.edu /vicepf
VOLSER: release could not be completed
Error in vos release command.
VOLSER: release could not be completed
>
> -----Original Message-----
> From: Lee Damon [mailto:nomad@ssli-mail.ee.washington.edu]
> Sent: Tuesday, April 15, 2003 1:56 PM
> To: Jenkins, Steven
> Subject: Re: [OpenAFS] problem in vos release
>
>
> (I thought I had 8 volumes replicated there, but I only have 7. Ah well
> :)
>
> These steps were all done just after running a syncvldb and syncserv
> against kinetic:/vicepf.
>
> : || stefen [221] ; vos stat kinetic
> No active transactions on kinetic
> : || stefen [222] ; vos listvol kinetic /vicepf
> Total number of volumes on server kinetic partition /vicepf: 7
> v.s8.cadence.2000.readonly 536872893 RO 5005012 K On-line
> v.s8.cadence.2002.readonly 536872896 RO 10982006 K On-line
> v.s8.purify.readonly 536872956 RO 396015 K On-line
> v.s8.synopsys.200011.readonly 536872962 RO 2 K On-line
> v.s8.synopsys.2002.readonly 536872965 RO 2 K On-line
> v.s8.synopsys.200205.readonly 536872968 RO 2 K On-line
> v.s8.synopsys.readonly 536872959 RO 5 K On-line
>
> Total volumes onLine 7 ; Total volumes offLine 0 ; Total busy 0
>
> : || stefen [223] ; vos remove kinetic /vicepf 1443364915 Volume
> 1443364915 does not exist on server and partition
> VOLSER: no such volume - location specified incorrectly or volume does
> not
> exist
> Error in vos remove command.
> VOLSER: no such volume - location specified incorrectly or volume does
> not
> exist
> : || stefen [224] ; vos addsite kinetic /vicepf v.s8.cadence.2002.ic
> Added replication site kinetic /vicepf for volume v.s8.cadence.2002.ic
> : || stefen [225] ; vos exa !$
> vos exa v.s8.cadence.2002.ic
> v.s8.cadence.2002.ic 1443364914 RW 2298260 K On-line
> potential.ee.washington.edu /vicepf
> RWrite 1443364914 ROnly 1443364915 Backup 0
> MaxQuota 50000000 K
> Creation Tue Apr 8 16:43:28 2003
> Last Update Thu Apr 10 10:50:53 2003
> 0 accesses in the past day (i.e., vnode references)
>
> RWrite: 1443364914 ROnly: 1443364915
> number of sites -> 3
> server potential.ee.washington.edu partition /vicepf RW Site
> server potential.ee.washington.edu partition /vicepf RO Site
> server kinetic.ee.washington.edu partition /vicepf RO Site --
> Not
> released
> : || stefen [226] ; time vos rel -verbose !$
> time vos rel -verbose v.s8.cadence.2002.ic
>
> v.s8.cadence.2002.ic
> RWrite: 1443364914 ROnly: 1443364915
> number of sites -> 3
> server potential.ee.washington.edu partition /vicepf RW Site
> server potential.ee.washington.edu partition /vicepf RO Site
> server kinetic.ee.washington.edu partition /vicepf RO Site --
> Not
> released
> This is a complete release of the volume 1443364914
> Recloning RW volume ...
> Creating new volume 1443364915 on replication site
> kinetic.ee.washington.edu:
> done.
> Starting ForwardMulti from 1443364915 to 1443364915 on
> kinetic.ee.washington.edu (full release).
> Release failed: VOLSER: Problems encountered in doing the dump ! The
> volume 1443364914 could not be released to the following 1 sites:
> kinetic.ee.washington.edu /vicepf
> VOLSER: release could not be completed
> Error in vos release command.
> VOLSER: release could not be completed
> 0.003u 0.001s 0:29.91 0.0% 0+0k 0+0io 204pf+0w
> : || stefen [227] ; vos exa v.s8.cadence.2002.ic
> v.s8.cadence.2002.ic 1443364914 RW 2298260 K On-line
> potential.ee.washington.edu /vicepf
> RWrite 1443364914 ROnly 1443364915 Backup 0
> MaxQuota 50000000 K
> Creation Tue Apr 8 16:43:28 2003
> Last Update Thu Apr 10 10:50:53 2003
> 0 accesses in the past day (i.e., vnode references)
>
> RWrite: 1443364914 ROnly: 1443364915 RClone: 1443364915
> number of sites -> 3
> server potential.ee.washington.edu partition /vicepf RW Site --
> New
> release
> server potential.ee.washington.edu partition /vicepf RO Site --
> New
> release
> server kinetic.ee.washington.edu partition /vicepf RO Site --
> Old
> release
> : || stefen [228] ; vos remove kinetic /vicepf 1443364915 Volume
> 1443364915 on partition /vicepf server kinetic.ee.washington.edu deleted
> : || stefen [229] ; vos exa v.s8.cadence.2002.ic
> v.s8.cadence.2002.ic 1443364914 RW 2298260 K On-line
> potential.ee.washington.edu /vicepf
> RWrite 1443364914 ROnly 1443364915 Backup 0
> MaxQuota 50000000 K
> Creation Tue Apr 8 16:43:28 2003
> Last Update Thu Apr 10 10:50:53 2003
> 0 accesses in the past day (i.e., vnode references)
>
> RWrite: 1443364914 ROnly: 1443364915 RClone: 1443364915
> number of sites -> 2
> server potential.ee.washington.edu partition /vicepf RW Site --
> New
> release
> server potential.ee.washington.edu partition /vicepf RO Site --
> New
> release
> : || stefen [230] ; time vos rel -verbose v.s8.cadence.2002.ic
>
> v.s8.cadence.2002.ic
> RWrite: 1443364914 ROnly: 1443364915 RClone: 1443364915
> number of sites -> 2
> server potential.ee.washington.edu partition /vicepf RW Site --
> New
> release
> server potential.ee.washington.edu partition /vicepf RO Site --
> New
> release
> This is a completion of the previous release
> updating VLDB ... done
> Released volume v.s8.cadence.2002.ic successfully
> 0.001u 0.001s 0:00.40 0.0% 0+0k 0+0io 205pf+0w
> : || stefen [231] ; vos exa v.s8.cadence.2002.ic
> v.s8.cadence.2002.ic 1443364914 RW 2298260 K On-line
> potential.ee.washington.edu /vicepf
> RWrite 1443364914 ROnly 1443364915 Backup 0
> MaxQuota 50000000 K
> Creation Tue Apr 8 16:43:28 2003
> Last Update Thu Apr 10 10:50:53 2003
> 0 accesses in the past day (i.e., vnode references)
>
> RWrite: 1443364914 ROnly: 1443364915
> number of sites -> 2
> server potential.ee.washington.edu partition /vicepf RW Site
> server potential.ee.washington.edu partition /vicepf RO Site
>
>
> I'm attaching the iostat output. Here's the tail end of the tcpdump
> output. I'm not seeing anything particularly revelatory in them.
>
> 10:49:43.118999 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44027:1480@1480+)
> 10:49:43.119112 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (66) (DF)
> 10:49:43.119123 potential.ee.washington.edu.afs3-volser >
> kinetic.ee.washington.edu.afs3-volser: rx data (1472) (frag
> 44027:1480@0+) 10:49:43.119228 potential.ee.washington.edu >
> kinetic.ee.washington.edu: (frag
> 44028:1256@4440)
> 10:49:43.119376 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44028:1480@2960+)
> 10:49:43.119508 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44028:1480@1480+)
> 10:49:43.119783 potential.ee.washington.edu.afs3-volser >
> kinetic.ee.washington.edu.afs3-volser: rx data (1472) (frag
> 44028:1480@0+) 10:49:43.120948 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (69) (DF)
> 10:49:43.120975 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (66) (DF)
> 10:49:43.121022 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (69) (DF)
> 10:49:43.121077 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (66) (DF)
> 10:49:43.121124 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (69) (DF)
> 10:49:43.121148 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (66) (DF)
> 10:49:43.122203 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44029:1256@4440)
> 10:49:43.122442 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44029:1480@2960+)
> 10:49:43.122566 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44029:1480@1480+)
> 10:49:43.122687 potential.ee.washington.edu.afs3-volser >
> kinetic.ee.washington.edu.afs3-volser: rx data (1472) (frag
> 44029:1480@0+) 10:49:43.122799 potential.ee.washington.edu >
> kinetic.ee.washington.edu: (frag
> 44030:1256@4440)
> 10:49:43.122916 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44030:1480@2960+)
> 10:49:43.123039 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44030:1480@1480+)
> 10:49:43.123091 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (69) (DF)
> 10:49:43.123119 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (66) (DF)
> 10:49:43.123162 potential.ee.washington.edu.afs3-volser >
> kinetic.ee.washington.edu.afs3-volser: rx data (1472) (frag
> 44030:1480@0+) 10:49:43.123179 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (69) (DF)
> 10:49:43.123233 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx ack (66) (DF)
> 10:49:43.123267 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44031:1256@4440)
> 10:49:43.123391 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44031:1480@2960+)
> 10:49:43.123520 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44031:1480@1480+)
> 10:49:43.123652 potential.ee.washington.edu.afs3-volser >
> kinetic.ee.washington.edu.afs3-volser: rx data (1472) (frag
> 44031:1480@0+) 10:49:43.124383 potential.ee.washington.edu >
> kinetic.ee.washington.edu: (frag
> 44032:1256@4440)
> 10:49:43.124584 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44032:1480@2960+)
> 10:49:43.124730 potential.ee.washington.edu > kinetic.ee.washington.edu:
> (frag
> 44032:1480@1480+)
> 10:49:43.125199 kinetic.ee.washington.edu.afs3-volser >
> potential.ee.washington.edu.afs3-volser: rx abort vol reply restore
> errcode
> 1492325122 (32) (DF)
>
> thanks for the suggestions,
> nomad
>
>
> >
> > Not good. This is the same volume you had trouble with taking a while
>
> > to release that you mentioned a few days ago?
> >
> > My suggestion would be:
> >
> > - vos stat your host until it has no transactions
> > - vos listvol the bad host, checking to see if the volume is there.
> > - assuming it's there, vos remove it (vos zap & vos remsite only as a
> > last resort)
> > - vos addsite it back
> > - vos release it again, -verbose this time. Capture iostat & network
> stats of your server. If you can, get tcpdump data (or ethereal, which
> can decode AFS packets quite well).
> >
> > But _before_ you do that, replicate another (small!) volume on that
> > partition and do a vos release, just to make sure everything is ok
> > w/your server & partition (my apologies if you've already done that --
>
> > I haven't read your previous posts but just scanned the headers).
> >
> > Steven
> >
> >
> > -----Original Message-----
> > From: Lee Damon [mailto:nomad@ssli-mail.ee.washington.edu]
> > Sent: Tue 4/15/2003 12:58 PM
> > To: Jenkins, Steven
> > Cc:
> > Subject: Re: [OpenAFS] problem in vos release
> > Tue Apr 15 08:50:38 2003 1 Volser: ReadVnodes: Restore aborted Tue Apr
>
> > 15 08:51:11 2003 1 Volser: Delete: volume 1443364915 deleted Tue Apr
> > 15 08:51:47 2003 1 Volser: CreateVolume: volume 1443364915
> > (v.s8.cadence
> > .2002.ic.readonly) created
> > Tue Apr 15 08:52:10 2003 1 Volser: ReadVnodes: Restore aborted Tue Apr
>
> > 15 08:52:26 2003 1 Volser: Delete: volume 1443364915 deleted Tue Apr
> > 15 09:25:28 2003 1 Volser: CreateVolume: volume 1443364915
> > (v.s8.cadence
> > .2002.ic.readonly) created
> > Tue Apr 15 09:25:51 2003 1 Volser: ReadVnodes: Restore aborted Tue Apr
>
> > 15 09:27:14 2003 1 Volser: Delete: volume 1443364915 deleted Tue Apr
> > 15 09:27:26 2003 1 Volser: CreateVolume: volume 1443364915
> > (v.s8.cadence
> > .2002.ic.readonly) created
> > Tue Apr 15 09:32:52 2003 trans 75064 on volume 1443364915 is older
> > than 300
> > seco
> > nds
> > Tue Apr 15 09:33:22 2003 trans 75064 on volume 1443364915 is older
> than 330
> > seco
> > nds
> > Tue Apr 15 09:33:52 2003 trans 75064 on volume 1443364915 is older
> than 360
> > seco
> > nds
> > Tue Apr 15 09:34:22 2003 trans 75064 on volume 1443364915 is older
> than 390
> > seco
> > nds
> > Tue Apr 15 09:39:42 2003 1 Volser: Delete: volume 1443364915 deleted
> > Tue Apr 15 09:40:04 2003 1 Volser: CreateVolume: volume 1443364915
> > (v.s8.cadence
> > .2002.ic.readonly) created
> > Tue Apr 15 09:40:36 2003 1 Volser: ReadVnodes: Restore aborted
> > Tue Apr 15 09:43:01 2003 1 Volser: Delete: volume 1443364915 deleted
> >
> >
> > > What does your volserver log on that box look like?
> > >
> > > Steven
> > > -----Original Message-----
> > > From: Lee Damon [mailto:nomad@ssli-mail.ee.washington.edu]
> > > Sent: Tuesday, April 15, 2003 12:48 PM
> > > To: openafs-info@openafs.org
> > > Subject: Re: [OpenAFS] problem in vos release
> > >
> > >
> > > I forgot to say.. /vicepf has 91GB available, so that's not the
> > > problem.
> > > :)
> > >
> > > nomad
> > >
> > > > Any suggestions on how I can find out what is going on here? I
> > > > can
> > > > vos release this volume to other fileservers, I can even vos
> release
> > > > it to a different /vicep? partition on the 'failing' server, but I
>
> > > > can't release it to the place I need it to live.
> > > >
> > > > Any and all help would be much appreciated.
> > > >
> > > > : || stefen [198] ; vos addsite kinetic /vicepf
> > > > v.s8.cadence.2002.ic
> > > > Added replication site kinetic /vicepf for volume
> v.s8.cadence.2002.ic
> > > > : || stefen [199] ; vos exa 1443364915
> > > > v.s8.cadence.2002.ic.readonly 1443364915 RO 2298260 K
> On-line
> > > > potential.ee.washington.edu /vicepf
> > > > RWrite 1443364914 ROnly 1443364915 Backup 0
> > > > MaxQuota 50000000 K
> > > > Creation Tue Apr 15 08:51:42 2003
> > > > Last Update Tue Apr 15 08:51:42 2003
> > > > 1668 accesses in the past day (i.e., vnode references)
> > > >
> > > > Could not fetch the information about volume 1443364915 from the
> > > > server
> > > > : No such device
> > > > Volume does not exist on server kinetic.ee.washington.edu as
> indicated
> > > by the
> > > > VLDB
> > > >
> > > > RWrite: 1443364914 ROnly: 1443364915 RClone: 1443364915
> > > > number of sites -> 3
> > > > server potential.ee.washington.edu partition /vicepf RW
> > > > Site
> > > > -- New
> > > > release
> > > > server potential.ee.washington.edu partition /vicepf RO
> Site
> > > -- New
> > > > release
> > > > server kinetic.ee.washington.edu partition /vicepf RO Site
>
> > > > --
> > > Old
> > > > release
> > > > : || stefen [200] ; vos rel -verbose v.s8.cadence.2002.ic
> > > >
> > > > v.s8.cadence.2002.ic
> > > > RWrite: 1443364914 ROnly: 1443364915 RClone: 1443364915
> > > > number of sites -> 3
> > > > server potential.ee.washington.edu partition /vicepf RW
> > > > Site
> > > > -- New
> > > > release
> > > > server potential.ee.washington.edu partition /vicepf RO
> Site
> > > -- New
> > > > release
> > > > server kinetic.ee.washington.edu partition /vicepf RO Site
>
> > > > --
> > > Old
> > > > release
> > > > This is a completion of the previous release
> > > > Creating new volume 1443364915 on replication site
> > > kinetic.ee.washington.edu:
> > > > done.
> > > > Starting ForwardMulti from 1443364915 to 1443364915 on
> > > > kinetic.ee.washington.edu (full release).
> > > > Release failed: VOLSER: Problems encountered in doing the dump !
> > > > The volume 1443364914 could not be released to the following 1
> sites:
> > > > kinetic.ee.washington.edu /vicepf
> > > > VOLSER: release could not be completed
> > > > Error in vos release command.
> > > > VOLSER: release could not be completed
> > > > : || stefen [201] ; vos remove kinetic /vicepf 1443364915
> > > > Volume 1443364915 on partition /vicepf server
> > > kinetic.ee.washington.edu deleted
> > > > : || stefen [202] ; vos addsite kinetic /vicepe
> > > > v.s8.cadence.2002.ic Added replication site kinetic /vicepe for
> > > > volume v.s8.cadence.2002.ic
> > > > : || stefen [203] ; vos rel -verbose v.s8.cadence.2002.ic
> > > >
> > > > v.s8.cadence.2002.ic
> > > > RWrite: 1443364914 ROnly: 1443364915 RClone: 1443364915
> > > > number of sites -> 3
> > > > server potential.ee.washington.edu partition /vicepf RW
> > > > Site
> > > > -- New
> > > > release
> > > > server potential.ee.washington.edu partition /vicepf RO
> Site
> > > -- New
> > > > release
> > > > server kinetic.ee.washington.edu partition /vicepe RO Site
>
> > > > --
> > > Old
> > > > release
> > > > This is a completion of the previous release
> > > > Creating new volume 1443364915 on replication site
> > > kinetic.ee.washington.edu:
> > > > done.
> > > > Starting ForwardMulti from 1443364915 to 1443364915 on
> > > > kinetic.ee.washington.edu (full release).
> > > > updating VLDB ... done
> > > > Released volume v.s8.cadence.2002.ic successfully
> > > > : || stefen [204] ; vos remove kinetic /vicepe
> > > v.s8.cadence.2002.ic.readonly
> > > > Volume 1443364915 on partition /vicepe server
> > > kinetic.ee.washington.edu deleted
> > > > : || stefen [205] ; vos addsite kinetic /vicepf
> > > > v.s8.cadence.2002.ic Added replication site kinetic /vicepf for
> > > > volume v.s8.cadence.2002.ic
> > > > : || stefen [206] ; time vos rel -verbose v.s8.cadence.2002.ic
> > > >
> > > > v.s8.cadence.2002.ic
> > > > RWrite: 1443364914 ROnly: 1443364915
> > > > number of sites -> 3
> > > > server potential.ee.washington.edu partition /vicepf RW
> Site
> > > > server potential.ee.washington.edu partition /vicepf RO
> Site
> > > > server kinetic.ee.washington.edu partition /vicepf RO Site
>
> > > > --
> > > > Not
> > > > released
> > > > This is a complete release of the volume 1443364914
> > > > Recloning RW volume ...
> > > > Creating new volume 1443364915 on replication site
> > > kinetic.ee.washington.edu:
> > > > done.
> > > > Starting ForwardMulti from 1443364915 to 1443364915 on
> > > > kinetic.ee.washington.edu (full release).
> > > > Release failed: VOLSER: Problems encountered in doing the dump !
> > > > The volume 1443364914 could not be released to the following 1
> sites:
> > > > kinetic.ee.washington.edu /vicepf
> > > > VOLSER: release could not be completed
> > > > Error in vos release command.
> > > > VOLSER: release could not be completed
> > > > 0.003u 0.000s 0:38.75 0.0% 0+0k 0+0io 204pf+0w
> > > > : || stefen [207] ; vos exa v.s8.cadence.2002.ic
> > > > v.s8.cadence.2002.ic 1443364914 RW 2298260 K
> On-line
> > > > potential.ee.washington.edu /vicepf
> > > > RWrite 1443364914 ROnly 1443364915 Backup 0
> > > > MaxQuota 50000000 K
> > > > Creation Tue Apr 8 16:43:28 2003
> > > > Last Update Thu Apr 10 10:50:53 2003
> > > > 0 accesses in the past day (i.e., vnode references)
> > > >
> > > > RWrite: 1443364914 ROnly: 1443364915 RClone: 1443364915
> > > > number of sites -> 3
> > > > server potential.ee.washington.edu partition /vicepf RW
> > > > Site
> > > > -- New
> > > > release
> > > > server potential.ee.washington.edu partition /vicepf RO
> Site
> > > -- New
> > > > release
> > > > server kinetic.ee.washington.edu partition /vicepf RO Site
>
> > > > --
> > > Old
> > > > release
> > > >
> > > >
> > > > thanks,
> > > > nomad
> > > > ----------- - Lee "nomad" Damon -
> \
> > > > work: nomad@ee.washington.edu
> \
> > > > play: nomad@castle.org or castle!nomad
> \
> > > >
> /\
> > > > Sr. Systems Admin, UWEE SSLI Lab
> /
> > > \
> > > > "Celebrate Diversity"
> /
> > > \
> > > >
> > > >
> > > > _______________________________________________
> > > > OpenAFS-info mailing list
> > > > OpenAFS-info@openafs.org
> > > > https://lists.openafs.org/mailman/listinfo/openafs-info
> > > nomad
> > > ----------- - Lee "nomad" Damon - \
> > > work: nomad@ee.washington.edu \
> > > play: nomad@castle.org or castle!nomad
> \
> > >
> /\
> > > Sr. Systems Admin, UWEE SSLI Lab /
> \
> > > "Celebrate Diversity" /
> \
> > >
> > >
> > > _______________________________________________
> > > OpenAFS-info mailing list
> > > OpenAFS-info@openafs.org
> > > https://lists.openafs.org/mailman/listinfo/openafs-info
> > >
> > nomad
> > ----------- - Lee "nomad" Damon - \
> > work: nomad@ee.washington.edu \
> > play: nomad@castle.org or castle!nomad \
> > /\
> > Sr. Systems Admin, UWEE SSLI Lab /
> \
> > "Celebrate Diversity" /
> \
> >
> >
> >
> >
> >
> >
> nomad
> ----------- - Lee "nomad" Damon - \
> work: nomad@ee.washington.edu \
> play: nomad@castle.org or castle!nomad \
> /\
> Sr. Systems Admin, UWEE SSLI Lab / \
> "Celebrate Diversity" / \
>
>
nomad
----------- - Lee "nomad" Damon - \
work: nomad@ee.washington.edu \
play: nomad@castle.org or castle!nomad \
/\
Sr. Systems Admin, UWEE SSLI Lab / \
"Celebrate Diversity" / \