[OpenAFS] DB elections failing

Matthew Cocker matt@cs.auckland.ac.nz
Mon, 12 Jan 2004 10:05:56 +1300


Hi

On Sunday morning at 2.38 am our DBs boxes started fqailing to elect a 
sync site. Now if we force a reelection we see the floowing for a little 
while

[root@afs-db1 sysconfig]# udebug localhost 7003
Host's addresses are: 130.216.35.2
Host's 127.0.0.1 time is Mon Jan 12 09:46:09 2004
Local time is Mon Jan 12 09:46:10 2004 (time differential 1 secs)
Last yes vote for 130.216.35.2 was 6 secs ago (not sync site);
Last vote started 6 secs ago (at Mon Jan 12 09:46:04 2004)
Local db version is 1073605010.28
I am sync site until -112086 secs from now (at Sun Jan 11 02:38:04 2004) 
(3 servers)
Recovery state 0
Sync site's db version is 1073605010.28
0 locked pages, 0 of them for write

Server (130.216.35.13): (db 0.0)
     last vote never rcvd
     last beacon never sent
     dbcurrent=0, up=0 beaconSince=0

Server (130.216.207.55): (db 0.0)
     last vote rcvd 6 secs ago (at Mon Jan 12 09:46:04 2004),
     last beacon sent 6 secs ago (at Mon Jan 12 09:46:04 2004), last 
vote was yes
     dbcurrent=0, up=1 beaconSince=1

Then it reverts back to this

[root@afs-db1 sysconfig]# udebug localhost 7003
Host's addresses are: 130.216.35.2
Host's 127.0.0.1 time is Mon Jan 12 09:46:11 2004
Local time is Mon Jan 12 09:46:11 2004 (time differential 0 secs)
Last yes vote for 130.216.35.2 was 8 secs ago (not sync site);
Last vote started 8 secs ago (at Mon Jan 12 09:46:03 2004)
Local db version is 1073605010.28
I am not sync site
Lowest host 130.216.35.2 was set 8 secs ago
Sync host 0.0.0.0 was set 1073853971 secs ago
Sync site's db version is 1073605010.28
0 locked pages, 0 of them for write
[root@afs-db1 sysconfig]#


Have we got a corrupted DB? Vos listaddrs has been giving the following 
output for a couple of days. At the moment people can still use afs 
space (read/write), but backups now longer work.

[root@afs-db1 logs]# vos listaddrs -local
afs-01-cs-ec.ec.auckland.ac.nz
afs-02-tmk-ec.ec.auckland.ac.nz
afs-03-tmk-ec.ec.auckland.ac.nz
afs-08-cs-ec.ec.auckland.ac.nz
afs-04-cs-ec.ec.auckland.ac.nz
afs-05-cs-ec.ec.auckland.ac.nz
afs-07-cs-ec.ec.auckland.ac.nz
afs-db1.ec.auckland.ac.nz
afs-09-tcs-ec.ec.auckland.ac.nz
afs-06-cs-ec.ec.auckland.ac.nz
afs-04-tcs-ec.ec.auckland.ac.nz
afs-10-ms-ec.ec.auckland.ac.nz
afs-11-fos-ec.ec.auckland.ac.nz
afs-12-cs-ec.ec.auckland.ac.nz
afs-13-cs-ec.ec.auckland.ac.nz
afs-14-se-ec.ec.auckland.ac.nz
vos: could not list the server addresses
vl: Index out of range