[OpenAFS-devel] Got one of those interesting "bunch of servers won't talk to a client" situations right at the moment...
Neulinger, Nathan
nneul@umr.edu
Thu, 17 Mar 2005 14:56:56 -0600
Client and server are both running builds from within the past few
months.=20
-bash-2.05b# /usr/afsws/bin/fs checks
These servers unavailable due to network or server problems:
afs-fs1.cc.umr.edu afs-fs17.cc.umr.edu afs-fs7.cc.umr.edu
afs-fs3.cc.umr.edu afs-fs15.cc.umr.edu afs-fs14.cc.umr.edu
afs-fs6.cc.umr.edu afs-fs10.cc.umr.edu afs-fs2.cc.umr.edu
afs-fs16.cc.umr.edu afs-fs13.cc.umr.edu afs-fs9.cc.umr.edu
afs-fs8.cc.umr.edu
Machine is otherwise fine, but a whole pile of servers won't talk to it.
In a network trace, a few of the servers are sending back rx abort
packets.=20
No connections to this client show up in rxdebug against the file
server.
I'm open to non-destructive testing on the servers at the moment, and
anything you think I should try on the client before I boot it, which
experience tells me will immediately correct the symptom.
Ideas? This is a long standing problem we have with afs, that comes up
frequently on some systems, but never reproducible. Since the machine
it's on is seeing it quite noticably, and I can afford to have it out of
service for a bit, I'm open to testing anything on it that y'all think
would help.=20
-- Nathan
14:41:58.754831 sysinst.cc.umr.edu.afs3-callback >
afs-fs13.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754858 sysinst.cc.umr.edu.afs3-callback >
afs-fs14.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754876 sysinst.cc.umr.edu.afs3-callback >
afs-fs15.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754889 sysinst.cc.umr.edu.afs3-callback >
afs-fs16.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754906 sysinst.cc.umr.edu.afs3-callback >
afs-fs1.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754920 sysinst.cc.umr.edu.afs3-callback >
afs-fs17.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754931 sysinst.cc.umr.edu.afs3-callback >
afs-fs2.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754943 sysinst.cc.umr.edu.afs3-callback >
afs-fs6.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754957 sysinst.cc.umr.edu.afs3-callback >
afs-fs7.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754970 sysinst.cc.umr.edu.afs3-callback >
afs-fs3.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.754982 sysinst.cc.umr.edu.afs3-callback >
afs-fs9.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.755034 afs-fs14.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755046 afs-fs15.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755069 sysinst.cc.umr.edu.afs3-callback >
afs-fs10.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.755083 sysinst.cc.umr.edu.afs3-callback >
afs-fs8.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:41:58.755102 afs-fs13.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755120 afs-fs6.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755138 afs-fs16.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755334 afs-fs9.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755382 afs-fs8.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755410 afs-fs3.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755486 afs-fs17.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755491 afs-fs10.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755604 afs-fs2.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx data fs reply get-time 2005/03/17
14:41:58 (36)
14:41:58.755707 sysinst.cc.umr.edu.afs3-callback >
afs-fs14.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755723 sysinst.cc.umr.edu.afs3-callback >
afs-fs15.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755734 sysinst.cc.umr.edu.afs3-callback >
afs-fs13.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755745 sysinst.cc.umr.edu.afs3-callback >
afs-fs6.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755757 sysinst.cc.umr.edu.afs3-callback >
afs-fs16.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755768 sysinst.cc.umr.edu.afs3-callback >
afs-fs9.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755779 sysinst.cc.umr.edu.afs3-callback >
afs-fs8.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755791 sysinst.cc.umr.edu.afs3-callback >
afs-fs3.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755802 sysinst.cc.umr.edu.afs3-callback >
afs-fs17.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755812 sysinst.cc.umr.edu.afs3-callback >
afs-fs10.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:58.755823 sysinst.cc.umr.edu.afs3-callback >
afs-fs2.cc.umr.edu.afs3-fileserver: rx ack (65)
14:41:59.758252 sysinst.cc.umr.edu.afs3-callback >
afs-fs1.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:42:00.778285 sysinst.cc.umr.edu.afs3-callback >
afs-fs1.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:42:00.778303 sysinst.cc.umr.edu.afs3-callback >
afs-fs7.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:42:01.771456 afs-fs1.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx abort (32)
14:42:01.774793 afs-fs7.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx abort (32)
14:42:02.308351 sysinst.cc.umr.edu.afs3-callback >
afs-fs1.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:42:02.818343 sysinst.cc.umr.edu.afs3-callback >
afs-fs7.cc.umr.edu.afs3-fileserver: rx ack (65)
14:42:03.328360 sysinst.cc.umr.edu.afs3-callback >
afs-fs7.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:42:04.858423 sysinst.cc.umr.edu.afs3-callback >
afs-fs1.cc.umr.edu.afs3-fileserver: rx data fs call get-time (32)
14:42:05.321829 afs-fs1.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx abort (32)
14:42:05.835009 afs-fs7.cc.umr.edu.afs3-fileserver >
sysinst.cc.umr.edu.afs3-callback: rx abort (32)
-- Nathan
------------------------------------------------------------
Nathan Neulinger EMail: nneul@umr.edu
University of Missouri - Rolla Phone: (573) 341-6679
UMR Information Technology Fax: (573) 341-4216