[OpenAFS] [1.2.7] Strange file server meltdown

[Russ Allbery]

Todd DeSantis <Todd_DeSantis@transarc.com> writes:

> In your other email, you mention that the host lock is being held, etc.
> During the "bos restart" the fileserver will shutdwon andthen the
> bosserver will restart it once it sees that it no longer is running.

Different site with the same problem, actually.  Thankfully, they gathered
more information than we managed to!

> When you got the rxdebug output from the fileserver, how many
> connections were there in total ?

At 3:55am, I got:

Free packets: 5, packet reclaims: 852, calls: 22851498, used FDs: 64
not waiting for packets.
1403 calls waiting for a thread
2 threads are idle

and 1,371 connections total.  At 4:08am, I got:

Free packets: 15, packet reclaims: 858, calls: 22851498, used FDs: 64
not waiting for packets.
2559 calls waiting for a thread
2 threads are idle

and 2,535 connections total.  So the number of connections *not* waiting
for a thread went down between the two intervals.  I did not use -allconn.

> If you used the -rxstats flag, what did the following line look like

>    1 server connections, 7 client connections, 7 peer structs, 4 call
> structs, 3 free call structs

> How many server connections
> 	 client connections

Unfortunately, I didn't use -rxstats (should have).  I'll do that next
time.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>