[OpenAFS] Re: volume 536871264 is busy or server is down, recheck

[Andrew Deason]

On Thu, 1 Apr 2010 10:33:07 +0800
"许冬" <xud@ihep.ac.cn> wrote:

> Hi,
> 
> I want to know how many parallel  read requests for one volume at the
> same time? or how many parallel read requests for one replication
> volume at the same time?

There is no real useful answer for that. There is a limit, however, to
the number of outstanding requests on a server (described below).

If 100 requests in parallel is your only load on that server, though,
the server should be able to handle that just fine.

> In our afs system, there are about one hundred people to read a
> volume parallelly, and each people will issus about 500 read
> requests.

Are these on different AFS clients, or all on the same one?

> I found the afs client's /var/log/message file often appear  some
> error information, such as "volume 536871264 is busy or server is
> down, recheck ".
> 
> so, I want to know its reason.

Do you mean the message "Waiting for busy volume 536871264"? There are a
number of reasons for that; only one I can think of is caused by load.
Is there anything in FileLog or VolserLog around the time that you see
these messages?

I believe it is possible to get this message if the server is overloaded
(possibly for a long enough period of time). If the number of
outstanding calls waiting for a servicing thread ("calls waiting")
exceeds a certain threshold, the client will get an error that can
result in that message. This threshold is by default the -rxpck setting
multiplied by 3/2. The default -rxpck setting is 150 (200 for -L), so
the default threshold is around 225 (or 300 for -L).

You can see how many calls are waiting for a thread with 'rxdebug
<server>'. For example:

   % rxdebug 192.168.1.100
   Trying 192.168.1.100 (port 7000):
   Free packets: 265, packet reclaims: 0, calls: 0, used FDs: 34
   not waiting for packets.
   0 calls waiting for a thread
   11 threads are idle
   Done.

shows that there are "0 calls waiting for a thread".

-- 
Andrew Deason
adeason@sinenomine.net