[OpenAFS] Re: byte-range lock errors

[Andrew Deason]

On Sat, 18 May 2013 02:38:19 +0200
Jaap Winius <jwinius@umrk.nl> wrote:

> I seem to remember a somwhat woozy Matt Benjamin delivering a
> presentation about a solution for this very issue in Pilsen back in
> 2010. Will his changes be included in a later version, or are they
> already in this one?

It will be in a later version. It will probably take a few years, since
the group dealing with wire protocol changes has some other things in
front of it. And dealing with that issue is not the simplest thing to
do, but nothing we do is :)

That doesn't stop you from trying an experimental version of it or
something, though. If you wanted to try it out, I'm sure Matt can help
you out. (I don't think I have any of that code, or if I do, I'm sure
it's very old.)

> > It doesn't affect you because you're running it on one machine,
> > presumably.
> 
> It may not affect me as a user, but as a sysadmin I am affected by
> anything abnormal in a log file on a system for which I am
> responsible... Like a splinter in my mind, driving me mad. ;-)

I just meant that in the context that you said you're seeing these
messages, but they don't seem to affect anything.

> > How often are you seeing these messages? There is some simple
> > rate-limiting so you're not supposed to see them for the same
> > process more often than once every 2 minutes. But maybe it's that
> > once-every-two-minutes that you're talking about :)
> 
> It's more often than that. Here's the frequency for a 10-minute period:
[...]
> That's exactly four times a minute on average. Also, in this case four  
> different pids were involved, all related to firefox-bin.

Well yeah, because there are 4 different pids involved. If the pids
rotate, the counter gets reset each time, so the simplistic rate
limiting we have doesn't really work.

> > I think running this should turn them off:
> >
> > fs messages -show console
> > ...
> 
> That works, but since I'd have to do this every time I reboot my  
> workstation, it's not exactly a very convenient solution.
[...]
> > Do you (or others) know what you'd want such an option to look like?
> > It could be yet another thing in /proc/sys/afs...
> 
> That would suite me just fine.

Okay, but that would also get cleared on reboot and would need to be set
again after the afs kernel module is loaded. So I'm not sure how much
better that is for you.

To just make all of the options clear, I can see a few different ways of
not spamming the logs with these:

 (1) Modify 'fs messages' to turn off messages with more granularity,
     and add an option to turn off the byte-range lock ones. (Or maybe
     have a message category for potentially-spammy optional messages
     like this.)
 (2) Add a sysctl option.
 (3) Add an afsd option.
 (4) Improve the automatic rate limiting so we remember more pids, or
     just globally limit the messages without looking at the pid.

(3) would allow you to specify this once and have it 'persist', assuming
that you have a good place to configure afsd options. But it's not
changeable at runtime, and when these messages go away we now have a
useless afsd option.

(2) would only let you turn this stuff off on Linux and Solaris; those
are the only platforms I remember where we have something sysctl-like to
change options like this.

I'm obviously not sure what (4) would look like. It's not practical to
remember every pid we've warned about, but we could easily remember the
last N pids with N around 16 or so?

Those are just my thoughts; anyone feel free to contribute their opinion
(hint hint). If (2) still sounds best for you, that's probably fine;
it's the easiest to implement, anyway :)

-- 
Andrew Deason
adeason@sinenomine.net