[OpenAFS-devel] locking fairness on Linux?

[John S. Bucy]

On Wed, May 04, 2005 at 04:19:40PM -0400, John S. Bucy wrote:

> Not only is it serializing but as far as I can tell, the first one
> runs exclusively until it finishes, then the second and so on.  

btw, if I add a sched_yield() to the loop in the test, it does what
you'd expect.

Also, it pipelines nicely if I run it in separate directories; the
throughput goes up linearly at least for the first few processes.
 
> So it sounds there's some kind of directory (vnode?) -level locking
> that has the characteristic that if you re-lock something in the same
> timeslice that you last released it, then you always win.

Reading a little further, the struct afs_lock has this characteristic.
Obtaining it consists of essentially while(still locked) sleep().  In
my example, the only way one of the starved threads will get to run is
if the running thread is preempted just after unlocking and one of
the starving ones wakes up from sleep and is scheduled in the next
timeslice.  I'd imagine that if I ran this for a long time that I
might see that happen once but haven't so far.

IMHO, the Right thing here is for struct afs_lock to be a wrapper for
a native primitive that has the desired properties.  It seems easy
enough to do but I have no idea if it would Just Work or if it would
break in strange and subtle ways because of assumptions elsewhere.
Looking around, there only seem to be ~10 global instances and its
used in struct afs_vcache (only in AIX32_ENV apparently) and in
struct memCacheEntry.




john