[OpenAFS-win32-devel] Re: file locking
Matt Benjamin
matt@linuxbox.com
Wed, 20 Jul 2005 17:09:40 -0400
1. Neat. Locking in the CM corresponding with what the Unix CMs do, ie,
whole-file?
Jeffrey Altman wrote:
>A couple of points I would like to throw out.
>
>In the long term I would like to see the afsd_service.exe support both
>the IFS and SMB/CIFS interfaces at the same time. Therefore, I would
>like to see locking implemented within the cache manager. Given the
>current demand I want to see locking available for the 1.4.1 release
>which will still not support the IFS. The IFS will be targeted either
>at 1.6 or 2.0 or whatever comes next.
>
>
2. Right.
>I am in no way committed to the existing cm_Lock / cm_Unlock
>implementations. If there is no good reason to have timed locks in the
>cache manager then we should re-write that code. Although, it would be
>worth spending some time to understand why the timed locks were added to
>that code in the first place. Perhaps it is due to some SMB/CIFS
>behavior that really should be implemented at SMB layer and not in the
>CM layer.
>
>I have asked Asanka Herath to develop a test application to exercise the
>various locking combinations. We can use this application to observe
>the current behavior of the AFS Client Service via the SMB/CIFS
>interface and test the expected behavior of new code.
>
>
3. Great.
>Jeffrey Altman
>
>
>Matt Benjamin wrote:
>
>
>
>>Eric,
>>
>>
>>
>>>jeff suggests:
>>>
>>>
>>
>>
>>>>How about we take this discussion to the openafs-win32-devel list?
>>>>
>>>>
>>>
>>>
>>>
>>>
>>Happy to do this on-list, if you both agree. Forgive me for putting
>>some snippets in from different mails.
>>
>>
>>
>>>>Eric,
>>>>
>>>>Excellent. This is the fun part of it, I assume for IFS only? If I
>>>>understand correctly, that effectively puts IFS client and traditional
>>>>client on a par wrt locking?
>>>>
>>>>
>>>yes, ifs only. the traditional client may not have functional byte-range
>>>locks, but, being of the advisory type, they are not extremely useful.
>>>
>>>
>>I think I need some help parsing that...
>>
>>I had written some trivial code to see what Win32 LockFile did in various scenarios. Didn't attempt ones with conflicting sessions. Running out of AFS (recent client) on a single machine, it certainly appears you can lock ranges as a reader of the Win32 API docs would expect, but of course these don't correspond to anything in AFS. I have assumed that these are IFS locks managed by windows file sharing, and that enforcement would also be what you would expect... (Please correct here, if I have this all wrong.)
>>
>>In the Linux client, I used the Trond-enhanced posix_lock_file_wait to
>>do more or less what you do for IFS in your patch. It wasn't hard to do
>>there either. A lock request fails if, after getting the posix lock,
>>the client doesn't already have, and cannot get, a corresponding
>>whole-file lock in AFS. Of course, this does not require an upcall in
>>that context, etc.
>>
>>If I follow the "alternatives" discussion correctly, the issues raised
>>_here_
>>
>>if the cache manager were to start enforcing mandatory locks (just as a
>>
>>
>>>scenario), there would still be problems. mapped file (or, paging i/o)
>>>requests should never be rejected because of lock conflicts. it is
>>>possible to bypass the lock on a region by using mem-mapped i/o. this is
>>>an expected and necessary behaviour; not completing such a request
>>>successfully is bad. so, if all instances have locks on them, and a
>>>paging request is received, the cache manager would reject the upcall
>>>read/write request. of course, in this case, the read and write calls
>>>could also be modified...
>>>
>>>
>>could be addressed if the code responsible for doing the upcall knew
>>what the RsRtl routines know about IO operations? I'm missing how doing
>>our own range reader-writer locks would address that, but, you don't
>>advocate that.
>>
>>In particular, it seems to me if the FsRtl routines exist, provide a
>>locking implementation for free that is likely the default implmentation
>>for everyone else, too ("**Lock* 'Em Up - Byte Range Locking")*, then it
>>seems the more difficult argument is for _not_ using them.
>>
>>Now, when you say the protocols are incompatible, it appears that the only issues unaccounted for are related to the time of validity in the cm locking interface?
>>
>>
>>
>>>i found the locking protocols to be incompatible.
>>>
>>>the cache manager allows locks to be placed and removed using the four
>>>parameters (instance, lock_type, offset, length). additionally, locks
>>>require a time-period of validity. these are advisory locks.
>>>
>>>
>>>
>>The time-period of validity throws me--seems to be a Windows CM-ism? I didn't run into this in afs_lockctl, nor do I see it in afs.h, afsint.xg, and so forth.
>>
>>
>>
>>>the ifs layer only receives read-write locks on regions: the caller is
>>>guaranteed exclusive access to the locked region. the lock request
>>>requires the four parameters (instance, offset, length, key). however,
>>>unlock requests have one of three forms: (instance, offset, length, key),
>>>(instance, key), (instance). it is possible to remove all file locks on a
>>>specified handle with a single call. if granted, these are mandatory
>>>locks.
>>>
>>>
>>This is not too dissimilar to what you see on the various Unixes, right?
>>
>>
>>
>>>a simpler solution is to use the kernel's FsRtl functions to manage locks,
>>>and to send the whole-file-lock request up to a daemon which spins on
>>>making lock requests as often as necessary (because of expiry). this is
>>>similar to smb_WaitingLocksDaemon. even if the lock is dropped by the cm,
>>>for one of any number of reasons, the mandatory lock is still enforced
>>>correctly in the kernel. this is what i would like to do.
>>>
>>>
>>Aside from the the lock expiry "spinning" behavior, the solution you propose does sound to me like the analog of the behavior I implemented, Nathan Neulinger described, etc.
>>
>>Would we need a lock-re-upping daemon if the Windows CM worked like the Unix cms in this regard?
>>
>>Matt
>>
>>my work-up of locking:
>>
>>
>>
>>
>>>>i could modify the cache manager's locking functions to conform to the ifs
>>>>model for lock requests. the ifs redirector would pass the lock requests
>>>>to the cache manager. for a read request, the cm would have to be
>>>>consulted to make sure there is no lock conflict (granted locks are
>>>>mandatory), and then the i/o request would be processed. if the lock is
>>>>destroyed by the cm, then improper i/o could take place. this is a
>>>>serious problem (ms office, for instance, may crash or fail), as i'm sure
>>>>you know.
>>>>
>>>>if the cache manager were to start enforcing mandatory locks (just as a
>>>>scenario), there would still be problems. mapped file (or, paging i/o)
>>>>requests should never be rejected because of lock conflicts. it is
>>>>possible to bypass the lock on a region by using mem-mapped i/o. this is
>>>>an expected and necessary behaviour; not completing such a request
>>>>successfully is bad. so, if all instances have locks on them, and a
>>>>paging request is received, the cache manager would reject the upcall
>>>>read/write request. of course, in this case, the read and write calls
>>>>could also be modified...
>>>>
>>>>another possibility is to not change the cache manager's code, and to keep
>>>>a list of locks in the kernel. then, an appropriate number of unlock
>>>>requests could be sent via upcalls. this is redundant at best.
>>>>
>>>>a simpler solution is to use the kernel's FsRtl functions to manage locks,
>>>>and to send the whole-file-lock request up to a daemon which spins on
>>>>making lock requests as often as necessary (because of expiry). this is
>>>>similar to smb_WaitingLocksDaemon. even if the lock is dropped by the cm,
>>>>for one of any number of reasons, the mandatory lock is still enforced
>>>>correctly in the kernel. this is what i would like to do.
>>>>
>>>>
>>
>>
>>Eric Williams wrote:
>>
>>
>>
>>>On Wed, 20 Jul 2005, Matt Benjamin wrote:
>>>
>>>
>>>
>>>
>>>
>>>>Eric,
>>>>
>>>>Excellent. This is the fun part of it, I assume for IFS only? If I
>>>>understand correctly, that effectively puts IFS client and traditional
>>>>client on a par wrt locking?
>>>>
>>>>
>>>>
>>>>
>>>yes, ifs only. the traditional client may not have functional byte-range
>>>locks, but, being of the advisory type, they are not extremely useful.
>>>
>>>
>>>
>>>
>>>
>>--
>>
>>Matt Benjamin
>>
>>The Linux Box
>>206 South Fifth Ave. Suite 150
>>Ann Arbor, MI 48104
>>
>>http://linuxbox.com
>>
>>tel. 734-761-4689
>>fax. 734-769-8938
>>cel. 734-216-5309
>>
>>
>>
>>
--
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI 48104
http://linuxbox.com
tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309