[OpenAFS-win32-devel] Re: file locking

Wed, 20 Jul 2005 16:46:35 -0400

This is a cryptographically signed message in MIME format.

--------------ms040504000305070001010503
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

A couple of points I would like to throw out.

In the long term I would like to see the afsd_service.exe support both
the IFS and SMB/CIFS interfaces at the same time.   Therefore, I would
like to see locking implemented within the cache manager.  Given the
current demand I want to see locking available for the 1.4.1 release
which will still not support the IFS.   The IFS will be targeted either
at 1.6 or 2.0 or whatever comes next.

I am in no way committed to the existing cm_Lock / cm_Unlock
implementations.   If there is no good reason to have timed locks in the
cache manager then we should re-write that code.   Although, it would be
worth spending some time to understand why the timed locks were added to
that code in the first place.   Perhaps it is due to some SMB/CIFS
behavior that really should be implemented at SMB layer and not in the
CM layer.

I have asked Asanka Herath to develop a test application to exercise the
various locking combinations.   We can use this application to observe
the current behavior of the AFS Client Service via the SMB/CIFS
interface and test the expected behavior of new code.

Jeffrey Altman

Matt Benjamin wrote:

> Eric,
> 
>>jeff suggests:
> 
> 
>>> How about we take this discussion to the openafs-win32-devel list?
>>  
>>
> Happy to do this on-list, if you both agree.  Forgive me for putting
> some snippets in from different mails.
> 
>>> Eric,
>>>
>>> Excellent.  This is the fun part of it, I assume for IFS only?  If I
>>> understand correctly, that effectively puts IFS client and traditional
>>> client on a par wrt locking?
> 
>> yes, ifs only.  the traditional client may not have functional byte-range
>> locks, but, being of the advisory type, they are not extremely useful.
> 
> I think I need some help parsing that...
> 
> I had written some trivial code to see what Win32 LockFile did in various scenarios.  Didn't attempt ones with conflicting sessions.  Running out of AFS (recent client) on a single machine, it certainly appears you can lock ranges as a reader of the Win32 API docs would expect, but of course these don't correspond to anything in AFS.  I have assumed that these are IFS locks managed by windows file sharing, and that enforcement would also be what you would expect...  (Please correct here, if I have this all wrong.)
> 
> In the Linux client, I used the Trond-enhanced posix_lock_file_wait to
> do more or less what you do for IFS in your patch.  It wasn't hard to do
> there either.  A lock request fails if, after getting the posix lock,
> the client doesn't already have, and cannot get, a corresponding
> whole-file lock in AFS.  Of course, this does not require an upcall in
> that context, etc.
> 
> If I follow the "alternatives" discussion correctly, the issues raised
> _here_
> 
> if the cache manager were to start enforcing mandatory locks (just as a
>>scenario), there would still be problems.  mapped file (or, paging i/o)
>>requests should never be rejected because of lock conflicts.  it is
>>possible to bypass the lock on a region by using mem-mapped i/o.  this is
>>an expected and necessary behaviour; not completing such a request
>>successfully is bad.  so, if all instances have locks on them, and a
>>paging request is received, the cache manager would reject the upcall
>>read/write request.  of course, in this case, the read and write calls
>>could also be modified...
> 
> could be addressed if the code responsible for doing the upcall knew
> what the RsRtl routines know about IO operations?  I'm missing how doing
> our own range reader-writer locks would address that, but, you don't
> advocate that.
> 
> In particular, it seems to me if the FsRtl routines exist, provide a
> locking implementation for free that is likely the default implmentation
> for everyone else, too ("**Lock* 'Em Up - Byte Range Locking")*, then it
> seems the more difficult argument is for _not_ using them. 
> 
> Now, when you say the protocols are incompatible, it appears that the only issues unaccounted for are related to the time of validity in the cm locking interface?  
> 
>>i found the locking protocols to be incompatible.
>>
>>the cache manager allows locks to be placed and removed using the four
>>parameters (instance, lock_type, offset, length).  additionally, locks
>>require a time-period of validity.  these are advisory locks.
>>
> 
> The time-period of validity throws me--seems to be a Windows CM-ism?  I didn't run into this in afs_lockctl, nor do I see it in afs.h, afsint.xg, and so forth.
> 
>>the ifs layer only receives read-write locks on regions:  the caller is
>>guaranteed exclusive access to the locked region.  the lock request
>>requires the four parameters (instance, offset, length, key).  however,
>>unlock requests have one of three forms:  (instance, offset, length, key),
>>(instance, key), (instance).  it is possible to remove all file locks on a
>>specified handle with a single call.  if granted, these are mandatory
>>locks.
> 
> This is not too dissimilar to what you see on the various Unixes, right?
> 
>>a simpler solution is to use the kernel's FsRtl functions to manage locks,
>>and to send the whole-file-lock request up to a daemon which spins on
>>making lock requests as often as necessary (because of expiry).  this is
>>similar to smb_WaitingLocksDaemon.  even if the lock is dropped by the cm,
>>for one of any number of reasons, the mandatory lock is still enforced
>>correctly in the kernel.  this is what i would like to do.
> 
> Aside from the the lock expiry "spinning" behavior, the solution you propose does sound to me like the analog of the behavior I implemented, Nathan Neulinger described, etc.
> 
> Would we need a lock-re-upping daemon if the Windows CM worked like the Unix cms in this regard? 
> 
> Matt
> 
> my work-up of locking:
> 
> 
>>
>>>
>>>i could modify the cache manager's locking functions to conform to the ifs
>>>model for lock requests.  the ifs redirector would pass the lock requests
>>>to the cache manager.  for a read request, the cm would have to be
>>>consulted to make sure there is no lock conflict (granted locks are
>>>mandatory), and then the i/o request would be processed.  if the lock is
>>>destroyed by the cm, then improper i/o could take place.  this is a
>>>serious problem (ms office, for instance, may crash or fail), as i'm sure
>>>you know.
>>>
>>>if the cache manager were to start enforcing mandatory locks (just as a
>>>scenario), there would still be problems.  mapped file (or, paging i/o)
>>>requests should never be rejected because of lock conflicts.  it is
>>>possible to bypass the lock on a region by using mem-mapped i/o.  this is
>>>an expected and necessary behaviour; not completing such a request
>>>successfully is bad.  so, if all instances have locks on them, and a
>>>paging request is received, the cache manager would reject the upcall
>>>read/write request.  of course, in this case, the read and write calls
>>>could also be modified...
>>>
>>>another possibility is to not change the cache manager's code, and to keep
>>>a list of locks in the kernel.  then, an appropriate number of unlock
>>>requests could be sent via upcalls.  this is redundant at best.
>>>
>>>a simpler solution is to use the kernel's FsRtl functions to manage locks,
>>>and to send the whole-file-lock request up to a daemon which spins on
>>>making lock requests as often as necessary (because of expiry).  this is
>>>similar to smb_WaitingLocksDaemon.  even if the lock is dropped by the cm,
>>>for one of any number of reasons, the mandatory lock is still enforced
>>>correctly in the kernel.  this is what i would like to do.
>>
> 
> 
> 
> 
> Eric Williams wrote:
> 
>>On Wed, 20 Jul 2005, Matt Benjamin wrote:
>>
>>  
>>
>>>Eric,
>>>
>>>Excellent.  This is the fun part of it, I assume for IFS only?  If I
>>>understand correctly, that effectively puts IFS client and traditional
>>>client on a par wrt locking?
>>>    
>>>
>>
>>yes, ifs only.  the traditional client may not have functional byte-range
>>locks, but, being of the advisory type, they are not extremely useful.
>>
>>  
>>
> 
> -- 
> 
> Matt Benjamin
> 
> The Linux Box
> 206 South Fifth Ave. Suite 150
> Ann Arbor, MI  48104
> 
> http://linuxbox.com
> 
> tel. 734-761-4689
> fax. 734-769-8938
> cel. 734-216-5309
> 
> 

--------------ms040504000305070001010503
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJPzCC
AvowggJjoAMCAQICAw7NrDANBgkqhkiG9w0BAQQFADBiMQswCQYDVQQGEwJaQTElMCMGA1UE
ChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv
bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EwHhcNMDUwNTI3MTc0MjQzWhcNMDYwNTI3MTc0MjQz
WjBrMQ8wDQYDVQQEEwZBbHRtYW4xFTATBgNVBCoTDEplZmZyZXkgRXJpYzEcMBoGA1UEAxMT
SmVmZnJleSBFcmljIEFsdG1hbjEjMCEGCSqGSIb3DQEJARYUamFsdG1hbkBjb2x1bWJpYS5l
ZHUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC+LutDu/YyHreNfoYd+ZtOjXsL
h67F2cmcVuBPBz+ZGDA+WpVEHrqXaZZO8acXBR5uAVfiwA1acE/kvD/CN5kAqx1VJuQ8Pvyk
iGHhUYTd27ZTliBIrptC7C/381gVwkS+a8jQFPJPO+OktZDzAYplGRY/MQCV8dIsvXUjucox
7TwTTdoLAJYRvHtfEcaCc6mO4ph6NeXQw8Grlx3IRAlTrkE5fBGyjH6R4fqnFTXRQAh1/bG+
i8hQvE6mud3mXdL2t7NP1Qxd9wW0/F/pnWY12IFP/luc3zEzIPvAe+nJluLuSEj0LZgP16mF
xBj1p+u9HPWcHRVX6q7+MQ0RWOv1AgMBAAGjMTAvMB8GA1UdEQQYMBaBFGphbHRtYW5AY29s
dW1iaWEuZWR1MAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEEBQADgYEAUDUuzxiq8bbI8vq2
swRK513RphZp+fepyKU5mwBI6aF4GcmqITQILtfTG2SXnjSeY99d+bjOdK1DJFvVh9aOy8mh
2NbEnqMnJIZtg5+eEU64DIV5bQdDRpi99H9vA0sRATIquut+3YHba+zArj0VkVof2VI+ToBu
sHdtSrZYo0gwggL6MIICY6ADAgECAgMOzawwDQYJKoZIhvcNAQEEBQAwYjELMAkGA1UEBhMC
WkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1Ro
YXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA1MDUyNzE3NDI0M1oXDTA2
MDUyNzE3NDI0M1owazEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVyaWMx
HDAaBgNVBAMTE0plZmZyZXkgRXJpYyBBbHRtYW4xIzAhBgkqhkiG9w0BCQEWFGphbHRtYW5A
Y29sdW1iaWEuZWR1MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvi7rQ7v2Mh63
jX6GHfmbTo17C4euxdnJnFbgTwc/mRgwPlqVRB66l2mWTvGnFwUebgFX4sANWnBP5Lw/wjeZ
AKsdVSbkPD78pIhh4VGE3du2U5YgSK6bQuwv9/NYFcJEvmvI0BTyTzvjpLWQ8wGKZRkWPzEA
lfHSLL11I7nKMe08E03aCwCWEbx7XxHGgnOpjuKYejXl0MPBq5cdyEQJU65BOXwRsox+keH6
pxU10UAIdf2xvovIULxOprnd5l3S9rezT9UMXfcFtPxf6Z1mNdiBT/5bnN8xMyD7wHvpyZbi
7khI9C2YD9ephcQY9afrvRz1nB0VV+qu/jENEVjr9QIDAQABozEwLzAfBgNVHREEGDAWgRRq
YWx0bWFuQGNvbHVtYmlhLmVkdTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBAUAA4GBAFA1
Ls8YqvG2yPL6trMESudd0aYWafn3qcilOZsASOmheBnJqiE0CC7X0xtkl540nmPfXfm4znSt
QyRb1YfWjsvJodjWxJ6jJySGbYOfnhFOuAyFeW0HQ0aYvfR/bwNLEQEyKrrrft2B22vswK49
FZFaH9lSPk6AbrB3bUq2WKNIMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TEL
MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du
MRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBT
ZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB
MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcx
NzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0
ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl
bWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnK
mVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/
cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8
YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4
oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5j
cmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy
LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4
Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowg
T2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCAzswggM3AgEB
MGkwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0
ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAgMOzaww
CQYFKw4DAhoFAKCCAacwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUx
DxcNMDUwNzIwMjA0NjM1WjAjBgkqhkiG9w0BCQQxFgQU+UaO+pCXdXA+8UJOZ5I/7ZONreQw
UgYJKoZIhvcNAQkPMUUwQzAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcN
AwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgweAYJKwYBBAGCNxAEMWswaTBiMQswCQYD
VQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE
AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECAw7NrDB6BgsqhkiG9w0B
CRACCzFroGkwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQ
dHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENB
AgMOzawwDQYJKoZIhvcNAQEBBQAEggEAEeOXlmdAKAKvxZi0T17SrjIXJO6gb+mCmHzjmqjn
plKkrpdFuHv/aQJiWBbmwqa0sVAcotj6yMsyaQJjWUM88zorFWNp//TzxjSB2L783OhJJtjk
xJrkBcNUwroY+lWtEfqvuQ9nYfk6EI+VIA0A2Rx9BCp2IarElNbvuLtIlL5CoXvrtVVh13h+
m6vR6suBNFhzM1HK29IyMRrcAn7nkGynSa0YACS1gJoJG6/MT/WGTh4ObeYAyRNzlov99P69
hdBAN+vUK0qD1g5YVzIY2MEOU2Qf0sjyIb9jSca2SXZoiQgj3/tQfGLNTPOMMr17wXTf3wB3
tX+LvuLzfeH/MQAAAAAAAA==
--------------ms040504000305070001010503--