[AFS3-std] File Server/Cache Manager Extended Status Information

[Jeffrey Altman]

This is a cryptographically signed message in MIME format.

--------------ms030402080300070206060509
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Matt Benjamin wrote:
> At the risk of getting really volatile discussion going, I should state
> that Jeff Altman's comments on IRC, raise topics not yet addressed here:
>
> 1.  Jeff's initial reaction in public IRC was that notifications cannot
> be asynchronous--we must require clients to poll for them.  Jeff A.
> appears to have changed his position, but asserted the previous position
> in very strong terms.  I think using the call back channel for
> notifications is highly preferable to another polling interface.  Has
> Jeff A.'s position in fact changed?  Is this in any way controversial?
Change notifications (aka callback breaks) are synchronous in AFS.
That is not a question available for debate.  The file server architecture
triggers callbacks synchronously with the operation that is producing
the change.  Callbacks can be delayed but must be delivered successfully
to the client before any other request can be processed from a client
with an outstanding callback.

This behavior is a core component of the AFS coherency model.

Any changes that are going to be integrated in the near term must
adhere to this model. 

While I will not rule out asynchronous notifications at some point
in the future after careful consideration of a cohesive architectural
design, I do not see such changes being accepted into production
branches anytime soon.

In addition, my comments to you were specifically regarding your
desire for the following locking exchange:

   1. client A requests blocking byte range write lock on fid 1.2.3
   2. server replies that the lock is not available but will send an
      asynchronous notification when it is
   3. another client drops the held lock
   4. server allocates lock to client A and sends asynchronous
      notification that the lock was allocated

As opposed to the existing model in which

   1. client A requests block byte range write lock on fid 1.2.3
   2. server replies EWOULDBLOCK
   3. another client drops the held lock
   4. server sends client A a callback indicating that the lock state on
      fid 1.2.3 has changed
   5. client decides whether or not to request the lock again.  if so,
      go to 1

The problem I see with your approach is that it imposes too much state 
on the file server.  The file server needs to be able to deal with the 
situations in which either client A is no longer reachable and is 
therefore holding the lock when others could be using it; and deal with 
the case where client A sends a lock request cancellation message racing 
with the lock allocation.

The existing model works today but suffers from the failure to know that 
the reason a callback was received was because the lock was dropped or 
because a Store occurred etc.
>
> 2. Jeff stated in IRC (in response to the discussion I initially
> started) that "when requesting FetchStatus or BulkStatus the client must
> be able to provide a callback filter for the events that it is
> interested in."  Assuming a polling interface, and also mandating
> fine-grained control over notifications on a per-object basis.  Is this
> mandate now retracted?  Moved to TellMeAboutYourself (not per object, 
> obviously, in that case)?
First improve the functionality of the existing implementation without 
breaking the existing model or adding additional state.  This includes 
sending more informative callback messages which permit the cache 
manager to make more intelligent decisions and minimize the need for 
reading unnecessary data from the file server.

Then we can work on a subsequent phase which permits the registration of 
multi-event callbacks requests.  These requests must permit the cache 
manager to specify which events are of interest.
>
> 3.  The term "call back" is too overloaded in AFS.  It confuses the
> meaning of a call back operation and the promise to make it.  Standard
> idiom, too, is about "breaking" call backs, but clearly, file change
> notifications are
>
> a. only logical if we have made the promise to call the client back, and
> b. they don't break the server's promise to call back the client again
jhutz hit this nail squarely on the head
>
> Jeffrey Hutzelman wrote:
> | --On Sunday, April 06, 2008 05:39:09 PM -0400 Jeffrey Altman
> | <jaltman@secure-endpoints.com> wrote:
>
> Yes, noted.
>
> |
> | Agree.  Clients should not have to do anything to receive extended
> | notifications other than advertise support for them.
> |
> | I'm inclined to agree, mostly.  I particularly like the idea of actually
> | distributing directory changes to clients holding callbacks on a
> | directory, as that could significantly improve performance in situations
> | where there is heavy contention for a directory, by eliminating the need
> | for clients to keep refetching that directory (if the client chooses to
> | apply the update locally).
> |
> | I agree with some of Matt's concerns about different triggers requiring
> | different extended data, but I'd really like to see a solution that does
> | not require a separate RPC for each trigger.
>
> Thanks, that is helpful.  Jeff's list of pseudo-prototypes suggest a
> small group of notification types and calls.  Would that be unreasonable.
>
> My sense is, people see it as more conservative, and therefore good, to
> have just one call back proc.  After reading my document, Jeff A. said
> he wanted to commit his own change, with just a "CallBackNew" to replace
> the old call back with _the_ new one.
> Obviously, no one wants to introduce unecessary RPCs, but frankly, a few
> clean calls should be more efficient than 1 hairy one with a switch and
> opaque buffers, at the client.  Faster too, I suspect.  Would you
> support that (Jeff H)?
Using multiple callback messages would make it hard to coalesce delayed 
callbacks of different types.

>
> |>>
> |> Why is there a range of byte range changes?  A StoreData operation
> |> does not take a list of independent commits.  Each StoreData will
> |> result in a BreakCallBack(). > Proc AccessChangeCallBack
> |
> | That's a sequence of byte ranges, to go with the sequence of FID's.  The
> | idea is to let you send several updates at once, possibly on different
> | FID's.  I suspect this is mostly useful on delayed callbacks.
> |
>
> As discussed in previous mail, it seems that there's a natural
> compression in batching notifications to one cache manager, especially
> to one file, grouped as Tom says, closely in time.  I assumed we would
> wish to support this.
This certainly is not something that would be considered in the near 
future by me with the one exception of delayed callbacks where the 
reason we are coalescing the messages is because the client is known to 
be unreachable.

Jeffrey Altman


--------------ms030402080300070206060509
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJeTCC
AxcwggKAoAMCAQICEALr5BE3U6n+HWCoLbyhohMwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE
BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT
I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDUzMTA2MTM1N1oX
DTA4MDUzMDA2MTM1N1owczEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVy
aWMxHDAaBgNVBAMTE0plZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQCsoz/0+s4Cn65n/3bU3shXw4y5u1uEMEsBOiqNU0PfIKGYQe95b1FKNbNAkctSdQT6GF5c
bhSnJPmb2OOb1frx64dlDgskaG561xa8XPA1aP8Cc+33dgsSLIxGEh97lyUYHEfWBC03KMCF
PKhZfcrGAXoVCrFBadnLAokQbUTFahVg/qQx2IT3wSj1sCIfV5UDuXcEKHCvRtEZIsSzu184
9Cj6I4nY5bt+r94kyDHM94MHYBJi+6tWLFRy2gkIB3HEPmxAiQrKljNpH9bOffiBLIAgmJ6d
1ZXepBXyexQbwOYvftpVlMEFHHQmdiwH3tj69hE78XvM5X9J+SbjbuNpAgMBAAGjOTA3MCcG
A1UdEQQgMB6BHGphbHRtYW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADAN
BgkqhkiG9w0BAQUFAAOBgQB8FShDN2Ig034Y5eyadiFDEtOvsIJ3Z2xV9aTL4u8xMlz1gZR1
AZAvCv+ZMMRRKWCsrG5tItV8DFPSfWAGMpInmMarA4f76JRLQEUhkRUg8GpkJM5ryk5EDakk
0oiBQcQD8A+UHwrcmaj3UWxQ9zCjDgU+1mY9nEQxZZyp4eeUfzCCAxcwggKAoAMCAQICEALr
5BE3U6n+HWCoLbyhohMwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoT
HFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25h
bCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA3MDUzMTA2MTM1N1oXDTA4MDUzMDA2MTM1N1ow
czEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVyaWMxHDAaBgNVBAMTE0pl
ZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRtYW5Ac2VjdXJlLWVuZHBv
aW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCsoz/0+s4Cn65n/3bU
3shXw4y5u1uEMEsBOiqNU0PfIKGYQe95b1FKNbNAkctSdQT6GF5cbhSnJPmb2OOb1frx64dl
DgskaG561xa8XPA1aP8Cc+33dgsSLIxGEh97lyUYHEfWBC03KMCFPKhZfcrGAXoVCrFBadnL
AokQbUTFahVg/qQx2IT3wSj1sCIfV5UDuXcEKHCvRtEZIsSzu1849Cj6I4nY5bt+r94kyDHM
94MHYBJi+6tWLFRy2gkIB3HEPmxAiQrKljNpH9bOffiBLIAgmJ6d1ZXepBXyexQbwOYvftpV
lMEFHHQmdiwH3tj69hE78XvM5X9J+SbjbuNpAgMBAAGjOTA3MCcGA1UdEQQgMB6BHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUFAAOB
gQB8FShDN2Ig034Y5eyadiFDEtOvsIJ3Z2xV9aTL4u8xMlz1gZR1AZAvCv+ZMMRRKWCsrG5t
ItV8DFPSfWAGMpInmMarA4f76JRLQEUhkRUg8GpkJM5ryk5EDakk0oiBQcQD8A+UHwrcmaj3
UWxQ9zCjDgU+1mY9nEQxZZyp4eeUfzCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAw
gdExCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUg
VG93bjEaMBgGA1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRp
b24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFp
bCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0w
MzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU
aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg
RnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV
+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfAr
hVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l0cUqVIUPSAR/
p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMGA1UdHwQ8
MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVtYWls
Q0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh
YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/
TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amc
OY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggNkMIID
YAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5
KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ
AuvkETdTqf4dYKgtvKGiEzAJBgUrDgMCGgUAoIIBwzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcN
AQcBMBwGCSqGSIb3DQEJBTEPFw0wODA0MDcyMDE4NDlaMCMGCSqGSIb3DQEJBDEWBBRvWJKF
0G4jkly3VYb0vHTTeaQW/DBSBgkqhkiG9w0BCQ8xRTBDMAoGCCqGSIb3DQMHMA4GCCqGSIb3
DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBhQYJKwYB
BAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcg
KFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3Vpbmcg
Q0ECEALr5BE3U6n+HWCoLbyhohMwgYcGCyqGSIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJa
QTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhh
d3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEALr5BE3U6n+HWCoLbyhohMwDQYJ
KoZIhvcNAQEBBQAEggEAF5K/QqCKxi9fUBZlNUOJd/TUTKBb20T4cHZycpM6Xdt3gKd/bGhF
4tUgr5m2TprVGSPfPFCgvjhIXNGwc+5uE3yA3+yT3unPAGUBDbfF/aZhG+yNoDgG/lzS7Hi0
faVcGR/h8eOEJI2W6GZQDkeNpnvqc/hE2fbMg/VfFAISKiWrkZaBCld7bp06bfXCeH6/Nrt0
DKZxqJd70RYLZPgf1ziFfFygrSv3ABSc0xGgha+mCKZHLx8reXwV/apY/Nd8UJHbQMJN2oJw
4Q2OIQjQVrLHWuY8EwBNqX0X9Mdec8/kLJPNM3zSLXMb4OewMdRRU7qOO6ai/+bojyP9fzn5
sAAAAAAAAA==
--------------ms030402080300070206060509--