[OpenAFS-devel] Callback Extended Information Draft 4 (Cross Posting from AFS3-Standardization)
Matt Benjamin
matt@linuxbox.com
Tue, 13 May 2008 11:02:15 -0400
This is a multi-part message in MIME format.
--------------030208080809080806000608
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi Developers,
I'm cross-posting this from the afs3-standardization list.
This draft attempts to incorporate suggested changes from earlier rounds
of discussion. Discussants, please call out places where I failed.
Document structure is at the request of OpenAFS GK, apologies to anyone
who finds it verbose.
Thanks,
Matt
- --
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI 48104
http://linuxbox.com
tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFIKa13JiSUUSaRdSURCNoMAJ9vyFglBEnUiveQIBQRvSVP59LDWACeK6+S
VxZN1f5nXf8DVB3+FjvXZUU=
=T7o2
-----END PGP SIGNATURE-----
--------------030208080809080806000608
Content-Type: text/plain;
name="callback_extension_d4.txt"
Content-Disposition: inline;
filename="callback_extension_d4.txt"
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by aa.linuxbox.com id m4DF2RsR013443
AFS Callback Extensions
Matt Benjamin <matt@linuxbox.com>
05/12/2008
Status of this Memo
This document specifies a standards track protocol extension for=20
the OpenAFS community, and requests discussion and suggestions=20
for improvements.
Key Words
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL=20
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and=20
"OPTIONAL" in this document are to be interpreted as described in=20
Internet Engineering Task Force RFC 2119.
Abstract
AFS cache-control strategy is callback (invalidate) based. The=20
AFS callback design allows a client to know when an object it has=20
cached is no longer consistent, but the callback notification=20
message itself provides no specific information about the=20
triggering event. This is a protocol inefficiency, as in several=20
scenarios it results in unnecessary round-trips to file servers=20
to verify file status information, file access information, or to=20
fetch file data which has not changed. We propose an extension of=20
the callback mechanism to provide information about the event(s)=20
triggering a callback, in the payload of the callback=20
notification message itself. The proposed mechanism eliminates=20
most or all unnecessary round-trips imposed by the current=20
callback mechanism, and simultaneously allows AFS implementations=20
to (efficiently) provide correct semantics in several scenarios=20
involving multiple writers (ie, where AFS currently provides=20
incorrect semantics).
Table of Contents
Table of Contents
Status of this Memo
Key Words
Abstract
Table of Contents
1 Introduction
2 The AFS Callback Mechanism
2.1 Description
2.2 Analysis
3 Extended Callback Interface
3.1 Backward Compatibility
3.2 Interface Changes
3.2.1 Procedures
3.2.2 Constants
3.2.3 Data Types
AFSExtendedCallback
AFSCBFileStatus
AFSCBDirStatus
AFSCB_NotificationData
3.3 Semantic Changes
3.3.1 DataVersion Rule
3.4 Callback Invocations
3.4.1 AFSExtendedCallback
Flags
ExtraFlags
DataVersion
ExpirationTime
Data
3.4.2 Reasons for Cancellation
AFSCB_Cancel_Shutdown
AFSCB_Cancel_CallbackGC
AFSCB_Cancel_VolumeOffline
AFSCB_Cancel_VolumeMoved
AFSCB_Cancel_LostMyMind
AFSCB_Cancel_IHateYou
3.4.3 ExtendedCallback Procedure
3.4.4 AFSCB_Event_StoreData
3.4.5 AFSCB_Event_StoreACL
3.4.6 AFSCB_Event_StoreStatus
3.4.7 AFSCB_Event_CreateFile
3.4.8 AFSCB_Event_MakeDir
3.4.9 AFSCB_Data_Symlink
3.4.10 AFSCB_Event_Link
3.4.11 AFSCB_Event_RemoveFile
3.4.12 AFSCB_Event_RemoveDir
3.4.13 AFSCB_Event_Rename
3.4.14 AFSCB_Event_Deleted
3.4.15 AFSCB_Event_ReleaseLock
4 Appendix A: XDR Grammar
1 Introduction
The AFS protocol provides a comprehensive framework for scalable,=20
secure, wide-area file sharing over IP networks. The AFS system=20
has historically distinguished itself through its emphasis on=20
scalability, a key source of which is client-side caching[1, 4].=20
File data, file and directory metadata, and access control=20
information may all be cached. Cache consistency is maintained=20
through client registration and an associated asynchronous=20
notification mechanism known as the callback.=20
The current AFS consistency model (which is of larger scope than=20
the callback mechanism, eg, it includes AFS sync-on-close=20
semantics) has allowed AFS to scale to large numbers of clients=20
(tens of thousands today), and to perform well under the=20
workloads for which AFS was originally designed.
However, AFS does not perform efficiently under other conditions,=20
such as when more than one client is interested in a file which=20
is changing--even if the file has only one writer, and many=20
readers[footnote:
A scenario which competing protocols efficiently support.=20
]. In general, the AFS protocol arguably (still, considering=20
improvements made between AFS-2 and AFS-3) places too little=20
emphasis on efficient caching of mutable data. The current AFS=20
consistency model is insufficient to correctly support=20
single-file, multiple-writer scenarios, including those required=20
for POSIX semantics, and therefore is insufficient to support=20
many applications which may be run correctly on competing=20
distributed file systems (eg, CIFS, Novell Netware, or NFSv4).
The efficiency of the current AFS cache management algorithm=20
could be substantially improved if specific triggering event=20
information and current status were included in the payload of=20
the callback notifications sent to clients. In particular,=20
inclusion of the current DataVersion number and affected byte=20
ranges in response to StoreData operations would significantly=20
reduce the need for cache revalidation and reconstruction traffic=20
in response to callbacks--in many cases, altogether. These=20
changes would allow efficient support for single-writer updates=20
on a file with multiple readers. More importantly, they would=20
permit AFS to correctly and efficiently support multiple writers=20
updating disjoint ranges on a single file, a prerequisite for=20
supporting granular file locking (and applications which require=20
it) in future.
2 The AFS Callback Mechanism
2.1 Description
When an AFS-3 client contacts a file server to perform any of=20
several operations on a file, or explicitly to fetch its status,=20
the file server includes in its RPC response an AFSCallBack=20
structure, representing the server's promise to call back the=20
client ``if any modifications are made to the data in the file''.[footnot=
e:
A key paper on AFS-2 has ``before allowing a modification by any=20
other workstation''[1]. The wording of this statement appears=20
calculated to imply that the file server's promise to execute=20
callbacks synchronously with the triggering operations (eg,=20
StoreData) specifically constitutes part of the AFS cache=20
consistency guarantee. In our analysis, it does not, though it=20
does contribute strongly to the simplification of the file server=20
design and to reduction of file server workload.
]The AFSCallBack structure contains the callback expiration time,=20
and two integer values treated as invariants.
When any client executes an operation which would change a file=20
(eg, StoreData), and in a variety of other situations, the file=20
server invalidates the client's cached copy by executing a call=20
to the CallBack[footnote:
formerly BreakCallback
] procedure in the client's RPC interface. (The call includes in=20
its arguments an AFSCallBack structure for each file being=20
invalidated. However, the value of passed AFSCallBack is unused=20
[eg, afs/afs_callback.c:661-2]). Between the time of issue and=20
either expiry or receipt of a callback, the client may consider=20
any information it has cached on a file to be consistent with the=20
file server's on-disk copy. Conversely, on receipt of a callback,=20
the client must consider that it knows nothing about the file.=20
Thus the client must re-establish a relationship with the file at=20
the file server before executing any further operations on it.[footnote:
Since AFS supports the notion of a read-only volume all of whose=20
files may only be updated transactionally as a group, AFS permits=20
a file server to issue a single callback when any file in a=20
read-only volume is accessed. This is a significant performance=20
optimization for, by definition, cache management of immutable=20
data, and so is not discussed further here.
]
The AFS callback mechanism obviates the need for clients to send=20
frequent cache validation requests before performing operations=20
on their locally cached copies of objects, reducing network=20
traffic as well as file server workload[4]. The callback=20
innovation has been since taken up, with variations, by other=20
distributed file system protocols[2, 3, 5].
2.2 Analysis
The AFS callback mechanism reliably notifies clients when=20
information they may have cached becomes invalidated, but omits=20
to send information it trivially knows, ie, the triggering event,=20
that could certainly be used by the client to more efficiently=20
manage cache state.
For example, consider the case where 2 clients A and B are=20
interested in a file, each having read chunks 1-15 into cache.=20
Now another client C initiates a change in the file, writing a=20
new state to chunk 45. This event today triggers a callback, but=20
also invalidates 30 chunks correctly cached on A and B, which,=20
should they remain interested, they must refetch (up to 2=20
megabytes of data, in this case). This scenario may seem=20
relatively unlikely to occur (but of course, probably does occur=20
reasonably often in environments where mutable data is common),=20
but a related scenario involving directory entries (omitted for=20
brevity) is much more common. In these cases, an AFS callback=20
mechanism capable of sending triggering event information with=20
the callback would have facilitated a more efficient result, at=20
small marginal cost. In another set of scenarios where a client A=20
has changed data in a file invalidated by non-overlapping stores=20
by B, a revised mechanism would be capable of delivering a=20
correct result, whereas a correct result would be impossible with=20
the mechanism in AFS today.
The justification for sending minimal information with the=20
callback is presumably to minimize the execution cost of the=20
callback procedure. The increased cost of sending a limited but=20
informative callback notification to clients, relative to sending=20
an uninformative one, is small. Analysis of the OpenAFS file=20
server code reveals that the file server always has the=20
information that would logically be sent as extended callback=20
information in response to file operations (eg, file ranges=20
affected by StoreData operations, or changed entries for various=20
directory modification operations).=20
For these reasons, enhancement of the AFS callback interface to=20
supply triggering event information seems likely to improve both=20
correctness and performance of AFS implementations, and=20
experimental implementation and profiling appear justified.
3 Extended Callback Interface
3.1 Backward Compatibility
AFS clients will indicate their preference to receive extended=20
callback notifications through a new client capability flag:
const CLIENT_CAPABILITY_EXT_CALLBACK =3D 0x0002;
3.2 Interface Changes
3.2.1 Procedures
We propose a new procedure ExtendedCallback in the client's RPC=20
interface. ExtendedCallback follows the style of the traditional=20
AFS CallBack procedure in accepting parallel sequences of FIDs=20
and structures:
typedef AFSExtendedCallBack AFSExtendedCallBackSeq<AFSCBMAX>;
proc ExtendedCallBack(
IN AFSCBFids *Fids_Array,
AFSExtendedCallBackSeq *CallBacks_Array
) multi =3D 65539;
As detailed in section [sub:Constants], AFSExtendedCallBackSeq=20
resolves to a sequence of AFSExtendedCallBack structures whose=20
type is an XDR union, discriminated on the callback event type.
3.2.2 Constants<sub:Constants>
The following callback event types are defined:
const AFSCB_Event_Cancel =3D 1; /* extended */
const AFSCB_Event_StoreData =3D 2; /* data in file changed */
const AFSCB_Event_StoreACL =3D 3; /* ACL changed on vnode */
const AFSCB_Event_StoreStatus =3D 4; /* status stored on vnode */
const AFSCB_Event_CreateFile =3D 5; /* file created in directory=20
vnode */
const AFSCB_Event_MakeDir =3D 6; /* dir created in directory vnode=20
*/
const AFSCB_Event_Symlink =3D 7; /* symlink created in directory=20
vnode */
const AFSCB_Event_Link =3D 8; /* hard link created in directory=20
vnode */
const AFSCB_Event_RemoveFile =3D 9; /* file removd from directory=20
vnode */
const AFSCB_Event_RemoveDir =3D 10; /* dir removed from directory=20
vnode */
const AFSCB_Event_Rename =3D 11; /* object renamed (moved) */
const AFSCB_Event_Deleted =3D 12; /* object no longer exists, ex=20
object */
const AFSCB_Event_ReleaseLock =3D 13; /* traditional AFS lock=20
released */
A flag constant is provided to indicate callback cancellation=20
along with an extended notification message of any of the above=20
types:
const AFSCB_Flag_Cancel =3D 1; /* Callback promise is cancelled */
The following constants indicate reasons for cancellation, when=20
(Flags & AFSCB_Flag_Cancel)
const AFSCB_Cancel_Shutdown =3D 1;
const AFSCB_Cancel_CallbackGC =3D 2;
const AFSCB_Cancel_VolumeOffline =3D 3;
const AFSCB_Cancel_VolumeMoved =3D 4;
const AFSCB_Cancel_LostMyMind =3D 5;
const AFSCB_Cancel_IHateYou =3D 6;=20
The following constants indicate direction (from or to called=20
back FID) in the atomic AFSCB_Event_Rename notification:
const AFSCB_Rename_From =3D 1;
const AFSCB_Rename_To =3D 2;
3.2.3 Data Types
AFSExtendedCallback
The AFSExtendedCallBack data type contains members Flags,=20
DataVersion, ExpirationTime, and Data, where Flags and ExtraFlags=20
provide extra information, DataVersion is a (possibly=20
incremented) DataVersion, ExpirationTime is a (possibly extended)=20
callback expiration time, and Data is an object of the=20
discriminated union type AFSCB_NotificationData:
struct AFSExtendedCallBack {
afs_uint32 Flags;
afs_uint32 ExtraFlags;
afs_uint32 DataVersion;
afs_uint32 ExpirationTime;
AFSCB_NotificationData Data;
};
A positive value in Flags for the AFSCB_Flag_Cancel bit indicates=20
cancellation of the callback upon receipt of the message. In that=20
event, a non-zero value of ExtraFlags indicates the reason for=20
the cancellation.
AFSCBFileStatus
The AFSCBFileStatus structure is a reduced-footprint=20
AFSFetchStatus replacement intended to communicate changed vnode=20
information in response to StoreData operations:
AFSCBFileStatus {
afs_uint64 ClientModTime;
};
AFSCBDirStatus
The AFSCBDirStatus structure is a reduced-footprint=20
AFSFetchStatus replacement intended to communicate changed vnode=20
information in response to directory change operations:
AFSCBDirStatus {
afs_uint32 LinkCount;
afs_uint64 ClientModTime;
};
AFSCB_NotificationData
AFSCB_NotificationData is a union discriminated by callback event=20
type, ie, its value may be any of the constants defined in=20
section [sub:Constants].
union AFSCB_NotificationData switch (afs_uint32 Event_Type) {
case AFSCB_Event_StoreData:
AFSCB_Data_StoreData u_data;
case AFSCB_Event_StoreACL:
void;
case AFSCB_Event_StoreStatus:
AFSCB_Data_StoreStatus u_data;
case AFSCB_Event_CreateFile:
AFSCB_Data_CreateFile u_data;
case AFSCB_Event_MakeDir:
AFSCBName_Data_MakeDir u_data;
case AFSCB_Event_Symlink:
AFSCB_Data_Symlink u_data;
case AFSCB_Event_Link:
AFSCBName_Data_Link u_data;
case AFSCB_Event_RemoveFile:
AFSCB_Data_RemoveFile u_data;
case AFSCB_Event_RemoveDir:
AFSCB_Data_RemoveDir u_data;
case AFSCB_Event_Rename:
AFSCB_Data_Rename u_data;
case AFSCB_Event_Deleted:
void;
case AFSCB_Event_ReleaseLock:
AFSCB_Data_Lock u_data;
case AFSCB_Event_Cancel:
void;
};
The types for the variant member u_data are enumerated and=20
discussed in detail in section [sub:Callback-Invocations].=20
3.3 Semantic Changes
A file server MAY send traditional callback messages, with=20
traditional semantics, to any AFS client in response to any=20
event. A file server MAY send extended callback notifications to=20
any client which has announced the capability to use the extended=20
interface, with the following semantics:
=E2=80=A2 extended callback notification messages, in general, preserve=20
the file server's callback promise to send further=20
notifications for the called-back FID
=E2=80=A2 the file server may revoke the callback promise with any=20
extended callback notification message, by setting the=20
AFSCB_Flag_Cancel bit in the Flags member of the=20
AFSExtendedCallback structure
=E2=80=A2 the AFSCB_Event_Cancel message is similar to a traditional AFS=20
callback, breaking the callback promise, and requesting the=20
client not request further status on the FID
3.3.1 DataVersion Rule
The various extended callback notification messages include=20
information a client may use to selectively invalidate or=20
reconstruct its cache. In interpreting each message, the client=20
MUST observe the dataversion rule, which states:
If the client's cached DataVersion is DataVersion or=20
(DataVersion-1), the client may invalidate or reconstruct its=20
cache using the type-dependent information contained in the=20
message. In all other cases, the client MUST regard the message=20
as equivalent to a traditional AFS callback.
The semantics of specific callback events are enumerated in=20
section [sub:Callback-Invocations].
3.4 Callback Invocations<sub:Callback-Invocations>
The various extended callback notification types generally=20
respond to specific events at the file server, but present a view=20
of it relevant to a specific callback promise at one client. In=20
one case (ie, AFSCB_Event_Rename), the file server is sending=20
notification of an event which effects two FIDs, either or both=20
of which may be cached by the receiving client. A structure of=20
type AFSExtendedCallback is sent with each extended callback=20
notification message, as noted above. Unless otherwise noted, FID=20
is the FID of the object that is the subject of the callback.=20
3.4.1 AFSExtendedCallback
The members of the AFSExtendedCallback structures are to be=20
interpreted as follows:
Flags
If the 1-bit (AFSCB_Flag_Cancel) is set, the notification effects=20
a callback break. The client may make use of the information sent=20
with the message.
ExtraFlags
If (Flags & AFSCB_Flag_Cancel), a non-zero value for ExtraFlags=20
indicates the reason for cancellation.
DataVersion
The value of DataVersion at completion of the event of which the=20
client is being notified.
ExpirationTime
The new expiration time asserted for the server's callback=20
promise, not necessarily different from the existing expiration=20
cached by the client.
Data
The message-specific data for this notification.
3.4.2 Reasons for Cancellation
The following reasons for cancellation are defined:
AFSCB_Cancel_Shutdown
The server or service is shutting down.
AFSCB_Cancel_CallbackGC
Callback has been disposed during periodic garbage collection.
AFSCB_Cancel_VolumeOffline
The volume associated with FID is now offline.
AFSCB_Cancel_VolumeMoved
The volume associated with FID has moved.
AFSCB_Cancel_LostMyMind
The server may be having problems related to provisioning an=20
insufficient number of callback structures.
AFSCB_Cancel_IHateYou
Callback has been administratively revoked.
3.4.3 ExtendedCallback Procedure
Extended callbacks are delivered through a new ExtendedCallback=20
procedure.
proc ExtendedCallBack(
IN HostIdentifier *Server,
AFSCBFids *Fids_Array,
AFSExtendedCallBackSeq *CallBacks_Array
) multi =3D 65539;
ExtendedCallback is modelled on the traditional CallBack=20
procedure, but adds UUIDs uniquely identifying the file server=20
host.
3.4.4 AFSCB_Event_StoreData
The notification is sent in response to a successful StoreData=20
RPC on FID. A structure of type AFSCB_Data_StoreData is sent with=20
the message.
struct AFSCB_Data_StoreData {
afs_uint64 StoreOffset;
afs_uint64 StoreLength;
afs_uint64 Length;
AFSCBFileStatus FileStatus;
};
StoreLength bytes were stored starting at position StoreOffset in=20
FID. Length is the current file length and FileStatus contains=20
the modification time of FID following the operation. The client=20
must regard cached file data in the range [StoreOffset,=20
StoreLength) as invalidated, and may regard data outside that=20
range as up-to-date. The client MUST discard undirtied cached=20
data in the invalidated range. The client MAY send dirtied data=20
in the invalidated range to the file server prior to discarding=20
(as allowed in current AFS semantics).
3.4.5 AFSCB_Event_StoreACL
ACL and/or access information cached by the client for FID, if=20
any, is invalidated.
3.4.6 AFSCB_Event_StoreStatus
A StoreStatus RPC was successfully executed on FID. A structure=20
of type AFSCB_Data_StoreStatus is sent with the message.
struct AFSCB_Data_StoreStatus {
struct AFSStoreStatus Status;
};
Status is the new AFSStoreStatus of FID.
3.4.7 AFSCB_Event_CreateFile
A file has been created in the vnode corresponding to FID. A=20
structure of type AFSCB_Data_CreateFile is sent with the message.
struct AFSCB_Data_CreateFile {
string Name<AFSNAMEMAX>;
AFSFid Fid;
AFSFetchStatus FidStatus;
AFSCBDirStatus DirStatus;
};
Name and Fid are, respectively, the name and FID of the created=20
file. FidStatus is the AFSFetchStatus of the created file, and=20
DirStatus the current modification time and link count of FID, at=20
the completion of the call.
3.4.8 AFSCB_Event_MakeDir
A directory has been created in the vnode corresponding to FID. A=20
structure of type AFSCB_Data_MakeDir is sent with the message.
struct AFSCB_Data_MakeDir {
string Name<AFSNAMEMAX>;
AFSFid Fid;
AFSFetchStatus FidStatus;
AFSCBDirStatus DirStatus;
};
Name and Fid are, respectively, the name and FID of the created=20
directory. FidStatus is the AFSFetchStatus of the created=20
directory, and DirStatus the current modification time and link=20
count of FID, at the completion of the call.
3.4.9 AFSCB_Data_Symlink
A symbolic link has been created in the vnode corresponding to=20
FID. A structure of type AFSCB_Data_Symlink is sent with the=20
message.
struct AFSCB_Data_Symlink {
string Name<AFSNAMEMAX>;
string LinkContents<AFSPATHMAX>;
AFSFetchStatus FidStatus;
AFSCBDirStatus DirStatus;
};
Name is the name of the symbolic link. The link points to=20
LinkContents. FidStatus is the AFSFetchStatus of the created=20
symbolic link, and DirStatus the current modification time and=20
link count of FID, at the completion of the call.
3.4.10 AFSCB_Event_Link
A hard link has been created in the vnode corresponding to FID. A=20
structure of type AFSCB_Data_Link is sent with the message.
struct AFSCB_Data_Link {
string Name<AFSNAMEMAX>;
AFSFid LinkTarget;
AFSFetchStatus FidStatus;
AFSCBDirStatus DirStatus;
};
Name is the name of the hard link. The link is a synonym for=20
LinkTarget. FidStatus is the AFSFetchStatus of the created=20
symbolic link, and DirStatus the current modification time and=20
link count of FID, at the completion of the call.
3.4.11 AFSCB_Event_RemoveFile
A file has been removed from the vnode corresponding to FID. A=20
structure of type AFSCB_Data_RemoveFile is sent with the message.
struct AFSCB_Data_RemoveFile {
string Name<AFSNAMEMAX>;
AFSCBDirStatus DirStatus;
};
Name indicates the removed entry. DirStatus the current=20
modification time and link count of FID, at the completion of the=20
call.
3.4.12 AFSCB_Event_RemoveDir
A directory has been removed from the vnode corresponding to FID.=20
A structure of type AFSCB_Data_RemoveDir is sent with the=20
message.
struct AFSCB_Data_RemoveDir {
string Name<AFSNAMEMAX>;
AFSCBDirStatus DirStatus;
};
Name indicates the removed entry. DirStatus the current=20
modification time and link count of FID, at the completion of the=20
call.
3.4.13 AFSCB_Event_Rename
A file or directory has been renamed, ie moved, from or to the=20
vnode corresponding to FID. A structure of type=20
AFSCB_Data_RemoveDir is sent with the message.
const AFSCB_Rename_From =3D 1;
const AFSCB_Rename_To =3D 2;
struct AFSCB_Data_Rename {
afs_uint32 Direction;
string Name<AFSNAMEMAX>;
AFSCBDirStatus FromStatus;
AFSCBDirStatus ToStatus;
};
Direction indicates whether FID is the source or the destination=20
directory of the move. Name indicates the removed entry.=20
FromStatus is the current modification time and link count of the=20
source directory vnode, and ToStatus is the current modification=20
time and link count of the destination directory vnode, at the=20
completion of the call.=20
To preserve atomicity, the AFSCB_Data_Rename message is=20
constructed so that changes to cached copies of both the source=20
and directory vnodes may be recovered from a single notification.=20
If a client owns callbacks for both the source and destination=20
FIDs, a file server MAY elect to send only one notification, for=20
either the source or the destination FID.
3.4.14 AFSCB_Event_Deleted
The object corresponding to FID not longer exists, and so may no=20
longer be cached. It is an ex-object.
3.4.15 AFSCB_Event_ReleaseLock
A traditional AFS whole-file lock has been released on FID. A=20
structure of type AFSCB_Data_Lock is sent with the message.
struct AFSCB_Data_Lock {
afs_uint32 LockType;
};
LockType is the type of the lock released.
Receipt of an AFSCB_Event_ReleaseLock notification in no way=20
implies an intention on the part of a file server to grant a lock=20
on FID to client. Non-receipt of a notification of this type in=20
no way implies non-release of locks that may be held on FID. The=20
file server SHOULD send notifications of this type only to=20
clients which have indicated probable interest in the event, eg,=20
by having recently requested a lock on FID.
4 Appendix A: XDR Grammar
#include "common.xg" /*Common structures & definitions*/
%#ifdef KERNEL
%#include "../afs/longc_procs.h"
%#endif
package RXAFSCB_
prefix S
statindex 6
/* callback event types, predominantly events on the vnode for=20
* which the callback is being made, but also (eg, Deleted) side=20
* effects of operations on related vnodes */
const AFSCB_Event_Cancel =3D 1; /* explicit cancel--callback=20
promise
* is broken, dont' bother=20
fetching=20
* new status */
const AFSCB_Event_StoreData =3D 2; /* data in file changed */
const AFSCB_Event_StoreACL =3D 3; /* ACL changed on vnode */
const AFSCB_Event_StoreStatus =3D 4; /* status stored on vnode */
const AFSCB_Event_CreateFile =3D 5; /* file created in directory=20
vnode */
const AFSCB_Event_MakeDir =3D 6; /* dir created in directory=20
vnode */
const AFSCB_Event_Symlink =3D 7; /* symlink created in=20
directory vnode */
const AFSCB_Event_Link =3D 8; /* hard link created in=20
directory vnode */
const AFSCB_Event_RemoveFile =3D 9; /* file removed from=20
directory vnode */
const AFSCB_Event_RemoveDir =3D 10; /* dir removed from directory=20
vnode */
const AFSCB_Event_Rename =3D 11; /* object renamed (moved) */
const AFSCB_Event_Deleted =3D 12; /* object no longer exists,=20
ex object */
const AFSCB_Event_ReleaseLock =3D 13; /* traditional AFS lock=20
released */
/* flags indended for use in AFSExtendedCallback Flags */
const AFSCB_Flag_Cancel =3D 1; /* Callback promise is cancelled */
/* flags intended for use in AFSExtendedCallback ExtraFlags,=20
* when (flags & AFSCB_Flag_Cancel), to indicate reason for
* cancellation */
const AFSCB_Cancel_Shutdown =3D 1;
const AFSCB_Cancel_CallbackGC =3D 2;
const AFSCB_Cancel_VolumeOffline =3D 4;
const AFSCB_Cancel_VolumeMoved =3D 8;
/* identical with decl in afsint.xg--this should move to=20
common.xg */
struct AFSStoreStatus {
afs_uint32 Mask;=20
afs_uint32 ClientModTime;
afs_uint32 Owner;
afs_uint32 Group;
afs_uint32 UnixModeBits;
afs_uint32 SegSize;
};
/* differential status to be send with StoreData msgs */
AFSCBFileStatus {
afs_uint64 ClientModTime;
};
/* differential status to be sent with directory change msgs */
AFSCBDirStatus {
afs_uint32 LinkCount;
afs_uint64 ClientModTime;
};
struct AFSCB_Data_StoreData {
afs_uint64 StoreOffset;
afs_uint64 StoreLength;
afs_uint64 Length;
AFSCBFileStatus FileStatus;
};
struct AFSCB_Data_StoreStatus {
struct AFSStoreStatus Status;
};
struct AFSCB_Data_CreateFile {
string Name<AFSNAMEMAX>;
AFSFid Fid;
AFSFetchStatus FidStatus;
AFSCBDirStatus DirStatus;
};
struct AFSCB_Data_MakeDir {
string Name<AFSNAMEMAX>;
AFSFid Fid;
AFSFetchStatus FidStatus;
AFSCBDirStatus DirStatus;
};
struct AFSCB_Data_Symlink {
string Name<AFSNAMEMAX>;
string LinkContents<AFSPATHMAX>;
AFSFetchStatus FidStatus;
AFSCBDirStatus DirStatus;
};
struct AFSCB_Data_Link {
string Name<AFSNAMEMAX>;
AFSFid LinkTarget;
AFSFetchStatus FidStatus;
AFSCBDirStatus DirStatus;
};
struct AFSCB_Data_RemoveFile {
string Name<AFSNAMEMAX>;
AFSCBDirStatus DirStatus;
};
struct AFSCB_Data_RemoveDir {
string Name<AFSNAMEMAX>;
AFSCBDirStatus DirStatus;
};
const AFSCB_Rename_From =3D 1;
const AFSCB_Rename_To =3D 2;
struct AFSCB_Data_Rename {
afs_uint32 Direction;
string Name<AFSNAMEMAX>;
AFSCBDirStatus FromStatus;
AFSCBDirStatus ToStatus;
};
struct AFSCB_Data_Lock {
afs_uint32 LockType;
};
union AFSCB_NotificationData switch (afs_uint32 Event_Type) {
case AFSCB_Event_StoreData:
AFSCB_Data_StoreData u_data;
case AFSCB_Event_StoreACL:
void;
case AFSCB_Event_StoreStatus:
AFSCB_Data_StoreStatus u_data;
case AFSCB_Event_CreateFile:
AFSCB_Data_CreateFile u_data;
case AFSCB_Event_MakeDir:
AFSCBName_Data_MakeDir u_data;
case AFSCB_Event_Symlink:
AFSCB_Data_Symlink u_data;
case AFSCB_Event_Link:
AFSCBName_Data_Link u_data;
case AFSCB_Event_RemoveFile:
AFSCB_Data_RemoveFile u_data;
case AFSCB_Event_RemoveDir:
AFSCB_Data_RemoveDir u_data;
case AFSCB_Event_Rename:
AFSCB_Data_Rename u_data;
case AFSCB_Event_Deleted:
void;
case AFSCB_Event_ReleaseLock:
AFSCB_Data_Lock u_data;
case AFSCB_Event_Cancel:
void;
};
/* extended callback structure */
struct AFSExtendedCallBack {
afs_uint32 Flags;
afs_uint32 ExtraFlags;
afs_uint32 DataVersion;
afs_uint32 ExpirationTime;
AFSCB_NotificationData Data;
};
/* this prototype follows the style of the traditional AFS=20
* CallBack proc, and is not intended to imply any change in=20
* style of invocation in the OpenAFS file server, eg, coalescing
* of call backs (though this might be interesting to explore in
* future) */
typedef AFSExtendedCallBack AFSExtendedCallBackSeq<AFSCBMAX>;
proc ExtendedCallBack(
IN HostIdentifier *Server,
AFSCBFids *Fids_Array,
AFSExtendedCallBackSeq *CallBacks_Array
) multi =3D 65539;
References
[1] Bradner, S., "Key words for use in RFCs to Indicate=20
Requirement Levels", BCP 14, RFC 2119, March 1997.
[2] Howard, J.H., Kazar, M.L., Menees, S.G., Nichols, D.A.,=20
Satyanarayanan, M., Sidebotham, R.N. and West, M. "Scale and=20
Performance in a Distributed File System" ACM Transactions on=20
Computer Systems, February 1988=20
[3] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., Beame,=20
C., Eisler, M., and D. Noveck, "Network File System (NFS) version=20
4 Protocol", RFC 3530, April 2003.
[4] Edward R Zayas, "AFS-3 Programmer's Reference: File=20
Server/Cache Manager Interface", Transarc Corporation,=20
FS-00-D162, 20th August 1991
[5] Paul J. Leach, Dilip C. Naik. A Common Internet File System=20
(CIFS/1.0) Protocol=20
[http://www.tools.ietf.org/html/draft-leach-cifs-v1-spec-01],=20
1997.
[6] Kazar, Michael Leon, "Synchronization and Caching Issues in=20
the Andrew File System," USENIX Conference Proceedings, USENIX=20
Association, Berkeley, CA, Dallas Winter 1988, pages 27-36.
[7] Lily B. Mummert, Mahadev Satyanarayanan: Large Granularity=20
Cache Coherence for Intermittent Connectivity. USENIX Summer=20
1994: 279-289
--------------030208080809080806000608--