[AFS3-std] Callback Extended Information Draft

Matt Benjamin matt@linuxbox.com
Fri, 09 May 2008 10:38:00 -0400


This is a multi-part message in MIME format.
--------------020008040206080201080504
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Written it I have.  Comment on it you may.

Thanks,

Matt

Ps.  Extra thanks to Jeff Altman, Tom Keiser, Derrick Brashear, Jeff
Hutzelmann for feedback and discussion so far.

- --

Matt Benjamin

The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIJGHHJiSUUSaRdSURCDPHAJwM0Z3COKwFoiUb2C4UBvAuqqGK5gCgiafj
BDuT7/w18P1U9Y7HSCtJ7rc=
=ZI/r
-----END PGP SIGNATURE-----

--------------020008040206080201080504
Content-Type: text/plain;
 name="callback_extension_d2.txt"
Content-Disposition: inline;
 filename="callback_extension_d2.txt"
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by aa.linuxbox.com id
	m49Ec8QV003647

AFS Callback Extensions

Matt Benjamin <matt@linuxbox.com>

05/09/2008

Status of this Memo

This document specifies a standards track protocol extension for=20
the OpenAFS community, and requests discussion and suggestions=20
for improvements.

Key Words

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL=20
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and=20
"OPTIONAL" in this document are to be interpreted as described in=20
Internet Engineering Task Force RFC 2119.

Abstract

AFS cache-control strategy is callback (invalidate) based. The=20
AFS callback design allows a client to know when an object it has=20
cached is no longer consistent, but the callback notification=20
message itself provides no specific information about the=20
triggering event. This is a protocol inefficiency, as in several=20
scenarios it results in unnecessary round-trips to file servers=20
to verify file status information, file access information, or to=20
fetch file data which has not changed. We propose an extension of=20
the callback mechanism to provide information about the event(s)=20
triggering a callback, in the payload of the callback=20
notification message itself. The proposed mechanism eliminates=20
most or all unnecessary round-trips imposed by the current=20
callback mechanism, and simultaneously allows AFS implementations=20
to (efficiently) provide correct semantics in several scenarios=20
involving multiple writers (ie, where AFS currently provides=20
incorrect semantics).

Table of Contents

Table of Contents

Status of this Memo
Key Words
Abstract
Table of Contents
    1 Introduction
    2 The AFS Callback Mechanism
        2.1 Description
        2.2 Analysis
    3 Extended Callback Interface
        3.1 Backward Compatibility
        3.2 Interface Changes
            3.2.1 Procedures
            3.2.2 Constants
            3.2.3 Data Types
                AFSExtendedCallback
                AFSCBFileStatus
                AFSCBDirStatus
                AFSCB_NotificationData
        3.3 Semantic Changes
            3.3.1 DataVersion Rule
        3.4 Callback Invocations
            3.4.1 AFSExtendedCallback
                Flags
                ExtraFlags
                DataVersion
                ExpirationTime
                Data
            3.4.2 Reasons for Cancellation
                AFSCB_Cancel_Shutdown
                AFSCB_Cancel_CallbackGC
                AFSCB_Cancel_VolumeOffline
                AFSCB_Cancel_VolumeMoved
                AFSCB_Cancel_LostMyMind
                AFSCB_Cancel_IHateYou
            3.4.3 AFSCB_Event_StoreData
            3.4.4 AFSCB_Event_StoreACL
            3.4.5 AFSCB_Event_StoreStatus
            3.4.6 AFSCB_Event_CreateFile
            3.4.7 AFSCB_Event_MakeDir
            3.4.8 AFSCB_Data_Symlink
            3.4.9 AFSCB_Event_Link
            3.4.10 AFSCB_Event_RemoveFile
            3.4.11 AFSCB_Event_RemoveDir
            3.4.12 AFSCB_Event_Rename
            3.4.13 AFSCB_Event_Deleted
            3.4.14 AFSCB_Event_ReleaseLock
    4 Appendix A: XDR Grammar


1 Introduction

The AFS protocol provides a comprehensive framework for scalable,=20
secure, wide-area file sharing over IP networks. The AFS system=20
has historically distinguished itself through its emphasis on=20
scalability, a key source of which is client-side caching[1, 4].=20
File data, file and directory metadata, and access control=20
information may all be cached. Cache consistency is maintained=20
through client registration and an associated asynchronous=20
notification mechanism known as the callback.=20

The current AFS consistency model (which is of larger scope than=20
the callback mechanism, eg, it includes AFS sync-on-close=20
semantics) has allowed AFS to scale to large numbers of clients=20
(tens of thousands today), and to perform well under the=20
workloads for which AFS was originally designed.

However, AFS does not perform efficiently under other conditions,=20
such as when more than one client is interested in a file which=20
is changing--even if the file has only one writer, and many=20
readers[footnote:
A scenario which competing protocols efficiently support.=20
]. In general, the AFS protocol arguably (still, considering=20
improvements made between AFS-2 and AFS-3) places too little=20
emphasis on efficient caching of mutable data. The current AFS=20
consistency model is insufficient to correctly support=20
single-file, multiple-writer scenarios, including those required=20
for POSIX semantics, and therefore is insufficient to support=20
many applications which may be run correctly on competing=20
distributed file systems (eg, CIFS, Novell Netware, or NFSv4).

The efficiency of the current AFS cache management algorithm=20
could be substantially improved if specific triggering event=20
information and current status were included in the payload of=20
the callback notifications sent to clients. In particular,=20
inclusion of the current DataVersion number and affected byte=20
ranges in response to StoreData operations would significantly=20
reduce the need for cache revalidation and reconstruction traffic=20
in response to callbacks--in many cases, altogether. These=20
changes would allow efficient support for single-writer updates=20
on a file with multiple readers. More importantly, they would=20
permit AFS to correctly and efficiently support multiple writers=20
updating disjoint ranges on a single file, a prerequisite for=20
supporting granular file locking (and applications which require=20
it) in future.

2 The AFS Callback Mechanism

2.1 Description

When an AFS-3 client contacts a file server to perform any of=20
several operations on a file, or explicitly to fetch its status,=20
the file server includes in its RPC response an AFSCallBack=20
structure, representing the server's promise to call back the=20
client ``if any modifications are made to the data in the file''.[footnot=
e:
A key paper on AFS-2 has ``before allowing a modification by any=20
other workstation''[1]. The wording of this statement appears=20
calculated to imply that the file server's promise to execute=20
callbacks synchronously with the triggering operations (eg,=20
StoreData) specifically constitutes part of the AFS cache=20
consistency guarantee. In our analysis, it does not, though it=20
does contribute strongly to the simplification of the file server=20
design and to reduction of file server workload.
]The AFSCallBack structure contains the callback expiration time,=20
and two integer values treated as invariants.

When any client executes an operation which would change a file=20
(eg, StoreData), and in a variety of other situations, the file=20
server invalidates the client's cached copy by executing a call=20
to the CallBack[footnote:
formerly BreakCallback
] procedure in the client's RPC interface. (The call includes in=20
its arguments an AFSCallBack structure for each file being=20
invalidated. However, the value of passed AFSCallBack is unused=20
[eg, afs/afs_callback.c:661-2]). Between the time of issue and=20
either expiry or receipt of a callback, the client may consider=20
any information it has cached on a file to be consistent with the=20
file server's on-disk copy. Conversely, on receipt of a callback,=20
the client must consider that it knows nothing about the file.=20
Thus the client must re-establish a relationship with the file at=20
the file server before executing any further operations on it.[footnote:
Since AFS supports the notion of a read-only volume all of whose=20
files may only be updated transactionally as a group, AFS permits=20
a file server to issue a single callback when any file in a=20
read-only volume is accessed. This is a significant performance=20
optimization for, by definition, cache management of immutable=20
data, and so is not discussed further here.
]

The AFS callback mechanism obviates the need for clients to send=20
frequent cache validation requests before performing operations=20
on their locally cached copies of objects, reducing network=20
traffic as well as file server workload[4]. The callback=20
innovation has been since taken up, with variations, by other=20
distributed file system protocols[2, 3, 5].

2.2 Analysis

The AFS callback mechanism reliably notifies clients when=20
information they may have cached is becomes invalidated, but=20
omits to send information it trivially knows, ie, the triggering=20
event, that could certainly be used by the client to more=20
efficiently manage cache state.

For example, consider the case where 2 clients A and B are=20
interested in a file, each having read chunks 1-15 into cache.=20
Now another client C initiates a change in the file, writing a=20
new state to chunk 45. This event today triggers a callback, but=20
also invalidates 30 chunks correctly cached on A and B, which,=20
should they remain interested, they must refetch (up to 2=20
megabytes of data, in this case). This scenario may seem=20
relatively unlikely to occur (but of course, probably does occur=20
reasonably often in environments where mutable data is common),=20
but a related scenario involving directory entries (omitted for=20
brevity) is much more common. In these cases, an AFS callback=20
mechanism capable of sending triggering event information with=20
the callback would have facilitated a more efficient result, at=20
small marginal cost. In another set of scenarios where a client A=20
has changed data in a file invalidated by non-overlapping stores=20
by B, a revised mechanism would be capable of delivering a=20
correct result, whereas a correct result would be impossible with=20
the mechanism in AFS today.

The justification for sending minimal information with the=20
callback is presumably to minimize the execution cost of the=20
callback procedure. The increased cost of sending a limited but=20
informative callback notification to clients, relative to sending=20
an uninformative one, is small. Analysis of the OpenAFS file=20
server code reveals that the file server always has the=20
information that would logically be sent as extended callback=20
information in response to file operations (eg, file ranges=20
affected by StoreData operations, or changed entries for various=20
directory modification operations).=20

For these reasons, enhancement of the AFS callback interface to=20
supply triggering event information seems likely to improve both=20
correctness and performance of AFS implementations, and=20
experimental implementation and profiling appear justified.

3 Extended Callback Interface

3.1 Backward Compatibility

AFS clients will indicate their preference to receive extended=20
callback notifications through a new client capability flag:

const CLIENT_CAPABILITY_EXT_CALLBACK      =3D 0x0002;

3.2 Interface Changes

3.2.1 Procedures

We propose a new procedure ExtendedCallback in the client's RPC=20
interface. ExtendedCallback follows the style of the traditional=20
AFS CallBack procedure in accepting parallel sequences of FIDs=20
and structures:

typedef AFSExtendedCallBack AFSExtendedCallBackSeq<AFSCBMAX>;

proc ExtendedCallBack(

    IN  AFSCBFids *Fids_Array,

    AFSExtendedCallBackSeq *CallBacks_Array

) multi =3D 65539;

As detailed in section [sub:Constants], AFSExtendedCallBackSeq=20
resolves to a sequence of AFSExtendedCallBack structures whose=20
type is an XDR union, discriminated on the callback event type.

3.2.2 Constants<sub:Constants>

The following callback event types are defined:

const AFSCB_Event_Cancel =3D 1; /* extended */

const AFSCB_Event_StoreData =3D 2; /* data in file changed */

const AFSCB_Event_StoreACL =3D 3; /* ACL changed on vnode */

const AFSCB_Event_StoreStatus =3D 4; /* status stored on vnode */

const AFSCB_Event_CreateFile =3D 5; /* file created in directory=20
vnode */

const AFSCB_Event_MakeDir =3D 6; /* dir created in directory vnode=20
*/

const AFSCB_Event_Symlink =3D 7; /* symlink created in directory=20
vnode */

const AFSCB_Event_Link =3D 8; /* hard link created in directory=20
vnode */

const AFSCB_Event_RemoveFile =3D 9; /* file removd from directory=20
vnode */

const AFSCB_Event_RemoveDir =3D 10; /* dir removed from directory=20
vnode */

const AFSCB_Event_Rename =3D 11; /* object renamed (moved) */

const AFSCB_Event_Deleted =3D 12; /* object no longer exists, ex=20
object */

const AFSCB_Event_ReleaseLock =3D 13; /* traditional AFS lock=20
released */

A flag constant is provided to indicate callback cancellation=20
along with an extended notification message of any of the above=20
types:

const AFSCB_Flag_Cancel =3D 1; /* Callback promise is cancelled */

The following constants indicate reasons for cancellation, when=20
(Flags & AFSCB_Flag_Cancel)

const AFSCB_Cancel_Shutdown =3D 1;

const AFSCB_Cancel_CallbackGC =3D 2;

const AFSCB_Cancel_VolumeOffline =3D 3;

const AFSCB_Cancel_VolumeMoved =3D 4;

const AFSCB_Cancel_LostMyMind =3D 5;

const AFSCB_Cancel_IHateYou =3D 6;=20

The following constants indicate direction (from or to called=20
back FID) in the atomic AFSCB_Event_Rename notification:

const AFSCB_Rename_From =3D 1;

const AFSCB_Rename_To =3D 2;

3.2.3 Data Types

  AFSExtendedCallback

The AFSExtendedCallBack data type contains members Flags,=20
DataVersion, ExpirationTime, and Data, where Flags and ExtraFlags=20
provide extra information, DataVersion is a (possibly=20
incremented) DataVersion, ExpirationTime is a (possibly extended)=20
callback expiration time, and Data is an object of the=20
discriminated union type AFSCB_NotificationData:

struct AFSExtendedCallBack {

    afs_uint32 Flags;

    afs_uint32 ExtraFlags;

    afs_uint32 DataVersion;

    afs_uint32 ExpirationTime;

    AFSCB_NotificationData Data;

};

A positive value in Flags for the AFSCB_Flag_Cancel bit indicates=20
cancellation of the callback upon receipt of the message. In that=20
event, a non-zero value of ExtraFlags indicates the reason for=20
the cancellation.

  AFSCBFileStatus

The AFSCBFileStatus structure is a reduced-footprint=20
AFSFetchStatus replacement intended to communicate changed vnode=20
information in response to StoreData operations:

AFSCBFileStatus {

    afs_uint64 ClientModTime;

};

  AFSCBDirStatus

The AFSCBDirStatus structure is a reduced-footprint=20
AFSFetchStatus replacement intended to communicate changed vnode=20
information in response to StoreData operations:

AFSCBDirStatus {

    afs_uint32 LinkCount;

    afs_uint64 ClientModTime;

};

  AFSCB_NotificationData

AFSCB_NotificationData is a union discriminated by callback event=20
type, ie, its value may be any of the constants defined in=20
section [sub:Constants].

union AFSCB_NotificationData switch (afs_uint32 Event_Type) {

case AFSCB_Event_StoreData:

    AFSCB_Data_StoreData u_data;

case AFSCB_Event_StoreACL:

    void;

case AFSCB_Event_StoreStatus:

    AFSCB_Data_StoreStatus u_data;

case AFSCB_Event_CreateFile:

    AFSCB_Data_CreateFile u_data;

case AFSCB_Event_MakeDir:

    AFSCBName_Data_MakeDir u_data;

case AFSCB_Event_Symlink:

    AFSCB_Data_Symlink u_data;

case AFSCB_Event_Link:

    AFSCBName_Data_Link u_data;

case AFSCB_Event_RemoveFile:

    AFSCB_Data_RemoveFile u_data;

case AFSCB_Event_RemoveDir:

    AFSCB_Data_RemoveDir u_data;

case AFSCB_Event_Rename:

    AFSCB_Data_Rename u_data;

case AFSCB_Event_Deleted:

    void;

case AFSCB_Event_ReleaseLock:

    AFSCB_Data_Lock u_data;

case AFSCB_Event_Cancel:

    void;

};



The types for the variant member u_data are enumerated and=20
discussed in detail in section [sub:Callback-Invocations].=20

3.3 Semantic Changes

A file server MAY send traditional callback messages, with=20
traditional semantics, to any AFS client in response to any=20
event. A file server MAY send extended callback notifications to=20
any client which has announced the capability to use the extended=20
interface, with the following semantics:

=E2=80=A2 extended callback notification messages, in general, preserve=20
  the file server's callback promise to send further=20
  notifications for the called-back FID

=E2=80=A2 the file server may revoke the callback promise with any=20
  extended callback notification message, by setting the=20
  AFSCB_Flag_Cancel bit in the Flags member of the=20
  AFSExtendedCallback structure

=E2=80=A2 the AFSCB_Event_Cancel message is similar to a traditional AFS=20
  callback, breaking the callback promise, and requesting the=20
  client not request further status on the FID

3.3.1 DataVersion Rule

The various extended callback notification messages include=20
information a client may use to selectively invalidate or=20
reconstruct its cache. In interpreting each message, the client=20
MUST observe the dataversion rule, which states:

If the client's cached DataVersion is DataVersion or=20
(DataVersion-1), the client may invalidate or reconstruct its=20
cache using the type-dependent information contained in the=20
message. In all other cases, the client MUST regard the message=20
as equivalent to a traditional AFS callback.

The semantics of specific callback events are enumerated in=20
section [sub:Callback-Invocations].

3.4 Callback Invocations<sub:Callback-Invocations>

The various extended callback notification types generally=20
respond to specific events at the file server, but present a view=20
of it relevant to a specific callback promise at one client. In=20
one case (ie, AFSCB_Event_Rename), the file server is sending=20
notification of an event which effects two FIDs, either or both=20
of which may be cached by the receiving client. A structure of=20
type AFSExtendedCallback is sent with each extended callback=20
notification message, as noted above. Unless otherwise noted, FID=20
is the FID of the object that is the subject of the callback.=20

3.4.1 AFSExtendedCallback

The members of the AFSExtendedCallback structures are to be=20
interpreted as follows:

  Flags

If the 1-bit (AFSCB_Flag_Cancel) is set, the notification effects=20
a callback break. The client may make use of the information sent=20
with the message.

  ExtraFlags

If (Flags & AFSCB_Flag_Cancel), a non-zero value for ExtraFlags=20
indicates the reason for cancellation.

  DataVersion

The value of DataVersion at completion of the event of which the=20
client is being notified.

  ExpirationTime

The new expiration time asserted for the server's callback=20
promise, not necessarily different from the existing expiration=20
cached by the client.

  Data

The message-specific data for this notification.

3.4.2 Reasons for Cancellation

The following reasons for cancellation are defined:

  AFSCB_Cancel_Shutdown

The server or service is shutting down.

  AFSCB_Cancel_CallbackGC

Callback has been disposed during periodic garbage collection.

  AFSCB_Cancel_VolumeOffline

The volume associated with FID is now offline.

  AFSCB_Cancel_VolumeMoved

The volume associated with FID has moved.

  AFSCB_Cancel_LostMyMind

The server may be having problems related to provisioning an=20
insufficient number of callback structures.

  AFSCB_Cancel_IHateYou

Callback has been administratively revoked.

3.4.3 AFSCB_Event_StoreData

The notification is sent in response to a successful StoreData=20
RPC on FID. A structure of type AFSCB_Data_StoreData is sent with=20
the message.

struct AFSCB_Data_StoreData {

    afs_uint64 StoreOffset;

    afs_uint64 StoreLength;

    afs_uint64 Length;

    AFSCBFileStatus FileStatus;

};

StoreLength bytes were stored starting at position StoreOffset in=20
FID. Length is the current file length and FileStatus contains=20
the modification time of FID following the operation. The client=20
must regard cached file data in the range [StoreOffset,=20
StoreLength) as invalidated, and may regard data outside that=20
range as up-to-date. The client MUST discard undirtied cached=20
data in the invalidated range. The client MAY send dirtied data=20
in the invalidated range to the file server prior to discarding=20
(as allowed in current AFS semantics).

3.4.4 AFSCB_Event_StoreACL

ACL and/or access information cached by the client for FID, if=20
any, is invalidated.

3.4.5 AFSCB_Event_StoreStatus

A StoreStatus RPC was successfully executed on FID. A structure=20
of type AFSCB_Data_StoreStatus is sent with the message.

struct AFSCB_Data_StoreStatus {

    struct AFSStoreStatus Status;

};

Status is the new AFSFetchStatus of FID.

3.4.6 AFSCB_Event_CreateFile

A file has been created in the vnode corresponding to FID. A=20
structure of type AFSCB_Data_CreateFile is sent with the message.

struct AFSCB_Data_CreateFile {

    string Name<AFSNAMEMAX>;

    AFSFid Fid;

    AFSFetchStatus FidStatus;

    AFSCBDirStatus DirStatus;

};

Name and Fid are, respectively, the name and FID of the created=20
file. FidStatus is the AFSFetchStatus of the created file, and=20
DirStatus the current modification time and link count of FID, at=20
the completion of the call.

3.4.7 AFSCB_Event_MakeDir

A directory has been created in the vnode corresponding to FID. A=20
structure of type AFSCB_Data_MakeDir is sent with the message.

struct AFSCB_Data_MakeDir {

    string Name<AFSNAMEMAX>;

    AFSFid Fid;

    AFSFetchStatus FidStatus;

    AFSCBDirStatus DirStatus;

};

Name and Fid are, respectively, the name and FID of the created=20
directory. FidStatus is the AFSFetchStatus of the created=20
directory, and DirStatus the current modification time and link=20
count of FID, at the completion of the call.

3.4.8 AFSCB_Data_Symlink

A symbolic link has been created in the vnode corresponding to=20
FID. A structure of type AFSCB_Data_Symlink is sent with the=20
message.

struct AFSCB_Data_Symlink {

    string Name<AFSNAMEMAX>;

    string LinkContents<AFSPATHMAX>;

    AFSFetchStatus FidStatus;

    AFSCBDirStatus DirStatus;

};

Name is the name of the symbolic link. The link points to=20
LinkContents. FidStatus is the AFSFetchStatus of the created=20
symbolic link, and DirStatus the current modification time and=20
link count of FID, at the completion of the call.

3.4.9 AFSCB_Event_Link

A hard link has been created in the vnode corresponding to FID. A=20
structure of type AFSCB_Data_Link is sent with the message.

struct AFSCB_Data_Link {

    string Name<AFSNAMEMAX>;

    AFSFid LinkTarget;

    AFSFetchStatus FidStatus;

    AFSCBDirStatus DirStatus;

};

Name is the name of the hard link. The link is a synonym for=20
LinkTarget. FidStatus is the AFSFetchStatus of the created=20
symbolic link, and DirStatus the current modification time and=20
link count of FID, at the completion of the call.

3.4.10 AFSCB_Event_RemoveFile

A file has been removed from the vnode corresponding to FID. A=20
structure of type AFSCB_Data_RemoveFile is sent with the message.

struct AFSCB_Data_RemoveFile {

    string Name<AFSNAMEMAX>;

    AFSCBDirStatus DirStatus;

};

Name indicates the removed entry. DirStatus the current=20
modification time and link count of FID, at the completion of the=20
call.

3.4.11 AFSCB_Event_RemoveDir

A directory has been removed from the vnode corresponding to FID.=20
A structure of type AFSCB_Data_RemoveDir is sent with the=20
message.

struct AFSCB_Data_RemoveDir {

    string Name<AFSNAMEMAX>;

    AFSCBDirStatus DirStatus;

};

Name indicates the removed entry. DirStatus the current=20
modification time and link count of FID, at the completion of the=20
call.

3.4.12 AFSCB_Event_Rename

A file or directory has been renamed, ie moved, from or to the=20
vnode corresponding to FID. A structure of type=20
AFSCB_Data_RemoveDir is sent with the message.

const AFSCB_Rename_From =3D 1;

const AFSCB_Rename_To =3D 2;



struct AFSCB_Data_Rename {

    afs_uint32 Direction;

    string Name<AFSNAMEMAX>;

    AFSCBDirStatus FromStatus;

    AFSCBDirStatus ToStatus;

};

Direction indicates whether FID is the source or the destination=20
directory of the move. Name indicates the removed entry.=20
FromStatus is the current modification time and link count of the=20
source directory vnode, and ToStatus is the current modification=20
time and link count of the destination directory vnode, at the=20
completion of the call.=20

To preserve atomicity, the AFSCB_Data_Rename message is=20
constructed so that changes to cached copies of both the source=20
and directory vnodes may be recovered from a single notification.=20
If a client owns callbacks for both the source and destination=20
FIDs, a file server MAY elect to send only one notification, for=20
either the source or the destination FID.

3.4.13 AFSCB_Event_Deleted

The object corresponding to FID not longer exists, and so may no=20
longer be cached. It is an ex-object.

3.4.14 AFSCB_Event_ReleaseLock

A traditional AFS whole-file lock has been released on FID. A=20
structure of type AFSCB_Data_Lock is sent with the message.

struct AFSCB_Data_Lock {

    afs_uint32 LockType;

};

LockType is the type of the lock released.

Receipt of an AFSCB_Event_ReleaseLock notification in no way=20
implies any intention on the part of a file server to grant a=20
lock on FID to client. Non-receipt of a notification of this type=20
in no way implies non-release of locks that may be held on FID.=20
The file server SHOULD send notifications of this type only to=20
clients which have indicated probable interest in the event, eg,=20
by having recently requested a lock on FID.

4 Appendix A: XDR Grammar

#include "common.xg" /*Common structures & definitions*/

%#ifdef KERNEL

%#include "../afs/longc_procs.h"

%#endif



package RXAFSCB_

prefix S

statindex 6



/* callback event types, predominantly events on the vnode for=20

* which the callback is being made, but also (eg, Deleted) side=20

* effects of operations on related vnodes */

const AFSCB_Event_Cancel =3D 1;       /* explicit cancel--callback=20
promise

                                     * is broken, dont' bother=20
fetching=20

                                     * new status */

const AFSCB_Event_StoreData =3D 2;    /* data in file changed */

const AFSCB_Event_StoreACL =3D 3;     /* ACL changed on vnode */

const AFSCB_Event_StoreStatus =3D 4;  /* status stored on vnode */

const AFSCB_Event_CreateFile =3D 5;   /* file created in directory=20
vnode */

const AFSCB_Event_MakeDir =3D 6;      /* dir created in directory=20
vnode */

const AFSCB_Event_Symlink =3D 7;      /* symlink created in=20
directory vnode */

const AFSCB_Event_Link =3D 8;         /* hard link created in=20
directory vnode */

const AFSCB_Event_RemoveFile =3D 9;   /* file removed from=20
directory vnode */

const AFSCB_Event_RemoveDir =3D 10;   /* dir removed from directory=20
vnode */

const AFSCB_Event_Rename =3D 11;      /* object renamed (moved) */

const AFSCB_Event_Deleted =3D 12;     /* object no longer exists,=20
ex object */

const AFSCB_Event_ReleaseLock =3D 13; /* traditional AFS lock=20
released */



/* flags indended for use in AFSExtendedCallback Flags */

const AFSCB_Flag_Cancel =3D 1; /* Callback promise is cancelled */



/* flags intended for use in AFSExtendedCallback ExtraFlags,=20

 * when (flags & AFSCB_Flag_Cancel), to indicate reason for

 * cancellation */

const AFSCB_Cancel_Shutdown =3D 1;

const AFSCB_Cancel_CallbackGC =3D 2;

const AFSCB_Cancel_VolumeOffline =3D 4;

const AFSCB_Cancel_VolumeMoved =3D 8;



/* identical with decl in afsint.xg--this should move to=20
common.xg */

struct AFSStoreStatus {

    afs_uint32 Mask;=20

    afs_uint32 ClientModTime;

    afs_uint32 Owner;

    afs_uint32 Group;

    afs_uint32 UnixModeBits;

    afs_uint32 SegSize;

};



/* differential status to be send with StoreData msgs */

AFSCBFileStatus {

afs_uint64 ClientModTime;

};



/* differential status to be sent with directory change msgs */

AFSCBDirStatus {

afs_uint32 LinkCount;

afs_uint64 ClientModTime;

};



struct AFSCB_Data_StoreData {

    afs_uint64 StoreOffset;

    afs_uint64 StoreLength;

    afs_uint64 Length;

    AFSCBFileStatus FileStatus;

};



struct AFSCB_Data_StoreStatus {

    struct AFSStoreStatus Status;

};



struct AFSCB_Data_CreateFile {

    string Name<AFSNAMEMAX>;

    AFSFid Fid;

    AFSFetchStatus FidStatus;

    AFSCBDirStatus DirStatus;

};



struct AFSCB_Data_MakeDir {

    string Name<AFSNAMEMAX>;

    AFSFid Fid;

    AFSFetchStatus FidStatus;

    AFSCBDirStatus DirStatus;

};



struct AFSCB_Data_Symlink {

    string Name<AFSNAMEMAX>;

    string LinkContents<AFSPATHMAX>;

    AFSFetchStatus FidStatus;

    AFSCBDirStatus DirStatus;

};



struct AFSCB_Data_Link {

    string Name<AFSNAMEMAX>;

    AFSFid LinkTarget;

    AFSFetchStatus FidStatus;

    AFSCBDirStatus DirStatus;

};



struct AFSCB_Data_RemoveFile {

    string Name<AFSNAMEMAX>;

    AFSCBDirStatus DirStatus;

};



struct AFSCB_Data_RemoveDir {

    string Name<AFSNAMEMAX>;

    AFSCBDirStatus DirStatus;

};



const AFSCB_Rename_From =3D 1;

const AFSCB_Rename_To =3D 2;



struct AFSCB_Data_Rename {

    afs_uint32 Direction;

    string Name<AFSNAMEMAX>;

    AFSCBDirStatus FromStatus;

    AFSCBDirStatus ToStatus;

};



struct AFSCB_Data_Lock {

    afs_uint32 LockType;

};



union AFSCB_NotificationData switch (afs_uint32 Event_Type) {

case AFSCB_Event_StoreData:

    AFSCB_Data_StoreData u_data;

case AFSCB_Event_StoreACL:

    void;

case AFSCB_Event_StoreStatus:

    AFSCB_Data_StoreStatus u_data;

case AFSCB_Event_CreateFile:

    AFSCB_Data_CreateFile u_data;

case AFSCB_Event_MakeDir:

    AFSCBName_Data_MakeDir u_data;

case AFSCB_Event_Symlink:

    AFSCB_Data_Symlink u_data;

case AFSCB_Event_Link:

    AFSCBName_Data_Link u_data;

case AFSCB_Event_RemoveFile:

    AFSCB_Data_RemoveFile u_data;

case AFSCB_Event_RemoveDir:

    AFSCB_Data_RemoveDir u_data;

case AFSCB_Event_Rename:

    AFSCB_Data_Rename u_data;

case AFSCB_Event_Deleted:

    void;

case AFSCB_Event_ReleaseLock:

    AFSCB_Data_Lock u_data;

case AFSCB_Event_Cancel:

    void;

};



/* extended callback structure */

struct AFSExtendedCallBack {

    afs_uint32 Flags;

    afs_uint32 ExtraFlags;

    afs_uint32 DataVersion;

    afs_uint32 ExpirationTime;

    AFSCB_NotificationData Data;

};



/* this prototype follows the style of the traditional AFS=20

 * CallBack proc, and is not intended to imply any change in=20

 * style of invocation in the OpenAFS file server, eg, coalescing

 * of call backs (though this might be interesting to explore in

 * future) */

typedef AFSExtendedCallBack AFSExtendedCallBackSeq<AFSCBMAX>;



proc ExtendedCallBack(

    IN AFSCBFids *Fids_Array,

    AFSExtendedCallBackSeq *CallBacks_Array

) multi =3D 65539;

References

[1] Bradner, S., "Key words for use in RFCs to Indicate=20
Requirement Levels", BCP 14, RFC 2119, March 1997.

[2] Howard, J.H., Kazar, M.L., Menees, S.G., Nichols, D.A.,=20
Satyanarayanan, M., Sidebotham, R.N. and West, M. "Scale and=20
Performance in a Distributed File System" ACM Transactions on=20
Computer Systems, February 1988=20

[3] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., Beame,=20
C., Eisler, M., and D. Noveck, "Network File System (NFS) version=20
4 Protocol", RFC 3530, April 2003.

[4] Edward R Zayas, "AFS-3 Programmer's Reference: File=20
Server/Cache Manager Interface", Transarc Corporation,=20
FS-00-D162, 20th August 1991

[5] Paul J. Leach, Dilip C. Naik. A Common Internet File System=20
(CIFS/1.0) Protocol=20
[http://www.tools.ietf.org/html/draft-leach-cifs-v1-spec-01],=20
1997.

[6] Kazar, Michael Leon, "Synchronization and Caching Issues in=20
the Andrew File System," USENIX Conference Proceedings, USENIX=20
Association, Berkeley, CA, Dallas Winter 1988, pages 27-36.

[7] Lily B. Mummert, Mahadev Satyanarayanan: Large Granularity=20
Cache Coherence for Intermittent Connectivity. USENIX Summer=20
1994: 279-289


--------------020008040206080201080504--