[AFS3-std] rpc refresh: FetchDirectory: discussion only

[Tom Keiser]

On Thu, Sep 10, 2009 at 7:22 PM, Jeffrey Hutzelman<jhutz@cmu.edu> wrote:
> --On Thursday, September 10, 2009 07:00:34 PM -0400 "Matt W. Benjamin"
> <matt@linuxbox.com> wrote:
>
>> 5. directory entries cannot be looked up, as server doesn't know
>> applicable normalization rules
>
> I'm not sure I'd go quite that far (and I'm pretty sure I was the one who
> made this argument). =A0I expect a lookup operation will _usually_ not be=
 as
> useful as doing the lookup on the client, for this reason and also due to
> performance concerns. =A0But occasionally it might, so I wouldn't complet=
ely
> rule out the option of a lookup operation.
>

The performance concerns in this space are something we shouldn't take
lightly.  Satyanarayanan, Kazar, and others have pointed out in
numerous papers that experimentation with early vice prototypes
revealed server-assisted path lookup to result in unacceptably high
round trip counts, and a large drain on server resources.  If we do
add a lookup RPC, we should implore implementors to use it sparingly
(if at all), or implement an adaptive algorithm whereby first N (for
small N relative to dir size) lookups to a given directory fid use
RXAFS_Lookup, and after that we start doing bulk fetches.  While XCB
would partially mitigate these effects due to the observed high degree
of temporal locality of reference for dnlc entries, keep in mind that
XCB can't solve for the spatial locality of reference problem that ls
-l, graphical file browsers, etc. bring to the fore (hence why I think
we need to push for threshold-based fallback to directory fetch,
should we standardize a lookup interface).

-Tom