[AFS3-std] rpc refresh: FetchDirectory: discussion only

[Matt W. Benjamin]

----- "Tom Keiser" <tkeiser@sinenomine.net> wrote:

> On Thu, Sep 10, 2009 at 7:00 PM, Matt W. Benjamin<matt@linuxbox.com>
> >
> > 4. rpc should permit a paging notion--I'm proposing that can be just
> be an offset into the list the server is producing at the current dv
> on DirFid--client can keep advancing by Entries_len until it's read
> all it wants
> >
> > 5. directory entries cannot be looked up, as server doesn't know
> applicable normalization rules
> >
> > 6. directory entries are not returned in some sorted order (with
> accompanying complexity), unless someone can really make a strong case
> to change precedent (jhutz)
> >
> 
> Let me tackle 4 and 6 together, as I think they're intertwined.  Both
> of these points seem to implicitly assume the current fileserver
> implementation.  I can easily envision a future fileserver that
> stores
> the directory object as, say, a B-tree.  In that case, returning a
> sorted subset is rather trivial, and addressing a subset by an
> ordinal
> rather awkward, as you're effectively trying to linearize the
> structure by something other than its natural sort order.

Yes.  It's definitely awkward in that respect.

> Furthermore, the proposed XDR proc requests a subset by an ephemeral
> ordinal without specifying the DV which was used to come up with said
> ordinal.  Future journalled/logged file servers would have
> insufficient information to compute the mapping of that (DV,ord) onto
> the subset of the directory object addressed by (DV',ord') that you
> really "wanted".

Ok.

> 
> I think the proc needs more fields.  For example, I'd like an OUT
> parameter for the server to communicate whether the data is sorted or
> unsorted (really, an enumeration in case we want to do something more
> advanced in future, e.g. send it in an order that allows for linear
> time to build the split stream data into an efficient structure).

Clearly the stream needs to be in some format or formats.  I asked Marcus for
some feedback on the idea of xdr-encoding sequences of entries, and sending 
piecewise on the stream. as one format.  Sorted formats and Jeff Altman's idea
to support client's natural byteorder (can't be xdr?) would be helpful to discuss.

> Additionally, I really think we should assert our current DV as an IN
> param to allow future implementations to do offset mapping should
> they
> so desire (and offset should become INOUT as a result).

Ok.  I assumed the client might abort the listing if it discovered that dv
had changed, we'll need a way for the server to inform the client it can continue
a coherent listing, I think.

> 
> As far as the addressing by ordinal issue, my preference would be for
> the "primary key" to be a discriminated union.  For the current
> generation of file servers, an ordinal makes perfect sense.  However,
> I think we should specify a second union entry which is a filename
> string.  Upon receipt, file servers supporting this lookup mechanism
> would return the block of entries which follow said entry.  This
> would, of course, require allocation of new error codes and
> capabilities to announce which lookup mechanism(s) the server
> supported.

How does this relate to your own prior comment--you're thinking of it being
only a placeholder for now?

> 
> Thoughts?
> 
> -Tom

-- 

Matt Benjamin

The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309