[AFS3-std] Request for a capability bit for RxOSD

[Felix Frank]

Jeffrey Hutzelman wrote (Wed Jun 17 2009 17:00:02 GMT+0200 (CEST))
> --On Monday, June 15, 2009 04:26:21 PM +0200 Felix Frank 
> <Felix.Frank@Desy.de> wrote:
> 
>> For OpenAFS+OSD, it would be helpful for the clients to identify OSD
>> capable fileservers early. Codepaths can then be separated more easily
>> for OSD access vs. normal fileserver access.
>>
>> Similarly, fileservers can store a hint about clients that are OSD
>> capable, and send OSD-related data only to those. Other clients will (as
>> is the case already) be forwarded actual file data instead of being
>> supplied with OSD metadata.
> 
> So really, you're asking for two capability bits -- one to advertise an 
> OSD server, and one to advertise that a client is OSD-capable.  I'm not 
> sure I understand the purpose of either.

Yes.

> Why does a client need to know that a particular fileserver is an OSD 
> server?  AFAIK the only behavioral difference is when accessing data of 
> vnodes which are _not_ directly resident on the server.  Knowing the 
> server is OSD doesn't save you from fetching vnode status, and doesn't 
> save you from the possibility that you might have to fetch data from the 
> server.  It does mean you might be able to use OSD access to reach some 
> vnodes, but that's going to be on a per-vnode basis, and it's going to 
> involve making different RPC's, so I don't see what a server-level "this 
> server might contain OSD vnodes" bit buys you.

All of the above is factually correct. The one practical advantage that 
I can really think of is that OpenAFS clients could use the absence of 
this bit to insulate themselves against servers that use spare fields 
(yet to be approved for RxOSD too, of course) for other purposes.
Local patches could even continue interpreting those fields differently 
inside codepaths that are only run if the server is known not to be OSD 
capable.

> Similarly, why does a server need to know a client is OSD-capable?  
> Servers don't decide whether to "forward actual file data" vs sending 
> OSD metadata; the OSD metadata is provided via FetchStatus and/or 
> another RPC, and calls to RXAFS_FetchData should still result in real 
> file data.  So the only important thing here is what RPC is being 
> called; a client that asks for OSD metadata can be assumed to want that, 
> and a client that calls RXAFS_FetchData can be assumed to want the data 
> that it asked for.

Same as above essentially - all true (as far as I can tell). But this 
would allow OpenAFS to prevent fileservers from using formerly spare 
fields in the first place unless the client specifically declares 
knowledge about their semantics. Again, this will prevent clients that 
use incompatible (and non-standard) enhancements to function even with 
OpenAFS+OSD servers that do use, say, SyncCounter (just speaking 
hypothetically now, I realize that's a separate discussion).

> I'm not opposed to adding capability bits for this, but first I want to 
> be clear on exactly what they're for and what the semantics are. 
> Particularly...

 From the top of my head:

> - What do you have to do/support in order to advertise the bit?

Server must implement a couple of essential OSD-related RPCs.
Clients must be able to interpret the specific protocol fields.

> - How are peers expected to behave when they see the bit?

Server behaves "normally" in terms of OSD.
Client makes an OsdPolicy-RPC prior to writing, to determine wether to 
use OSD or not.
(I may have some of that wrong.)

> - How are peers expected to behave when they do not see the bit?

Server does not use OSD-specific protocol fields.
No restrictions for the client.

> - What is the expected behavior when talking to an old peer that doesn't 
> understand the bit?

Advertising the bit implies no assumptions about what either client nor 
server is supposed to send/return. I'm not sure I understand the question.

> - When can the bit change?

Fileservers cannot loose or gain the capability, but on the same 
machine, a different binary with different capabilities can be started, 
of course.
Clients can be configured (not) to use OSD using an fs command.

> - What happens if a peer which supports the new feature fails to notice 
> the bit?

If the server won't detect the capability it will treat the client as 
non-OSD-aware.
What the client does is a matter of design - given my statements above 
however, it would be cleanest to disallow a client to treat a server 
that is not known to have the capability as an OSD-fileserver. That way, 
the client won't mistake messages from incompatibly patched servers for 
OSD related data.

> - Are there any security implications, particularly when capabilities 
> are unauthenticated?

I have no response to this. I don't see how such information can impact 
security at all.

Bottom line is that OSD does and will function without any capability 
bit. What I hope to achieve is greater acceptance by potential users, 
smaller reluctance from the AFS community to approve use of 
reserved/unused protocol fields and for OpenAFS, more thorough code 
segregation of RxOSD (again, in the interest of user acceptance).

Regards
  - Felix