[OpenAFS-devel] Progress on Linux in-kernel RxRPC library

[Kyle Moffett]

[I've rearranged your points to better flow with my response]

On Mar 22, 2005, at 20:13, Jeffrey Hutzelman wrote:
> So here comes the fun part...
> Just as with other transports, the Rx sockets interface should
> not allow two distinct processes to bind the same port(*).
>
> (*) There is one exception: for full generality, it might be
> desirable to allow two servers running in different processes
> to bind to different Rx services running on the same port.  If
> that is done, then we'd have to come up with a rule to
> determine when that is allow, and who (if anyone) gets to make
> calls from that port.  There is nothing in AFS today that
> requires this capability.

How about a new socket option "SO_RXRPC_SPLITPORT" or similar,
such that if the first process to bind to a port and service
specifies that option, then each additional process that also
uses the option it can bind to other unused services on the
same UDP port.  Such a mutual sharing agreement would work for
all services.  We should also add an additional option:
"SO_RXRPC_REUSESERVICE" or similar, that, when the "SPLITPORT"
option is set, allows a process to immediately reuse the same
service number.  SO_RXRPC_SPLITPORT might require SO_REUSEPORT
as well, but it should make it possible for user processes to
specify the reuse and sharing policies in a sane way.

> Calls made via a socket bound to a specific port must actually
> originate from that port, so it stands to reason that calls
> made via two different sockets bound to different ports will
> not share connections.

Of course.  This fixes most of the basic case.

> So the problem is _almost_ solved.  It just requires one
> additional constraint, which is that a call made via an
> unbound socket must not share a connection with a call made
> via a bound socket.

Or perhaps we should add another option "SO_RXRPC_SHARECONN"
for outgoing calls that overrides the above constraint when
all processes involved set it.

> Obviously, calls made via different unbound sockets can share
> connections, provided the credentials are the same.  And,
> calls made via sockets bound to the same port can also share
> connections.  Since the cache manager will bind its port, no
> user-mode process will be able to share connections with it,
> and everything works out.

And if there is ever a revised AFS protocol that allows such
connection sharing, then the involved processes can specify
the corresponding socket options to override the defaults.

> It's not so much by connection as by endpoint, as I described
> above.  The key point is that calls being made via the same
> connection are assumed to be from the same process, which I
> think is a reasonable assumption.

Ah, agreed.  It would be nice to allow the sharing if agreed on
by all processes, though, which is why I specified the above
socket options.

>> Well, other models are:
>> Create a thread to handle a specific connection from a
>> specific user, including all the RPCs they ever need to
>> make. Or, create a bunch of worker threads beforehand and
>> just have them switch keys whenever they need to change
>> which user they act as.
>
> Yeah.  I dunno; forcing what I see as a parameter into what is
> essentially (thread-)global state just seems wrong to me.

David Howells and I have been discussing whether it would be
better to specify it per-connection with setsockopt or via a
search in the keyrings, and while it might be more powerful to
allow direct specification, it doesn't match as well with the
idea of a set of "credentials" with which all accesses are
done.

On the other hand, one idea I was just thinking about is to
allow the user to specify credentials with a setsockopt option
such that a key id of -1 indicates that it should perform a
keyring search.  Ideally we should try to come up with the
cleanest and most expressive API/ABI possible, because anything
we get merged with _either_ codebase is not likely to be very
changeable once applied, due to binary compatibility concerns.
(Unless of course you're talking about kernel-internal ABI,
which changes whenever somebody thinks they can make it work
better, which is quite frequently).  Maybe in a couple weeks
or so I'll have a patch for you to look at.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r  
!y?(-)
------END GEEK CODE BLOCK------