[OpenAFS-devel] RX Performance

[chas williams]

>Are the default values in RX for window size, ack rate etc. still adequate for
>today's networks? I tried to play around a bit: increased # of pcks/jumbogram,
>receive/transmit window size etc. (every time bluntly changing rx_globals.h 
>and rebuilding  libafsrpc.a). Nothing really happened. The only improvement we
>got was cranking up the MTU to around 9K (on a 10GB card that supported it).

i believe the fundamental problem with rx is this idea of packets inside
a jumbogram.  there has been some work on this see the citi papers that
have been mentioned on this list before.  rx spends time adding/removing
rx headers on the 'packets' inside the jumbogram.  this is costly.
further, it calls writev() to send the jumbogram to disk.  each one of
those writes inside the writev() is limited by the size of a 'standard' rx
datagram (1400 or so bytes).  you will never see decent disk performance
with such tiny write sizes.  i believe sending is handling similarly,
although typically the network layer recopies the iovec into a single
buffer and then sends it.  if you go through the archives, i believe i
sent some gperf's of the fileserver.  

we need new jumbograms that contain a single header and data.  this would
go a long way towards improving the rx protocol.