[OpenAFS-devel] datagrams really arent big enough?

[Chas Williams]

Subject: jumbo datagrams really arent jumbo enough?

during some other testing a while ago (cira afs 3.5) i noticed a problem
with datagram size.  my packet size always seemed to be limited to 5688
while the mtu size of our network is 9218 (yes, it atm)

i believe i have tracked it down to the following:

/* Given an interface MTU size, and the peer's advertised max receive
 * size, calculate an adjisted maxMTU size that makes efficient use
 * of our packet buffers when we are sending AFS 3.4a jumbograms. */
int rxi_AdjustMaxMTU(int mtu, int peerMaxMTU)
{
    int maxMTU = mtu * rxi_nSendFrags;
    maxMTU = MIN(maxMTU, peerMaxMTU);
    return rxi_AdjustIfMTU(maxMTU);
}

maxMTU gets limited by rxi_nSendFrags which is defined to be:

rx_globals.h:#define RX_MAX_FRAGS 4
rx_globals.h:EXT int rxi_nSendFrags INIT(RX_MAX_FRAGS)

and in rx.c:rxi_ReceiveAckPacket we can see:

      /* Get the maximum packet size to send to this peer */
      rx_packetread(np, rx_AckDataSize(ap->nAcks), sizeof(afs_int32),
                    &tSize);
      tSize = (afs_uint32)ntohl(tSize);
      tSize = (afs_uint32)MIN(tSize, rx_MyMaxSendSize);
      tSize = rxi_AdjustMaxMTU(peer->natMTU, tSize);

natMTU is 1444 (i guess this is the 'natural mtu size of an rx packet'),
which means the max tSize will ever be is 5776--signicantly less
than my interface mtu size.

at the time i increased RX_MAX_FRAGS to 6 to get afs to fill out the
send/recv buffers.  its what has been running on our afs servers for
a while now (over a year) and i havent seen a problem.  of course only
the afs clients with the 'fixed' code use the bigger packets.  possibly
tSize shouldnt be computed using the peer's natMTU? since RX_MAX_PACKET_SIZE
is 16384, i would think RX_MAX_FRAGS would need to be a bit larger?