[OpenAFS-devel] RX_MAX_FRAGS (yet again)

[Nickolai Zeldovich]

> i believe RX_MAX_FRAGS is the number of 'datagrams' to put in a 
> jumbogram (a jumbogram just contains a bunch of 'regular' datagrams)

Are you thinking of RX_MAX_DGRAM_PACKETS?  The comment in rx_globals.h
seems to suggest that is the case:

  #define RX_MAX_DGRAM_PACKETS 6 /* max packets per jumbogram */
  EXT int rxi_nDgramPackets INIT(RX_MAX_DGRAM_PACKETS);

rxi_nDgramPackets is also used to cap peer->maxDgramPackets, which the
number of packets that rxi_SendXmitList() batches into a list before
passing it to rxi_SendList(), which makes the jumbogram.

On the other hand, RX_MAX_FRAGS seems to be only used to compute maxMTU
from ifMTU in rx_kcommon.c and a few other places.

> i really dont think that this is the case with the rx code as is.
> just checking with rxdebug, one of the peers that doesnt have an active
> ethernet (it has an ethernet, its never configed up) shows:
> 
> Peer at host 134.207.10.21, port 7001
>         ifMTU 8524      natMTU 1444     maxMTU 5692

Hmm, the problem seems to be that the peer is advertisting a natMTU of
1444 and a maxMTU of 5692; rxdebug'ing 134.207.10.21 and 134.207.10.8
respectively shows:

Peer at host 134.207.10.69, port 7000
        ifMTU 1444      natMTU 1444     maxMTU 5692
Peer at host 134.207.10.69, port 7000
        ifMTU 1444      natMTU 1444     maxMTU 5692

so for some reason it's deciding that its local interface MTU is only
1500 bytes..

Interestingly, ADAPT_MTU doesn't seem to be enabled for Solaris kernel
code (at the top of rx_misc.h), which causes rxi_InitPeerParams() to
default to an ifMTU of 1444.  Could you instrument rxi_InitPeerParams
to see why ifMTU isn't being set to 9K even on Linux with ADAPT_MTU?
Don't have any ATM cards to test this myself. :-)

> rx_maxReceiveSize could grow to beyond ~4*1444, but the rx code seems
> to take the min of max_packet_size or maxmtu and maxmtu is constructed
> from rxi_nRecvFrags which is initialized to RX_MAX_FRAGS:
> 
> 	maxmtu = rxmtu * rxi_nRecvFrags + ((rxi_nRecvFrags-1) * UDP_HDR_SIZE);

Sure, but the problem here is that rxmtu is only 1444, whileas it should
be something closer to 9K.

(And yeah, raising RX_MAX_FRAGS probably solves the immediate problem by
giving you a larger maxMTU and thus rxi_AdjustDgramPackets lets you stuff
more Rx packets into a jumbogram, but there's still some bug w/ ifMTU.)

-- kolya