[OpenAFS-devel] datagrams really arent big enough?

[Default]

> i am not really jumping to conclusions.  i have run my modified version of
> 3.6 in a production environment for over a year w/o seeing any problems.
> RX_MAX_FRAGS needs to be bigger than 4!  yes, this might use a little more
> memory on your server and clients but if you are using larger mtu's i should
> think that you expect this behavior.  as for one of your previous statements:

Have you run it with this configuration?

server --- ethernet ---- router ---- ethernet ----- client 

or 

server === ATM === router --- internet --- router === ATM === client

especially with several different kinds of routers, and some decent load on 
the router.

That's the one I'm worried about, and now I remember why.  It has to do with 
the way routers are designed.  Cisco routers (used to, anyway) have queues 
that were only about four packets deep.  So if you tried to send a jumbogram > 
four frags, it would overflow the queue and one frag would get dropped.  Rx 
will dutifully resend the jumbogram, IP will fragment, blast out the frags, 
overrun the router's input queue, and so on -- forever.  Now, my information 
about Cisco routers is a few years old, so they may have deeper queues already.
 
> >4. kicking up the max packet size also increases the max receive window, I 
> >think, increases the memory requirements of the server, and increases the 
> >amount of data that must be resent in the event of a packet loss, so you might 
> >(probably not, but maybe) have actually reduced performance slightly.
> 
> using a larger packet wont increase the max recv window size.  that's a 
> different issue entirely.  additionally, since rx has been improved lately
> (with congestion control, slow start, etc)  it should be able to throttle
> back in the case that packets are lost.

Hm.  Ok.  I'm not convinced by *should*, though.  Even with the improved 
congestion control, once a RX packet's size is chosen, that packet's size
can't be reduced on subsequent resends, if I'm reading this code correctly.

> and yes, this does improve performance only slightly (and only if you
> have an mtu size greater than ~5k) since the bottlenecks tend to be
> elsewhere but its a step forward.  RX_MAX_FRAGS need to be 11, which is
> how many frags will fit into MAX_RX_PACKET_SIZE

Do you really want IP-over-ATM jumbograms of 16KB each?  That would equal one 
9188 byte ip datagram and one ~7000 byte datagram for each packet.  It would 
save some protocol processing overhead, as IP reassembly is cheaper than RX, 
but as far as bandwidth efficiency goes, it would be better to stick with 
RX_MAX_FRAGS equal to 6.