[OpenAFS] ADS communications issue?
Douglas E. Engert
Fri, 11 Sep 2009 14:56:24 -0500
Eric Chris Garrison wrote:
> Okay, we "solved" the ADS/MTU problem for one supercomputer... what it
> turned up was that our AFS servers were only able to do 1500 MTU, but the
> router they were on accepted jumbo (9000 MTU) frames and passed them on,
> so they fragmented. We had them set up our VLAN to not accept jumbo
> frames, and that supercomputer cluster was up and running fine.
> However, there's another supercomputer that had similar problems that were
> NOT solved by this. In fact, the problem there turns out to be
> fragmentation during aklog when talking to the AD servers, not when
> talking to our AFS servers.
> The traceroute shows that the DF (do not fragment) flag is set, and a
> packet of 2441 was being sent, which is bigger than 1500. It's
> fragmenting somewhere closer to the ADS servers, which are themselves set
> to 1500 MTU, according to their admins.
> So is the DF flag necessary? If not, how can we change that?
Look at the krb5.conf parameter:
udp_preference_limit = 1
This would say for any packets less then 1 byte, use TCP instead.
SO no fragmentation.
> I'm suspecting the solution's in the network, not in AFS, however. We
> just have to convince our network engineers (for the three networks the
> packets cross) to believe that. :)
> As an aside, the AD admins answered a question of mine about making the
> ADS tickets smaller as suggested here... by changing the flag immediately,
> rather than just saying whether they could or could not. When they did
> so, it made getting new tokens impossible, aklog saying that the key
> number didn't match. The AD admins were puzzled, since they didn't think
> this would generate a new key/kvno, but when they unchecked the box,
> everything was okay again.
> We'll have to test that before we try it again. If they're correct, and
> flipping that switch doesn't generate a new key, why does it break aklog?
> Would a server restart be necessary? Would I need a new keytab generated
> to update the KeyFile with asetkey?
Douglas E. Engert <DEEngert@anl.gov>
Argonne National Laboratory
9700 South Cass Avenue
Argonne, Illinois 60439