[OpenAFS-devel] Bad AFS performance over wide area due to packet fragmentation problems

[Jeffrey Altman]

This is a cryptographically signed message in MIME format.

--------------ms080708030702090302080100
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Rainer Toebbicke wrote:
> It's not the first time that people at external sites complain about bad
> AFS performance from our servers.
> 
> Usually, it boils down to that every now and then one of the routers or
> firewalls in the external networking clouds starts to choke on IP packet
> fragmentation - when we repeat the tests with "-nojumbo" everything
> works like a charm. The problem is that every time its the users who
> detect this, it takes some time until the problem surfaces at the
> sysadmin level, and the networking guys swear that there wasn't a change
> since ages.
> 
> As '-nojumbo' has a measurable price on our own local network where
> fragmentation does not exhibit any problem we hesitate to run everything
> in that mode.
> 
> Ideally, RX would be adaptive and stop using jumbograms when they cause
> problems, but I understand that the algorithm to detect those reliably
> can be challenging.
> 
> How about a config file a la NetRestrict then that turns off jumbograms
> to external sites, or allows them to others?
> 
> Any brilliant ideas?

Rainer:

There is a bug in Rx related to -nojumbo that Derrick identified
yesterday and for which we are testing a patch.  There are three
instances of rxi_AdjustDgramPackets() which instead of being passed
the constant RX_MAX_FRAGS (4) should instead be passed 'rxi_nSendFrags'
which is set to 1 when -nojumbo is enabled.   By always setting this
value to RX_MAX_FRAGS, the sender always attempts to send a jumbogram
even when that entity explicitly has said it does not want it.

The patch can be found at:

/afs/andrew.cmu.edu/usr/shadow/rx-nojumbo-really.diff

As another work around, the client can set the RxMaxMTU size to 1431.
This value is small enough to disable the sending of jumbo grams.

Prior to 1.5.34 the Windows client set RxMaxMTU size to 1260 which
in effect disabled the use of jumbograms on both the clients and
the servers.  This was done to permit OAFW to work with the Cisco
VPN 4.x client.  I removed this in 1.5.34 because it the Cisco VPN
5.x client had been widely deployed and does not suffer from the
inability to send IP fragments.

Unfortunately, there are a large number of other network devices
that also have trouble transmitting IP fragments.  Removing the
RxMaxMTU restriction enabled the OAFW client to once again start
sending jumbograms even though the RxNoJumbo setting is enabled.  The
above patch fixes this problem.

Jeffrey Altman

--------------ms080708030702090302080100
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJeTCC
AxcwggKAoAMCAQICEDsE+kRcmomW1hYG6BoqhGEwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE
BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT
I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDUzMDE5MTUyOVoX
DTA5MDUzMDE5MTUyOVowczEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVy
aWMxHDAaBgNVBAMTE0plZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQCtf5bVJdYFtHIrV2XALpA5oaMu7FPYU7RP7vJhd8Cu9Kd9ud2crX2pHK4avuPaYb4Vg9qI
zPrPadePhJ3OWwNt1ZlUlpc5URnOfpg/I9iymZBUSnCFVLuIvoncacqyUlzqdYEF8XGEoEL6
6bj8uoCSX0D7ZjZiAS8993NvgiPYpf10acMyWQ4max+P7Wg9T03Nw2F6EsmP6gWxBRsekTXe
N6QjJdvaK0846lDqeBFoCEzIUMQXj2kiXVPCPEdxPc/L1sDMYf0GLaDIg8qyThpGd0X6DwfK
3RWcMy8DV7Q5Z+jSEdPn5X0l4anOTrjr3IwE57MC3bVs0EEpUODTzftnAgMBAAGjOTA3MCcG
A1UdEQQgMB6BHGphbHRtYW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADAN
BgkqhkiG9w0BAQUFAAOBgQA9kndmeLrdQOUbhNGGms/FnfDyraH4OjA4PIIMOCbGWK0YXczs
/Fqn4XkT70SG4s8v4Zg6TaAcJrZBVcZQXyzrhlF2Zev/g69zZMHQe+2r4i/3FBVKAtFCoea1
vgwJ5TfZYlKvt4D0Z4zexu9Y0VwCIR4plWjVD76zC2CGB/2fhjCCAxcwggKAoAMCAQICEDsE
+kRcmomW1hYG6BoqhGEwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoT
HFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25h
bCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA4MDUzMDE5MTUyOVoXDTA5MDUzMDE5MTUyOVow
czEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVyaWMxHDAaBgNVBAMTE0pl
ZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRtYW5Ac2VjdXJlLWVuZHBv
aW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCtf5bVJdYFtHIrV2XA
LpA5oaMu7FPYU7RP7vJhd8Cu9Kd9ud2crX2pHK4avuPaYb4Vg9qIzPrPadePhJ3OWwNt1ZlU
lpc5URnOfpg/I9iymZBUSnCFVLuIvoncacqyUlzqdYEF8XGEoEL66bj8uoCSX0D7ZjZiAS89
93NvgiPYpf10acMyWQ4max+P7Wg9T03Nw2F6EsmP6gWxBRsekTXeN6QjJdvaK0846lDqeBFo
CEzIUMQXj2kiXVPCPEdxPc/L1sDMYf0GLaDIg8qyThpGd0X6DwfK3RWcMy8DV7Q5Z+jSEdPn
5X0l4anOTrjr3IwE57MC3bVs0EEpUODTzftnAgMBAAGjOTA3MCcGA1UdEQQgMB6BHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUFAAOB
gQA9kndmeLrdQOUbhNGGms/FnfDyraH4OjA4PIIMOCbGWK0YXczs/Fqn4XkT70SG4s8v4Zg6
TaAcJrZBVcZQXyzrhlF2Zev/g69zZMHQe+2r4i/3FBVKAtFCoea1vgwJ5TfZYlKvt4D0Z4ze
xu9Y0VwCIR4plWjVD76zC2CGB/2fhjCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAw
gdExCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUg
VG93bjEaMBgGA1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRp
b24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFp
bCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0w
MzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU
aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg
RnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV
+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfAr
hVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l0cUqVIUPSAR/
p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMGA1UdHwQ8
MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVtYWls
Q0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh
YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/
TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amc
OY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggNkMIID
YAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5
KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ
OwT6RFyaiZbWFgboGiqEYTAJBgUrDgMCGgUAoIIBwzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcN
AQcBMBwGCSqGSIb3DQEJBTEPFw0wODA4MjExMjQ3NDNaMCMGCSqGSIb3DQEJBDEWBBTg8e1z
sugdM/O5eCBJGsGpBjS09zBSBgkqhkiG9w0BCQ8xRTBDMAoGCCqGSIb3DQMHMA4GCCqGSIb3
DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBhQYJKwYB
BAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcg
KFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3Vpbmcg
Q0ECEDsE+kRcmomW1hYG6BoqhGEwgYcGCyqGSIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJa
QTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhh
d3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEDsE+kRcmomW1hYG6BoqhGEwDQYJ
KoZIhvcNAQEBBQAEggEAnzUAA07+l5Hf3kPlLMGDnL2TgittefBx5nA5szm5MLhpXR8JENmC
0IXqT6NWjafT5Up57OJcu0b6wm7cybbhsKLuYkRDI2KPc/aZdz287Zu1auo+onE9Fjrn8dFQ
RrklWIcVjZTOE1k4plUI8DpLpU3TAl9hrjOzkvZAt28xM7EoQYzKoTibvwU5xvRkjQee8Ddd
nHP6ATtvdA56/dU3tqTOaGV0Le+LIxcR/IJoJmigyo/bh+qPZRQ/npwFoG5WZMyhtrC/TehR
UgwdGRXZcNWIw2cqZcmhAjxtcIlNM2/Hp269C403a2N/JK9rnDlEV01LQoCYFyOwhhWbDuix
oQAAAAAAAA==
--------------ms080708030702090302080100--