[OpenAFS] Why is speed of AFS loopback adapter set to 10 Mb, even if physical interface is Gb capable?

[Jeffrey Altman]

This is a cryptographically signed message in MIME format.

--------------ms030606090502050201000107
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 3/26/2010 2:22 PM, Holger Rauch wrote:
> Hi everybody,
>=20
> I installed OpenAFS for Windows 1.572 on a Windows 7 Professional (64
> bit) system with all system updates applied in conjunction with
> Kerberos for Windows (KfW) and the Network Identity Manager. I
> downloaded all packages from Secure Endpoints' web site. The PC is
> consists of current HW (Intel Core2Duo, 4 to 8 GB RAM, etc.).

1.5.74 is the current release.

> Unfortunately, the maximum transfer speed on Windows is about 8-10
> MB/sec when I copy local files to an OpenAFS volume.=20

This is common when encryption is in use, jumbo grams are disabled, and
the file server is running with its defaults.

> All involved
> network components are gigabit capable and I don't experience this
> problem when doing file transfers to a native ext3 filesystem using
> scp, for example. (I consider this comparable since the transfers are
> encrypted as well, in case of SSH/SCP even with a much stronger
> encryption algorithm).

The performance of a secure transfer protocol is very roughly comparable
to the cost of the encryption algorithm itself (fcrypt, DES, RC4, AES,
etc.), the encryption mode, and the number of encryption operations that
must be performed (size of the chunks).  For AFS without jumbograms the
size of the data portion of a packet is under 1444 bytes per encryption
operation.  For something like SSH/SCP which is a tcp/ip stream protocol
instead of packet based, the number of operations are significantly small=
er.

The AFS Rx fcrypt also does not lend itself particularly well to
pipeline operations.

Disabling encryption of the data will result in significant performance
improvements and the cost of sending your data in the clear.

> The server is a Debian Lenny system running OpenAFS 1.4.11 obtained
> from the Debian backports repository. The system is a QNAP TS-809 Pro
> with an external HD for the OS. The QNAP HDs are Seagate 7200k RPM
> drives of the Enterprise series used with SW RAID 5, totalling 6,4 TB
> in capacity. OpenAFS volumes reside on /vicep partitions which in turn
> reside on Logical Volumes (LVs). (We're only 25 users, so this scheme
> works perfectly well and has the advantage that the size of the
> underlying LV implicitly determines the quota, so I don't have to
> worry about setting OpenAFS quotas).
>=20
> The questions are thus:
>=20
> - Can I change the speed of the loopback adapter on Windows 7? If so, h=
ow?

The Microsoft loopback adapter has no speed (it is virtual and doesn't
report one.  When an adapter does not report a link speed it is reported
as 10Mbit.  This has no impact.

> - Why does the loopback adapter's speed default to 10 Mb instead of
>   the speed of the physical interface (Gb) at all?

The loopback adapter is not associated with the physical interface.  All
AFS Rx traffic to the file server is sent over the physical interfaces.
 The loopback adapter is used as a method of binding a Netbios name
"AFS" that is visible only to the local machine.  If the name was
published on a physical adapter, there would be no mechanism for
providing a common UNC name on all machines.

> - What's a "normal" transfer speed for OpenAFS when run in gigabit
>   network environemnts?

On 64-bit Win7, the SMB-to-AFS gateway is limited to approximately
65MB/second having nothing to do with the AFS Cache Manager to File
Server interface speed.

The AFS Redirector on 64-bit Win7 is currently producing sustained read
performance from the cache manager of 380MB/sec.  The AFS Redirector
implementation is not publicly available as yet.  When the 1.7 branch is
ready for code submissions the AFS Redirector code base will be
integrated and test releases will be made available.

The maximum throughput of an Rx connection (without encryption) on a
network that supports an MTU size of 9000 octets and clients and file
servers configured to use jumbograms is somewhere between 260MB/sec and
280MB/sec.

---

In reading the rest of the thread, there appears to be a mixture of
reports describing Windows and Unix cache manager performance numbers.
In this e-mail you are asking specifically about Windows.  I'm not sure
that the discussion of tuning Unix cache managers is of any value to you.=


On Windows, there is one type of cache.  It is a memory mapped paging fil=
e.

As others have mentioned, accessing lots of small files versus a small
number of large files will also result is lower bandwidth numbers.  This
is especially true on Windows because properly implementation of file
sharing operations requires that file locks be obtained for each and
every file open.  This overhead will not be present in SSH/SCP.

Jeffrey Altman



--------------ms030606090502050201000107
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJeTCC
AxcwggKAoAMCAQICEAMF9RTCGOz151fTpHLih+cwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE
BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT
I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA5MDgyODA0MDExOVoX
DTEwMDgyODA0MDExOVowczEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVy
aWMxHDAaBgNVBAMTE0plZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQDZNscYIvF6xzGSAfa/QUIqiElyn0EUxL2b86eKiYqe91bj0gLr/MJoErLnb+OmokxqSAH6
y0zlFqSbiFwgNM8m69K6m/6YO+x3+5zBc+u6snwTWMEWygnhx3rQ/lMhoQOgArraL+/k9aWL
kNdaXQKk6EZVW9pfV2A4Lk4DoZGFjY8tJRWWDLlFkYnxDuIEpLYwJpwakv3QHOaq/G8KW0iE
jVhVzPobuZzwD2tuepY/bsClwqxz/gfAEpUvAn/lYTqnoT7RYljZlCIdbrgcG/HSYMxAy1Zp
Yh8Fx+9cqsG8O4nqo26SVfYZvrYhh8m6OqW8Vakdt7vBLCTa/QhIdJ4hAgMBAAGjOTA3MCcG
A1UdEQQgMB6BHGphbHRtYW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADAN
BgkqhkiG9w0BAQUFAAOBgQBvbvJNXUJ4atv1CExIe0J38jZqoEUTttkXOfCDT9e3mSmVboOK
ifHDyLZQC4qSsCUfP7vdwAXjKtjak22HbfX2sEKCUgtnOkxRqXMM2V/NW/ESNVQZF0TO7L/Z
cW3icObO9FIZCSmgFMt2Al7VPfMQmaJNlqu9SLmXSwbRFJ5b4zCCAxcwggKAoAMCAQICEAMF
9RTCGOz151fTpHLih+cwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoT
HFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25h
bCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA5MDgyODA0MDExOVoXDTEwMDgyODA0MDExOVow
czEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVyaWMxHDAaBgNVBAMTE0pl
ZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRtYW5Ac2VjdXJlLWVuZHBv
aW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDZNscYIvF6xzGSAfa/
QUIqiElyn0EUxL2b86eKiYqe91bj0gLr/MJoErLnb+OmokxqSAH6y0zlFqSbiFwgNM8m69K6
m/6YO+x3+5zBc+u6snwTWMEWygnhx3rQ/lMhoQOgArraL+/k9aWLkNdaXQKk6EZVW9pfV2A4
Lk4DoZGFjY8tJRWWDLlFkYnxDuIEpLYwJpwakv3QHOaq/G8KW0iEjVhVzPobuZzwD2tuepY/
bsClwqxz/gfAEpUvAn/lYTqnoT7RYljZlCIdbrgcG/HSYMxAy1ZpYh8Fx+9cqsG8O4nqo26S
VfYZvrYhh8m6OqW8Vakdt7vBLCTa/QhIdJ4hAgMBAAGjOTA3MCcGA1UdEQQgMB6BHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQUFAAOB
gQBvbvJNXUJ4atv1CExIe0J38jZqoEUTttkXOfCDT9e3mSmVboOKifHDyLZQC4qSsCUfP7vd
wAXjKtjak22HbfX2sEKCUgtnOkxRqXMM2V/NW/ESNVQZF0TO7L/ZcW3icObO9FIZCSmgFMt2
Al7VPfMQmaJNlqu9SLmXSwbRFJ5b4zCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAw
gdExCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUg
VG93bjEaMBgGA1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRp
b24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFp
bCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0w
MzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU
aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg
RnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV
+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfAr
hVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l0cUqVIUPSAR/
p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMGA1UdHwQ8
MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVtYWls
Q0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh
YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/
TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amc
OY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggNxMIID
bQIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5
KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ
AwX1FMIY7PXnV9OkcuKH5zAJBgUrDgMCGgUAoIIB0DAYBgkqhkiG9w0BCQMxCwYJKoZIhvcN
AQcBMBwGCSqGSIb3DQEJBTEPFw0xMDA0MjYwMTQyNDlaMCMGCSqGSIb3DQEJBDEWBBTUnuh4
RT0GSwDm0yLLCmtTgf7mJDBfBgkqhkiG9w0BCQ8xUjBQMAsGCWCGSAFlAwQBAjAKBggqhkiG
9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcN
AwICASgwgYUGCSsGAQQBgjcQBDF4MHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0
ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVl
bWFpbCBJc3N1aW5nIENBAhADBfUUwhjs9edX06Ry4ofnMIGHBgsqhkiG9w0BCRACCzF4oHYw
YjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4x
LDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhADBfUUwhjs
9edX06Ry4ofnMA0GCSqGSIb3DQEBAQUABIIBAFlg9sfI1JDrpSkp6LvS0VDiCbMXXmeY7It9
qpGOQ7FoUqk01hLWD/6s476aDqoRpx4COGBVbMTmWFhknoCwAFbnlmTY3rhVV+4Vdxus7vY/
3s3/p2TRI7UG68TW+t4Jrj9ZOKDV4fFjr862vMKS/xN96qIluubrXfdnLFOYivwR8FRkjQYi
DXJSpcrwYBYi9mZHXM6ZLFn+Wq0AghcU+nLFN6yCQu2GPHPE37B1OUzSOhBk23l6wAtROW8V
/Jw6iZzZvoD5gE/H2hD77DIk4N70eVyV8nBrlRyZ6zpLIXwWXjlXm4i75wI1mWG+w0P0RTKY
qK0R2cTD+BIlC0nXSHIAAAAAAAA=
--------------ms030606090502050201000107--