[OpenAFS] Evaluating AFS for in house use, RFCs...

[Jeffrey Altman]

This is a cryptographically signed message in MIME format.

--------------ms000600080005090706020508
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Mitch Collinsworth wrote:
> 
> On Fri, 3 Feb 2006, Jeffrey Altman wrote:
> 
>> I don't have hard numbers.  Its a simple reality.  If you are using an
>> AFS cache manager with anything close to a decent cache size, if you
>> need to access a file more than once, then the second time you access
>> it assuming it has not changed the reads will occur a local disk speed
>> instead of WAN speed.
> 
> That's a lot of if's.  If you're generating reams of data today and
> are only going to come back and analyze it tomorrow or next week then
> the cache is not necessarily any help.  If you're generating many GB
> of data then the cache still thrashes.  Remember the old UMich paper?

90% of users access the same small number of files.  Whether they be
application DLLs or data files.    Those files fit within a 1GB cache.

Now if you are using AFS to store generated data, then the cache isn't
going to help.  The question is:

  what is your usage model?

>>> The vast majority of users don't care about any of these operations.
>>> They just want to save data and retrieve it.  Or maybe it's save it
>>> and forget about it.  While installing the client might be a good
>>> thing, the user isn't going to buy it based on this logic.  If we had
>>> numbers to show how much faster it is, that they would buy.
>>
>> Then might I suggest you start running some performance tests.
> 
> That's probably a good idea.  My point is that the last time I saw
> numbers on this, AFS/RX did NOT handily beat Samba/SMB.  It was in
> fact the other way around.  Discovering that this has changed would
> be very helpful in convincing people to give AFS a try.

Using what for testing and in what environment?

If you were testing IBM AFS or OpenAFS 1.2 on Windows against the
built-in SMB client on a LAN then I would agree with you.   The Windows
AFS client from those days was so broken not only did performance suck
but it crashes extremely frequently.   The scary part is that there are
still so many people using it.  I get the crash reports that are sent to
Microsoft.  There are literally high hundreds of reported crashes a
month from those AFS clients on Windows.

Since 1.2 there have been significant performance improvements in the
Windows client which I documented in the periodic OpenAFS for Windows
Status Reports I publish.

OpenAFS 1.4 over OpenAFS 1.2:
  54% improvement in writes with crypt
  125% improvement in reads with crypt
  143% improvement in writes without crypt
  263% improvement in reads without crypt

OpenAFS 1.4 over IBM AFS 3.6 2.53:
  31% improvement in writes without crypt
  35% improvement in reads without crypt
  (IBM doesn't support crypt mode in the Windows client)

The threading of the RX library has been significantly improved in
OpenAFS 1.4 and a number of race conditions would have resulted in
performance degradation have been fixed.

The OAFW cache manager has been enhanced.  The cache is now stored
between AFS Client Service restarts.  Rodney Dyer spoke yesterday about
how he is running applications out of AFS on Windows.  One of those
applications being run entirely out of AFS is ProEngineer.  This
application has an 88MB foot print that must be loaded each time it
starts.   It also writes its configuration data to the user's redirected
profile directory which is stored in AFS.  The application starts in
approximately 80 seconds via AFS with an empty cache and in 20 seconds
when the cache is populated.   This is a big difference for end users
and it makes it possible to run applications such as these out of AFS.

I run Microsoft Office 2003 out of AFS.  I would never consider doing
that over SMB.  I had experience in a prior company of running
applications off of a Windows 2000 Server via SMB.   There was nothing
but complaints from users.   The SMB client does not cache and unless
you marked all of the EXE and DLL files so that Windows would copy
them entirely into the paging file, the constant re-reading of the
binary data from the file server made the use of the application
unbearable.  With AFS this is not a problem.

However, I go back to the earlier comment about the usage model.
If what you are doing is generating lots of data to write into file
system for archival storage, then the cache manager does not help
you and the default AFS chunk size will hurt as well.   The block
size determines how much data is written per StoreData RPC.  The default
chunk size for the Windows client in 1.2 and in IBM AFS was 4K.  In 1.4
it is now 128K.  However, if you are doing a lot of writing of large
quantities of data you will want to increase this to something much
larger.  The max chunk size is 1024MB.  You can set the default chunk
size as part of a transform you apply to the OpenAFS for Windows MSI.

Jeffrey Altman




--------------ms000600080005090706020508
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJXzCC
AwowggJzoAMCAQICAw7NrTANBgkqhkiG9w0BAQQFADBiMQswCQYDVQQGEwJaQTElMCMGA1UE
ChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv
bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EwHhcNMDUwNTI3MTc0NzU3WhcNMDYwNTI3MTc0NzU3
WjBzMQ8wDQYDVQQEEwZBbHRtYW4xFTATBgNVBCoTDEplZmZyZXkgRXJpYzEcMBoGA1UEAxMT
SmVmZnJleSBFcmljIEFsdG1hbjErMCkGCSqGSIb3DQEJARYcamFsdG1hbkBzZWN1cmUtZW5k
cG9pbnRzLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKjPyrF+rdjOUSK/
bWwZHdx5p1+y6iiCd4vvYEVDxouYFp5C/fZEWm5n45ubBUbMSUI1MAZN6ooEoH09UTj6BXhM
S8B987ls81dKOIUphTF2jOzq8gsFmeA15yHMRAD20LqUWeLyvYk8FCNQw+dsKMMhX+WdsxOm
RY/1jPkJL6oN8kEwoUFkOX9/OfWWh6oFnV6faiEHUKDMFubsb9X0KVD8iIeR7Cxz7i4kXqRX
wMlp2fyoxcDIJrBaTY8nA++g3p34IkWt1a5po6g683nIgSnGpwYIwuJheBqSEZfLYWa+1KdD
6Sn27Ud94GqUvPVG5jC6zVC5EJ2aWuoAu+nNuV8CAwEAAaM5MDcwJwYDVR0RBCAwHoEcamFs
dG1hbkBzZWN1cmUtZW5kcG9pbnRzLmNvbTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBAUA
A4GBADtvO//tjiAV6VJGtoNtrl34mB5jGyGTiotzw8riB6zz0GvY11bcWDmp6JKif+pVG+8L
IySDosbuva13qu2HwYUxBmWc7CoNd2k9kRlcrfbDUTTrGOZK8qyqNqT3gQZTAa9ZnUI0su9G
y/n2o5bQcaYdqR3htNrpvdLSPOWhILOXMIIDCjCCAnOgAwIBAgIDDs2tMA0GCSqGSIb3DQEB
BAUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBM
dGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTAeFw0w
NTA1MjcxNzQ3NTdaFw0wNjA1MjcxNzQ3NTdaMHMxDzANBgNVBAQTBkFsdG1hbjEVMBMGA1UE
KhMMSmVmZnJleSBFcmljMRwwGgYDVQQDExNKZWZmcmV5IEVyaWMgQWx0bWFuMSswKQYJKoZI
hvcNAQkBFhxqYWx0bWFuQHNlY3VyZS1lbmRwb2ludHMuY29tMIIBIjANBgkqhkiG9w0BAQEF
AAOCAQ8AMIIBCgKCAQEAqM/KsX6t2M5RIr9tbBkd3HmnX7LqKIJ3i+9gRUPGi5gWnkL99kRa
bmfjm5sFRsxJQjUwBk3qigSgfT1ROPoFeExLwH3zuWzzV0o4hSmFMXaM7OryCwWZ4DXnIcxE
APbQupRZ4vK9iTwUI1DD52wowyFf5Z2zE6ZFj/WM+Qkvqg3yQTChQWQ5f3859ZaHqgWdXp9q
IQdQoMwW5uxv1fQpUPyIh5HsLHPuLiRepFfAyWnZ/KjFwMgmsFpNjycD76DenfgiRa3Vrmmj
qDrzeciBKcanBgjC4mF4GpIRl8thZr7Up0PpKfbtR33gapS89UbmMLrNULkQnZpa6gC76c25
XwIDAQABozkwNzAnBgNVHREEIDAegRxqYWx0bWFuQHNlY3VyZS1lbmRwb2ludHMuY29tMAwG
A1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEEBQADgYEAO287/+2OIBXpUka2g22uXfiYHmMbIZOK
i3PDyuIHrPPQa9jXVtxYOanokqJ/6lUb7wsjJIOixu69rXeq7YfBhTEGZZzsKg13aT2RGVyt
9sNRNOsY5kryrKo2pPeBBlMBr1mdQjSy70bL+fajltBxph2pHeG02um90tI85aEgs5cwggM/
MIICqKADAgECAgENMA0GCSqGSIb3DQEBBQUAMIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMM
V2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBlIFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25z
dWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZpY2F0aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYD
VQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNv
bmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcNMDMwNzE3MDAwMDAwWhcNMTMwNzE2MjM1OTU5
WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRk
LjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EwgZ8wDQYJ
KoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVzVftOucqZWh5owHUEcJ3f6f+jHuy9zfVb8hp2
vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnwK4Vaqj9xVsuvPAsH5/EfkTYkKhPPK9Xzgnc9
A74r/rsYPge/QIACZNenprufZdHFKlSFD0gEf6e20TxhBEAeZBlyYLf7AgMBAAGjgZQwgZEw
EgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8EPDA6MDigNqA0hjJodHRwOi8vY3JsLnRoYXd0
ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFpbENBLmNybDALBgNVHQ8EBAMCAQYwKQYDVR0R
BCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVMYWJlbDItMTM4MA0GCSqGSIb3DQEBBQUAA4GB
AEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSdf0whuPg2H6otnzYvwPQcUCCTcDz9reFhYsPZ
Ohl+hLGZGwDFGguCdJ4lUJRix9sncVcljd2pnDmOjCBPZV+V2vf3h9bGCE6u9uo05RAaWzVN
d+NWIXiC3CEZNd4ksdMdRv9dX2VPMYIDOzCCAzcCAQEwaTBiMQswCQYDVQQGEwJaQTElMCMG
A1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBl
cnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECAw7NrTAJBgUrDgMCGgUAoIIBpzAYBgkqhkiG
9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0wNjAyMDMxNzUxMjJaMCMGCSqG
SIb3DQEJBDEWBBTPseTLIPGxnC9WOs8wmgoLDDHUmTBSBgkqhkiG9w0BCQ8xRTBDMAoGCCqG
SIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG
9w0DAgIBKDB4BgkrBgEEAYI3EAQxazBpMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3
dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJl
ZW1haWwgSXNzdWluZyBDQQIDDs2tMHoGCyqGSIb3DQEJEAILMWugaTBiMQswCQYDVQQGEwJa
QTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhh
d3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECAw7NrTANBgkqhkiG9w0BAQEFAASC
AQCjvsVTUU01aztxHW4tuQcVUFqOFjfkMO1ttMViINiG1s0eQ4iDttSXbs+bkmTRo3XJsdb/
wWaPn4rcYyx0Yp7Ora2PT9+ibR+TrGLWnlM0eHKAbaYP+aeXcGo/HpIqYxnqxOoZUkwI3zwe
LhCrY+Fr3jBacvr111mJhzaI92XbwkILlsf3c1IdrcH0JF9XscTxGPdJ0yGe5sJmVeJD8yp3
xp8mlA9XsEg7H5ExDLnhrIT0by+hreUiIHCv5+dIgnND2ZEH1XSem+7G5h7JysxHGx8QhIYy
OGsV19szJtj2L8Kz/neWKmhUSeElZjmDMapQpBdnWJH+usrNmPlnZQtmAAAAAAAA
--------------ms000600080005090706020508--