[OpenAFS] status of samba serving AFS file space? other non-native windows access?

Jeffrey Altman jaltman@secure-endpoints.com
Mon, 16 Oct 2006 17:05:20 -0400


This is a cryptographically signed message in MIME format.

--------------ms020504020103030704090006
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Danno:

I suspect that if people filed more bug reports when problems were
experienced that things would get fixed faster.  I understand that folks
are reluctant to file reports when they can't reproduce the problems or
when there are no resources available to assist in debugging or
identifying the issues.  Folks want a product that works and we want
to provide it.  Unfortunately, unless reports are received we have no
idea that there is a problem.  Each organization uses a different subset
of the functionality and therefore has the potential to identify
problems that others are not experiencing.

This is particularly frustrating for me because I am often hearing
rumors that organizations are having problems but when I ask for details
no one can provide them and bug reports are never submitted.

As it turns out about three weeks ago an organization brought to my
attention a serious problem in the Windows client that might have
impacted your users as it occurs when multiple clients are actively
creating/deleting files from the same directory.  To reproduce the
problem I wrote some test code which creates, writes, reads, closes,
deletes temporary files and executed it from multiple times and from
multiple machines in the same directory.  What I discovered was not
pleasant:

 * if the client has dirty buffers and the file has been deleted
   the buffers will remain dirty forever (or until the persistent
   AFS Cache is destroyed).

 * dirty buffers were not being written to the file server in a
   timely manner.  It could take more than 23 minutes for all of
   the buffers in a 80MB cache to be checked and written to the
   file server.  Dirty buffers are not flushed when a file is
   closed because if writing the dirty buffers takes longer than
   the CIFS timeout period, the CIFS client will tear down the
   SMB virtual circuit which in turn would invalidate all existing
   file handles and outstanding locks.

 * there were race conditions which could result in stat cache entry
   reference leaks.  A stat cache entry with a reference cannot be
   recycled.  If all stat cache entries are in use, then the OpenAFS
   client will panic when a new file is accessed.  If the reference
   counts wraps back to zero, the AFS client will panic when freeing a
   reference.  (stat cache entries are persistent so reference leaks
   accumulate over time.)

 * All pioctl operations would leak reference counts on the stat
   cache entries for the directory.  The Explorer Shell extension
   performs many pioctl calls on the directory each time it is
   refreshed.

 * If the client panics, all dirty buffers will be lost and Windows
   may restart the AFS Client Service so that the end user doesn't
   even realize that there was a crash.

Although they are not announced yet, OpenAFS 1.4.2 (205) and 1.5.9 (902)
builds address these issues.   There are links to the Windows installers
from

   http://www.openafs.org/windows.html

With regards to the distinctions between 1.4.2 and 1.5.9.  The rate
of new development on the Windows client far exceeds that of development
on any other platform.  This is primarily due to the fact that there has
been so much more that has been required for the Windows client to play
catchup with the rest of the platforms and because Microsoft has
released several new platform variants (XP SP2, 2003 SP1, 64-bit
XP/2003, and Vista).   I do not believe that the 1.5.9 client is any
less stable than the 1.4.2 client.  In fact, the reverse is problem true.

One performance problem that we have seen reported many times is that
the Windows clients performance gets really really slow after a
directory is listed multiple times in Explorer (or a File Open/Save
dialog.)  We narrowed the problem down to access control issues and
the lack of support in the Windows AFS client for the Inline Bulk Status
RPC.  Instead of performing a single RPC to obtain the status
information for the contents of an entire directory, the Windows client
performs a separate FetchStatus RPC for each entry.  If the user does
not have permission to obtain the status info for an entry, the file
server would send an error (rx abort EACCESS) to the client.  If the
file servers sends too many errors in a row, the rx library will attempt
to pace the client to prevent a denial of service against the file
server.   Support for Inline Bulk Status calls were added to 1.5.9 but
the changes are considered too large to pull into the 1.4 series.

The 1.5.9 release also contains support for a number of other "features":

    * Support for 64-bit Large Files
    * Support for 64-bit Windows XP/2003/R2/Vista
    * Implements Windows Byte Range Locking backed by AFS File Server
      Locks
    * Uses GetCapabilities RPCs to probe the server status
    * Logs fs crypt state to the Windows Event Log
    * Supports DebugOutputString debugging of the RX Library
    * New command: fs uuid [-generate]
    * Improved CIFS protocol capatibility
    * Hard Dead and Connection Timeout values restricted to the CIFS
      Session Timeout value.

Note that the 1.4.x and earlier clients would not reject attempts
to read/write data beyond 2GB.  The data would simply be corrupted.

In answer to your question regarding Samba.  There are several sites
that I work with who have used Samba as a gateway for users on MacOS X
and Windows that do not have AFS clients installed.  The number one
issue that they complain about is the fact that in order to use the
--fake-kaserver functionality in conjunction with either a Kerberos
KDC authentication or an LDAP authentication, the clients have to be
configured to send username/password in the clear.  Sending the user's
Kerberos password in the clear is not a desirable solution.  This may be
improved with Vista clients since Vista will negotiate TLS first and
then perform the SMB authentication on top of that.   Even if you are
willing to require that your users first VPN (or otherwise tunnel) to
the Samba server, you still have the problem that the most recent
MacOS X and Windows client OSs won't send username/password in the clear
out of the box.

Locking via the Samba server is not going to be improved.

WebDAV is a work in progress.  UMich has demonstrated CoSign, mod_dav
and mod_waklog working together but I believe there are still issues
to resolve.  Here is a link to the mod_waklog presentation from this
year's best practice workshop:

  http://www.pmw.org/afsbpw06/talks/jarod.willn/index.html

Regarding SFTPdrive, I have not used it.  I suspect if you can setup the
SSH server to access your files via AFS then the product will work.
I would guess that similar to the FTP shell extensions it works by
copying the file locally and then copying it back when the user is
finished.

Jeffrey Altman
Secure Endpoints Inc.

Dan Pritts wrote:
> Hi folks -
> 
> what's the current status of using samba to serve files from an 
> AFS backend to windows clients?  I could easily use Linux or Solaris
> as the samba server.  Other platforms would be possible.
> 
> how does handle file locking?  How does it handle authentication?
> 
> can you successfully use MS office documents from the share?
> 
> We're having some issues with deploying the windows AFS client to some
> of our staff, and I'm considering my alternatives.  
> 
> Other alternatives folks are using would be interesting too (webdav?).  
> 
> Anyone have any experience with this?
> 
>   http://www.sftpdrive.com/
> 
> [background] 
> 
> since i know folks will be interested, here's the scoop with our issues
> with the windows client.  I am not interested in starting any flame wars,
> and really i'm not interested in discussing the problems we've had with
> the windows client.  I personally am interested in making AFS work,
> but it has the potential to become a huge political issue within our
> organization and i'd like to avoid that.
> 
> Bottom line, our accounting staff has had a lot of bad experience with
> testing afs on the windows desktop.  Some of it has been in the form
> of long delays accessing files (which I think was related to recent
> bug fixes in the server code, and I think this problem is gone). 
> 
> some of it has been mysterious issues with changes not being saved.
> I think this is really because they are simultaneously editing AFS
> files with MS office, because they refuse to drag files to their desktop
> before opening and the 1.4 client we're using doesn't try to create any
> AFS locks.   Yes i know that 1.5 should solve this problem but i've been
> worried what else will happen with the unstable branch.
> 
> The situation is a lot better with the 1.4.2 rc client, but they're
> at the point that they have lost patience with problems and I need to
> give them something that just works.  I'm hoping to avoid that solution
> being a windows file server, since we will continue to use afs within
> the organization, but it's not out of the question.
> 
> thanks, 
> danno
> --
> Dan Pritts, System Administrator
> Internet2
> office: +1-734-352-4953 | mobile: +1-734-834-7224
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info

--------------ms020504020103030704090006
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJeTCC
AxcwggKAoAMCAQICEBW00lKwoWJXt8wbmTl1M0kwDQYJKoZIhvcNAQEEBQAwYjELMAkGA1UE
BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT
I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MDUyNzIyMDMzMloX
DTA3MDUyNzIyMDMzMlowczEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVy
aWMxHDAaBgNVBAMTE0plZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQC19SD7DncCP/+wfQlLzAAcxf1nJ/7UQgh4o/nxzvuY55XwHdLQjqWuFUnM5vecfyZKwq0o
fGCucDfcQbSIrkhHD9z4TZ8vDaYWVY9Foz8Rp8G0PNdbRFoFtfJbaeVBX5hG3aQXIc/T1b9U
8uN3kLyqXAFIGWKO8DJVGTKKtOiPVOp1U+9CwujyYmUSKF+suutKABhhK1ZGHsTnFczLZ2g0
ma0H7PiFJ2kLfOf///07E1fbr4IRb+cd87kpWLcjtEZ0rbBr9HlOy9dkeEii/qFoo1ahfKCD
A9bNErMiOXA3dudaNNzXlN/70slq5fboBXbepamJGrsnXYcCsS9+LtCTAgMBAAGjOTA3MCcG
A1UdEQQgMB6BHGphbHRtYW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADAN
BgkqhkiG9w0BAQQFAAOBgQDBzWhkrW+ol3iyT1rV8ZBQB0+z/6dFH3djQfNf7jDXNoXx4Vbo
pA7BAR4ihAPibv7j7ZaxmyMxWiDACRGS934uvUS0K6L6q14hTWMostJgFsAEDArrmbrES03v
L3EVETiGFqTB2sLp5MLc6+z+72pLXRuDPL3lO2GOQuBbILswRzCCAxcwggKAoAMCAQICEBW0
0lKwoWJXt8wbmTl1M0kwDQYJKoZIhvcNAQEEBQAwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoT
HFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25h
bCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MDUyNzIyMDMzMloXDTA3MDUyNzIyMDMzMlow
czEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVyaWMxHDAaBgNVBAMTE0pl
ZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRtYW5Ac2VjdXJlLWVuZHBv
aW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC19SD7DncCP/+wfQlL
zAAcxf1nJ/7UQgh4o/nxzvuY55XwHdLQjqWuFUnM5vecfyZKwq0ofGCucDfcQbSIrkhHD9z4
TZ8vDaYWVY9Foz8Rp8G0PNdbRFoFtfJbaeVBX5hG3aQXIc/T1b9U8uN3kLyqXAFIGWKO8DJV
GTKKtOiPVOp1U+9CwujyYmUSKF+suutKABhhK1ZGHsTnFczLZ2g0ma0H7PiFJ2kLfOf///07
E1fbr4IRb+cd87kpWLcjtEZ0rbBr9HlOy9dkeEii/qFoo1ahfKCDA9bNErMiOXA3dudaNNzX
lN/70slq5fboBXbepamJGrsnXYcCsS9+LtCTAgMBAAGjOTA3MCcGA1UdEQQgMB6BHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQQFAAOB
gQDBzWhkrW+ol3iyT1rV8ZBQB0+z/6dFH3djQfNf7jDXNoXx4VbopA7BAR4ihAPibv7j7Zax
myMxWiDACRGS934uvUS0K6L6q14hTWMostJgFsAEDArrmbrES03vL3EVETiGFqTB2sLp5MLc
6+z+72pLXRuDPL3lO2GOQuBbILswRzCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAw
gdExCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUg
VG93bjEaMBgGA1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRp
b24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFp
bCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0w
MzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU
aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg
RnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV
+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfAr
hVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l0cUqVIUPSAR/
p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMGA1UdHwQ8
MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVtYWls
Q0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh
YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/
TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amc
OY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggNkMIID
YAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5
KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ
FbTSUrChYle3zBuZOXUzSTAJBgUrDgMCGgUAoIIBwzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcN
AQcBMBwGCSqGSIb3DQEJBTEPFw0wNjEwMTYyMTA1MjBaMCMGCSqGSIb3DQEJBDEWBBSeed8n
Rbjs+1Qbsa8iWoAYI5bbSTBSBgkqhkiG9w0BCQ8xRTBDMAoGCCqGSIb3DQMHMA4GCCqGSIb3
DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBhQYJKwYB
BAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcg
KFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3Vpbmcg
Q0ECEBW00lKwoWJXt8wbmTl1M0kwgYcGCyqGSIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJa
QTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhh
d3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEBW00lKwoWJXt8wbmTl1M0kwDQYJ
KoZIhvcNAQEBBQAEggEAlHkmbVNr6Rh1vmoyfp9ZfkIk9TXDj8sKYGVUkKryZPWacwI9kvGz
Wh4BUcGqeHjeRYc6fpp3EM5Y1/QJOUbj3L4jcTtgS6BBcbueZUUHLSniAT1hTKmhBCyrQKxe
/GskelctuC0Q7xVb336bAYEfULVfpsekMkdUAmFSbtppceTEuDpiQ7BRBbub1dLshLf6HSC0
bPbPLkyzbErHIa3YQwcVsQDHhc+1BDh5YEsIcNpBO56GAebWbX78RgS8pzFQCpU0wEVC96tr
KDYvc6qiP890ghHxsT64PAK/QMxTKQtaR7FYsEFnjWvS55ls8AYdxCWsJtrQwnUWQxbxlNqr
3QAAAAAAAA==
--------------ms020504020103030704090006--