"OpenAFS for Windows development road map" was Re: [OpenAFS] 1.4.1-rc2 build question

[Jeffrey Altman]

This is a cryptographically signed message in MIME format.

--------------ms090307070504080806020207
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

What I am reading in this thread is that people are afraid of the 
unknown.  I am going to make this argument because no one is saying "do 
not implement this functionality" nor are they saying keep this 
functionality out of my cell.   What is being argued for is a method 
that temporarily provides the end user choice over whether or not they 
use this functionality other than by choosing which client version they 
install on their machine.

Nathan has put forward an idea that has been supported by all 
respondents that involves a multi-staged implementation.  I'm going to 
associate dates with some hypothetical releases because I think the 
version numbers are somewhat irrelevant:

  * Nov 2005 - Stable Windows client without byte range locking

  * Jan 2006  - Stable Windows client with optional but disabled byte 
range locking

  * Apr 2006 - Stable Windows client with byte range locking that can be 
disabled

  * Jun 2006  - Stable Windows client with byte range locking that is 
mandatory to use

which differs from the original proposal of:

  * Nov 2005 - Stable Windows client without byte range locking

  * Jan 2006 -  Stable Windows client with byte range locking that is 
mandatory to use

Of course, the reality is that except for the managed environments in 
which the choice of OpenAFS Client on the machine is controlled by 
Active Directory Group Policy or an equivalent functionality, the choice 
of which OpenAFS client the user deploys is an individual choice heavily 
influenced by the recommendations of the organization from whom they 
obtain most of their software.  There are numerous organizations that do 
not currently deploy OpenAFS 1.4.0 to their users.  Searching with 
Google reveals that there are dozens of sites with OpenAFS downloads or 
instructions for installation OpenAFS that indicate that their users 
should install a specific build anywhere from 1.2.10 to 1.3.63 to 1.3.71 
to 1.3.82 to 1.3.87.   In fact while I believe that all of these sites 
should be recommending 1.4.0, I am not aware of a single organization 
that is currently recommending that 1.4.0 be deployed.   This is not 
because they do not feel that 1.4.0 is quality software but simply that 
they do not feel there are enough differences between 1.3.87 and 1.4.0 
to warrant advising their users to upgrade.

When I speak to those that are contributing resources to further the 
development of OpenAFS, the things I am told would encourage them to 
upgrade their users are new features:

* Byte Range Locking

* 64-bit Windows implementations

* Replacement of the existing AFS credential management systray tool

* Client side Unicode character-set and large file support

* Strong cryptographic protection of client and server data exchanges 
enforceable by the AFS servers

* Disconnected Operations

* Installable File System

Most of these features will be implemented in OpenAFS for Windows within 
the next nine months.    Here are some dates of when I expect they will 
be available in the development branch of the repository and when they 
might end up in a shipping product:

* Byte Range Locking:  in the repository now, shipping now

* 64-bit Windows: in the repository now, shipping in first quarter of 2006

* Replacement Credentials Manager:  MIT KFW 3.0 containing the Network 
Identity Manager is in Beta.  Secure Endpoints is providing the AFS 
Credential Manager plug-in which is available now in Beta form.  The 
final installer for the AFS plug-in will give the user the choice of 
disabling the existing AFS Credentials Manager.  Incorporation into 
OpenAFS by next summer.

* Client side Unicode character-set and large file support:  development 
is planned for the first quarter 2006.  Expect completion by May 2006 
and inclusion in an OpenAFS release by the summer.  

* Strong cryptographic protection (RXGK).   An architectural design by 
Jeffrey Hutzelman, Love Hörnquist Åstrand, Derek Atkins, and myself has 
been completed.   The write-up needs to be completed for publication 
this month.   I anticipate a rough implementation of the rx security 
class by the end of January.  Actual deployment for roll out will depend 
upon how long it takes to implement the required cache manager and 
server functionality.  A year from now is certainly a reasonable 
production goal (if not sooner).

* Disconnected Operations:  no current plans

* Installable File System:  Eric Williams contributed an incomplete 
implementation that is currently on the repository trunk.  I hope to 
finish this work for sometime in 2006.

Of these major features, the one that is going to be most disruptive is 
the Unicode support.   The reason that this is going to be a headache is 
that currently the character-set used by 99.9% of the Windows clients 
for directory entries is IBM Code Page 437/850.   This is due to the 
fact that the Windows AFS Cache Manager is really implemented as an 
SMB/CIFS gateway service and the SMB/CIFS support only handles the older 
Windows 3.1 and OS/2 compatible protocol and not the newer variants.  
Therefore, Windows itself converts all path names from Unicode to one of 
the what they refer to as OEM code pages before communicating with the 
AFS Client Service.   When SMB/CIFS Unicode support is added, this 
translation will no longer exist.   The AFS client service will receive 
UTF16 encoded Unicode.   The challenge here is that pathnames stored as 
CP437/850 within AFS volumes will not be able to be read by AFS clients 
that process UTF8 encoded Unicode (as does MacOSX).   A similar problem 
already exists if users try to move between Windows and Solaris (in most 
cases ISO Latin 1) and try to share files.   The problem is that there 
is going to be no smooth transition if there are users that have 
significant numbers of files that are named with non-ASCII characters.   
Losing access to their files entirely is going to be a bigger problem 
than any locking related issues.

One of the points that I am attempting to make is that the rate of 
change in the Windows client is going to continue to out pace the rate 
of change in the Unix-based implementations for at least the next year 
as it continues to play catch up.   During this time there is going to 
be a significant impact on end-users with new user interfaces and 
behavioral experiences depending on which client the user chooses to 
install.   As long as the version numbers of the Windows are locked to 
the Unix-based distributions, it is not going to be possible to avoid 
adding significant changes into the existing 1.4 stable series.  The way 
I rationalize it is that the OpenAFS version number is tied to a set of 
functionality implemented in the servers.   As long as the server 
functionality does not change in a significant manner, the version 
number will not be bumped.  My intention is that after 1.4.1 is released 
that I will maintain two sets of Windows releases simultaneously.   The 
1.4.xxxx series is the stable series that organizations are encouraged 
to deploy.  The 1.5.xxxx series will be the development series in which 
new functionality will be baked prior to being pulled back into a 
1.4.xxxx release.   Using this model 1.5.1 will be released this month 
with the 64-bit Windows support which will be pulled into 1.4.2 sometime 
between now and March.  The Unicode/Large File support will end up in 
the 1.5 series around April and will end up in a 1.4.3 with the new UI 
around June.  The version numbers and dates are all subject to change. 

Now there have been installers with the Byte Range Locking code 
available for download for several months now.   They have been 
announced both on the openafs-win32-devel and openafs-info mailing 
lists.   The users of 1.4.1 RC1 and now RC2 have this code.   In the 
months that these installers have been available I have received a total 
of one bug report.   I will be more than happy to add the desired switch 
to disable the functionality if the consensus is that it is required but 
I keep finding myself asking the same question, if I disable Byte Range 
Locking in 1.4.1 what incentive is there for you to upgrade your users 
to 1.4.1 as opposed to 1.4.0?   The bugs in the 1.4.0 client are so 
minor that simply obtaining a fix for those bugs if you are not actively 
experiencing them is not a reason to upgrade.  Releasing 1.4.1 is not 
going to make the 1.4.0 release suddenly become "unstable" and 
unusable.   Too much effort was put into developing and testing it for 
that to happen. 

I am left wondering what is really behind the fears associated with this 
functionality.   I interpreted Terry's original post as an objection to 
the release on the expectation that it would require changes to the 
software running on the servers not changes to the ACLs that are 
assigned to the the user volumes.   As such he wanted to be able to 
disable that functionality or perhaps enable functionality on the 
servers that would cause the Windows client behavior in his cell to 
appear to be unchanged.   There were several discussions in other forums 
that were previously held that discussed the possibility of hacking the 
servers to always interpret "rl" as "rlk".  It was concluded that this 
functionality should not be added to openafs.

I am left with the following question: if I disable this functionality 
for three months and turn it on by default in 1.4.2, is the state of the 
world going to change enough to make a difference?

Jeffrey Altman




--------------ms090307070504080806020207
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJXzCC
AwowggJzoAMCAQICAw7NrTANBgkqhkiG9w0BAQQFADBiMQswCQYDVQQGEwJaQTElMCMGA1UE
ChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNv
bmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EwHhcNMDUwNTI3MTc0NzU3WhcNMDYwNTI3MTc0NzU3
WjBzMQ8wDQYDVQQEEwZBbHRtYW4xFTATBgNVBCoTDEplZmZyZXkgRXJpYzEcMBoGA1UEAxMT
SmVmZnJleSBFcmljIEFsdG1hbjErMCkGCSqGSIb3DQEJARYcamFsdG1hbkBzZWN1cmUtZW5k
cG9pbnRzLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKjPyrF+rdjOUSK/
bWwZHdx5p1+y6iiCd4vvYEVDxouYFp5C/fZEWm5n45ubBUbMSUI1MAZN6ooEoH09UTj6BXhM
S8B987ls81dKOIUphTF2jOzq8gsFmeA15yHMRAD20LqUWeLyvYk8FCNQw+dsKMMhX+WdsxOm
RY/1jPkJL6oN8kEwoUFkOX9/OfWWh6oFnV6faiEHUKDMFubsb9X0KVD8iIeR7Cxz7i4kXqRX
wMlp2fyoxcDIJrBaTY8nA++g3p34IkWt1a5po6g683nIgSnGpwYIwuJheBqSEZfLYWa+1KdD
6Sn27Ud94GqUvPVG5jC6zVC5EJ2aWuoAu+nNuV8CAwEAAaM5MDcwJwYDVR0RBCAwHoEcamFs
dG1hbkBzZWN1cmUtZW5kcG9pbnRzLmNvbTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBAUA
A4GBADtvO//tjiAV6VJGtoNtrl34mB5jGyGTiotzw8riB6zz0GvY11bcWDmp6JKif+pVG+8L
IySDosbuva13qu2HwYUxBmWc7CoNd2k9kRlcrfbDUTTrGOZK8qyqNqT3gQZTAa9ZnUI0su9G
y/n2o5bQcaYdqR3htNrpvdLSPOWhILOXMIIDCjCCAnOgAwIBAgIDDs2tMA0GCSqGSIb3DQEB
BAUAMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBM
dGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTAeFw0w
NTA1MjcxNzQ3NTdaFw0wNjA1MjcxNzQ3NTdaMHMxDzANBgNVBAQTBkFsdG1hbjEVMBMGA1UE
KhMMSmVmZnJleSBFcmljMRwwGgYDVQQDExNKZWZmcmV5IEVyaWMgQWx0bWFuMSswKQYJKoZI
hvcNAQkBFhxqYWx0bWFuQHNlY3VyZS1lbmRwb2ludHMuY29tMIIBIjANBgkqhkiG9w0BAQEF
AAOCAQ8AMIIBCgKCAQEAqM/KsX6t2M5RIr9tbBkd3HmnX7LqKIJ3i+9gRUPGi5gWnkL99kRa
bmfjm5sFRsxJQjUwBk3qigSgfT1ROPoFeExLwH3zuWzzV0o4hSmFMXaM7OryCwWZ4DXnIcxE
APbQupRZ4vK9iTwUI1DD52wowyFf5Z2zE6ZFj/WM+Qkvqg3yQTChQWQ5f3859ZaHqgWdXp9q
IQdQoMwW5uxv1fQpUPyIh5HsLHPuLiRepFfAyWnZ/KjFwMgmsFpNjycD76DenfgiRa3Vrmmj
qDrzeciBKcanBgjC4mF4GpIRl8thZr7Up0PpKfbtR33gapS89UbmMLrNULkQnZpa6gC76c25
XwIDAQABozkwNzAnBgNVHREEIDAegRxqYWx0bWFuQHNlY3VyZS1lbmRwb2ludHMuY29tMAwG
A1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEEBQADgYEAO287/+2OIBXpUka2g22uXfiYHmMbIZOK
i3PDyuIHrPPQa9jXVtxYOanokqJ/6lUb7wsjJIOixu69rXeq7YfBhTEGZZzsKg13aT2RGVyt
9sNRNOsY5kryrKo2pPeBBlMBr1mdQjSy70bL+fajltBxph2pHeG02um90tI85aEgs5cwggM/
MIICqKADAgECAgENMA0GCSqGSIb3DQEBBQUAMIHRMQswCQYDVQQGEwJaQTEVMBMGA1UECBMM
V2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBlIFRvd24xGjAYBgNVBAoTEVRoYXd0ZSBDb25z
dWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZpY2F0aW9uIFNlcnZpY2VzIERpdmlzaW9uMSQwIgYD
VQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgQ0ExKzApBgkqhkiG9w0BCQEWHHBlcnNv
bmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcNMDMwNzE3MDAwMDAwWhcNMTMwNzE2MjM1OTU5
WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRk
LjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0EwgZ8wDQYJ
KoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVzVftOucqZWh5owHUEcJ3f6f+jHuy9zfVb8hp2
vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnwK4Vaqj9xVsuvPAsH5/EfkTYkKhPPK9Xzgnc9
A74r/rsYPge/QIACZNenprufZdHFKlSFD0gEf6e20TxhBEAeZBlyYLf7AgMBAAGjgZQwgZEw
EgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8EPDA6MDigNqA0hjJodHRwOi8vY3JsLnRoYXd0
ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFpbENBLmNybDALBgNVHQ8EBAMCAQYwKQYDVR0R
BCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVMYWJlbDItMTM4MA0GCSqGSIb3DQEBBQUAA4GB
AEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSdf0whuPg2H6otnzYvwPQcUCCTcDz9reFhYsPZ
Ohl+hLGZGwDFGguCdJ4lUJRix9sncVcljd2pnDmOjCBPZV+V2vf3h9bGCE6u9uo05RAaWzVN
d+NWIXiC3CEZNd4ksdMdRv9dX2VPMYIDOzCCAzcCAQEwaTBiMQswCQYDVQQGEwJaQTElMCMG
A1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBl
cnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECAw7NrTAJBgUrDgMCGgUAoIIBpzAYBgkqhkiG
9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0wNTEyMDMwNjI2NTdaMCMGCSqG
SIb3DQEJBDEWBBTRv+1kAaZav1YHX1tKvOV08Ou9wjBSBgkqhkiG9w0BCQ8xRTBDMAoGCCqG
SIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG
9w0DAgIBKDB4BgkrBgEEAYI3EAQxazBpMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3
dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJl
ZW1haWwgSXNzdWluZyBDQQIDDs2tMHoGCyqGSIb3DQEJEAILMWugaTBiMQswCQYDVQQGEwJa
QTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhh
d3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECAw7NrTANBgkqhkiG9w0BAQEFAASC
AQChPtXi3l7OI/6LpIYnMHKCEyG9buXFAGm582W5R0YQDwAg09EnhCo8j/i2crMHh6Eg4wjb
3R7YURfuXYD9em4dVROAR58Nw+bB9xVsNOBCmyL5SX/pxGv1dQ7gqcxRvCzx6JuDZrTAimrf
Op/sEkr9N5rebnza6CFk+0iQSDvbOfD/cfH0GjF2vKpsUDoHbeY+FuT0dM6yx3ZJJOVJeOem
fmbmExU0/egwSSzSL7uVK0ez6nbDyZ7qdhTIA3FGhKLEQ8fp50iOF6umHwiMqUxCAHABhvvA
ZsFLz7EEfOHvsSxpmkNIVK7ssW7g1l+I+/14o34HSNAOk41Hzky24YPFAAAAAAAA
--------------ms090307070504080806020207--