"OpenAFS for Windows development road map" was Re: [OpenAFS] 1.4.1-rc2 build question

Jeffrey Altman jaltman@secure-endpoints.com
Sat, 03 Dec 2005 01:26:57 -0500

This is a cryptographically signed message in MIME format.

Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

What I am reading in this thread is that people are afraid of the 
unknown.  I am going to make this argument because no one is saying "do 
not implement this functionality" nor are they saying keep this 
functionality out of my cell.   What is being argued for is a method 
that temporarily provides the end user choice over whether or not they 
use this functionality other than by choosing which client version they 
install on their machine.

Nathan has put forward an idea that has been supported by all 
respondents that involves a multi-staged implementation.  I'm going to 
associate dates with some hypothetical releases because I think the 
version numbers are somewhat irrelevant:

  * Nov 2005 - Stable Windows client without byte range locking

  * Jan 2006  - Stable Windows client with optional but disabled byte 
range locking

  * Apr 2006 - Stable Windows client with byte range locking that can be 

  * Jun 2006  - Stable Windows client with byte range locking that is 
mandatory to use

which differs from the original proposal of:

  * Nov 2005 - Stable Windows client without byte range locking

  * Jan 2006 -  Stable Windows client with byte range locking that is 
mandatory to use

Of course, the reality is that except for the managed environments in 
which the choice of OpenAFS Client on the machine is controlled by 
Active Directory Group Policy or an equivalent functionality, the choice 
of which OpenAFS client the user deploys is an individual choice heavily 
influenced by the recommendations of the organization from whom they 
obtain most of their software.  There are numerous organizations that do 
not currently deploy OpenAFS 1.4.0 to their users.  Searching with 
Google reveals that there are dozens of sites with OpenAFS downloads or 
instructions for installation OpenAFS that indicate that their users 
should install a specific build anywhere from 1.2.10 to 1.3.63 to 1.3.71 
to 1.3.82 to 1.3.87.   In fact while I believe that all of these sites 
should be recommending 1.4.0, I am not aware of a single organization 
that is currently recommending that 1.4.0 be deployed.   This is not 
because they do not feel that 1.4.0 is quality software but simply that 
they do not feel there are enough differences between 1.3.87 and 1.4.0 
to warrant advising their users to upgrade.

When I speak to those that are contributing resources to further the 
development of OpenAFS, the things I am told would encourage them to 
upgrade their users are new features:

* Byte Range Locking

* 64-bit Windows implementations

* Replacement of the existing AFS credential management systray tool

* Client side Unicode character-set and large file support

* Strong cryptographic protection of client and server data exchanges 
enforceable by the AFS servers

* Disconnected Operations

* Installable File System

Most of these features will be implemented in OpenAFS for Windows within 
the next nine months.    Here are some dates of when I expect they will 
be available in the development branch of the repository and when they 
might end up in a shipping product:

* Byte Range Locking:  in the repository now, shipping now

* 64-bit Windows: in the repository now, shipping in first quarter of 2006

* Replacement Credentials Manager:  MIT KFW 3.0 containing the Network 
Identity Manager is in Beta.  Secure Endpoints is providing the AFS 
Credential Manager plug-in which is available now in Beta form.  The 
final installer for the AFS plug-in will give the user the choice of 
disabling the existing AFS Credentials Manager.  Incorporation into 
OpenAFS by next summer.

* Client side Unicode character-set and large file support:  development 
is planned for the first quarter 2006.  Expect completion by May 2006 
and inclusion in an OpenAFS release by the summer.  

* Strong cryptographic protection (RXGK).   An architectural design by 
Jeffrey Hutzelman, Love Hörnquist Åstrand, Derek Atkins, and myself has 
been completed.   The write-up needs to be completed for publication 
this month.   I anticipate a rough implementation of the rx security 
class by the end of January.  Actual deployment for roll out will depend 
upon how long it takes to implement the required cache manager and 
server functionality.  A year from now is certainly a reasonable 
production goal (if not sooner).

* Disconnected Operations:  no current plans

* Installable File System:  Eric Williams contributed an incomplete 
implementation that is currently on the repository trunk.  I hope to 
finish this work for sometime in 2006.

Of these major features, the one that is going to be most disruptive is 
the Unicode support.   The reason that this is going to be a headache is 
that currently the character-set used by 99.9% of the Windows clients 
for directory entries is IBM Code Page 437/850.   This is due to the 
fact that the Windows AFS Cache Manager is really implemented as an 
SMB/CIFS gateway service and the SMB/CIFS support only handles the older 
Windows 3.1 and OS/2 compatible protocol and not the newer variants.  
Therefore, Windows itself converts all path names from Unicode to one of 
the what they refer to as OEM code pages before communicating with the 
AFS Client Service.   When SMB/CIFS Unicode support is added, this 
translation will no longer exist.   The AFS client service will receive 
UTF16 encoded Unicode.   The challenge here is that pathnames stored as 
CP437/850 within AFS volumes will not be able to be read by AFS clients 
that process UTF8 encoded Unicode (as does MacOSX).   A similar problem 
already exists if users try to move between Windows and Solaris (in most 
cases ISO Latin 1) and try to share files.   The problem is that there 
is going to be no smooth transition if there are users that have 
significant numbers of files that are named with non-ASCII characters.   
Losing access to their files entirely is going to be a bigger problem 
than any locking related issues.

One of the points that I am attempting to make is that the rate of 
change in the Windows client is going to continue to out pace the rate 
of change in the Unix-based implementations for at least the next year 
as it continues to play catch up.   During this time there is going to 
be a significant impact on end-users with new user interfaces and 
behavioral experiences depending on which client the user chooses to 
install.   As long as the version numbers of the Windows are locked to 
the Unix-based distributions, it is not going to be possible to avoid 
adding significant changes into the existing 1.4 stable series.  The way 
I rationalize it is that the OpenAFS version number is tied to a set of 
functionality implemented in the servers.   As long as the server 
functionality does not change in a significant manner, the version 
number will not be bumped.  My intention is that after 1.4.1 is released 
that I will maintain two sets of Windows releases simultaneously.   The 
1.4.xxxx series is the stable series that organizations are encouraged 
to deploy.  The 1.5.xxxx series will be the development series in which 
new functionality will be baked prior to being pulled back into a 
1.4.xxxx release.   Using this model 1.5.1 will be released this month 
with the 64-bit Windows support which will be pulled into 1.4.2 sometime 
between now and March.  The Unicode/Large File support will end up in 
the 1.5 series around April and will end up in a 1.4.3 with the new UI 
around June.  The version numbers and dates are all subject to change. 

Now there have been installers with the Byte Range Locking code 
available for download for several months now.   They have been 
announced both on the openafs-win32-devel and openafs-info mailing 
lists.   The users of 1.4.1 RC1 and now RC2 have this code.   In the 
months that these installers have been available I have received a total 
of one bug report.   I will be more than happy to add the desired switch 
to disable the functionality if the consensus is that it is required but 
I keep finding myself asking the same question, if I disable Byte Range 
Locking in 1.4.1 what incentive is there for you to upgrade your users 
to 1.4.1 as opposed to 1.4.0?   The bugs in the 1.4.0 client are so 
minor that simply obtaining a fix for those bugs if you are not actively 
experiencing them is not a reason to upgrade.  Releasing 1.4.1 is not 
going to make the 1.4.0 release suddenly become "unstable" and 
unusable.   Too much effort was put into developing and testing it for 
that to happen. 

I am left wondering what is really behind the fears associated with this 
functionality.   I interpreted Terry's original post as an objection to 
the release on the expectation that it would require changes to the 
software running on the servers not changes to the ACLs that are 
assigned to the the user volumes.   As such he wanted to be able to 
disable that functionality or perhaps enable functionality on the 
servers that would cause the Windows client behavior in his cell to 
appear to be unchanged.   There were several discussions in other forums 
that were previously held that discussed the possibility of hacking the 
servers to always interpret "rl" as "rlk".  It was concluded that this 
functionality should not be added to openafs.

I am left with the following question: if I disable this functionality 
for three months and turn it on by default in 1.4.2, is the state of the 
world going to change enough to make a difference?

Jeffrey Altman

Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature