[OpenAFS] deploying software via afs

[Rodney M Dyer]

At 12:00 PM 2/2/2006, David Bear wrote:
>I am wondering if anyone has written on ways to use afs to deploy 
>application software. I'm looking at deploying things like cygwin, python, 
>and even microsoft office apps via afs.

Boy are you asking for the world.  This thread should be entitled 
"Deploying Windows Applications (1001 things Microsoft never told 
you)".  Software installation and change management of enterprise 
applications can read like a "War and Peace" novel to those uninitiated.

All of our software is distributed to the local client via AFS.  We have 
over 125 Windows applications on each client.  We have over 600 XP 
clients.  The method(s) used to deliver applications to the client machines 
isn't exactly simple, but it's no mind-bender either.  It's just a lot of 
elbow grease (this is why you work in IT right?).  You can buy some tools 
to help for this, or you can roll your own method.  Some people use 
Microsoft's Systems Management Server, others use Novell's Zenworks, still 
others use Macromedia's AdminStudio.  But these are only tools, and these 
are somewhat limited to the setup styles they can perform.  Even with these 
tools it is likely that you will need to augment them with some install 
scripting.  It really depends on how far you need to take it, and whether 
you run your machines with users in L.U.A. mode or not.  Some people use 
Active Directory's built in ability to use Group Policy to "publish" 
applications to install.  Even here, you usually end up writing an install 
script, and it is really hard to do change management with this method.

What did we do?  We rolled our own of course (I always make things 
complicated for myself).  Let me see if I can nutshell this one for you...

1.  In AFS you create a distribution point for each software title or 
suite.  For example, for Office 2003 we create a folder called 
mso2003.  The "mso2003" is our "application reference code".  Every 
application suite should have it's own alphanumeric reference code.  Ours 
are generally 16 characters or less consisting of numbers and letters 
without spaces.

2.  In each distribution folder you create a system setup script.  This 
script can be written in any scripting language you prefer, but should be 
written with some method to return error variables to the caller such as 
"out of disk space", or "install error", "success", etc.  You should create 
a standard interface for the caller so that all your distribution point 
install scripts generally use the same method.

3.  The distribution folder contains sub-folders containing whatever files 
are necessary to install the application such as files to be copied to the 
PC, registry changes, MSI installer scripts, setup exe's, etc.  The 
distribution point may also be the location that the application is run 
from in the case where you are installing the app to run "off the 
net".  There are sticky issues here because some apps require the ability 
to write into their own application directory (BAD APP!  BAD!!!).  Because 
we use replicated AFS volumes for our distribution points, applications 
that write into their own program locations are always installed 
locally.  Sometimes however you can do registry or configure file 
modifications so that the application will use other locations for 
writing.  Application installation tweaking is an art form unto itself and 
can be quite costly in terms of the IT time involved.  Over the years we've 
used many "hacks" to make our applications run correctly on our locked down 
user machines.  It's a big P.I.T.A.

4.  The algorithm your system setup script uses should generally follow the 
process...

      a.  Check to see if the application is already installed.
            Our software setup scripts create a flag file on the
            local box when they complete successfully such
            as "c:\admin\software\mso2003.txt".  Your setup script
            an check for this flag file.

      b.  Check to see if the amount of disk free space is
            enough for the application.

      c.  Other checks where necessary, including software
           dependency checks.  Check for other software flag files.

      d.  Run your application setup.  This is highly dependent on
            how you are installing the application and is the most
            variable (complicated).

            Some applications are silently installed by running the
            setup with special options.  Others are installed with MSI
            files.  Some may use keyboard scripts.  Some may use
            xcopy.exe to copy files out of AFS to the local disk.  Still
            others might need only to copy the DLLs.  If you've "diffed"
            your application installed then you will need to install registry
            settings as well.  This involves the highly complicated method
            of "application re-wrapping" where you copy down the
            changes that you saw the application's setup make to a
            test system earlier.

            All of these setup methods run into complications.  You need
            to wait for the setup method to complete.  Some setup scripts
            don't wait, they simply "background" themselves so you will
            have to come up with a method to "block" your system setup
            script until the setup exe, MSI, or copy completes.

            Our shop has developed several tools for helping to install
            our applications.  Here's an example of one we use...
            http://www.coe.uncc.edu/~rmdyer/software/waitfortext/

            ...and here's some example software setup scripts...
            http://www.coe.uncc.edu/~rmdyer/software/example_scripts/app_install_scripts

            Some installs scripts are quite complex like this one...
            http://www.coe.uncc.edu/~rmdyer/software/example_scripts/app_install_scripts/ongard2005/syssetup.cmd.txt

         You will notice that all of these scripts generally use the same 
install
            method.  This is no coincidence.  Our application diff'ing tool 
creates
            these scripts for us.  The stucture of the scripts is somewhat 
out of
            date, but we've never needed to modify it.  We intend to for 
Windows
            Vista.

            Some of our software is installed using "application 
re-wrapping", or
            "diff'ing".  In our system setup scripts for these applications 
we end up
            copying the files and registry changes that were found from the 
diff
            out of AFS, to the local machine.  This is complicated by the 
fact that
            you really need to check the DLLs first, then perform renames 
if the
            existing DLLs are in-use, and even further possibly perform MAR, or
            move-after-reboot operations (pending file rename operations).
            You should NEVER copy a DLL out of AFS to the local machine without
            first guaranteeing that the one you are copying has "increased" in
            version number.  Even still, sometimes on "rare" occasions this may
            not be the "right" thing to do.

      e.  If your setup completed successfully then you write some flag
           file showing the application is installed, then you return "success"
           to the caller (the parent script).  If your application fails to 
install,
           then you return a failure notice.

5.  You need some method to call all of your system setup scripts for every 
application to be installed.  This is generally done with a parent "update" 
script.  It is the responsibility of the parent "update" script to get the 
list of system setup scripts to run, execute them in the order that you 
specify, then finish.  In the parent script, you can perform pre-setup, and 
post-setup script operations.  You may need to reboot.  Some of our 
application system setup scripts need to tell the parent caller to "reboot 
the machine" before installing any other application setup scripts.  So in 
our case the parent gets a special code from the system setup script that 
says "reboot for phase2".  After reboot, the child script may have re 
configured the machine to auto-logon as admin and run some further special 
scripting (or not).  In either case, after reboot, the parent script starts 
again and continues where it left off in the list of application setup 
scripts to install.  Here's a log of the entire process of software 
application on my current workstation as an example...
http://www.coe.uncc.edu/~rmdyer/software/example_scripts/update.log
This log file was created by the parent script.  The log file shows 
multiple reboots because sometimes updates require that.  The parent script 
manages the entire state of the machine update process (including reboots 
if necessary) throughout the update.  The parent script is coded with a 
update reference code in the format "yyyymmddtttt" format.  We use this 
reference code to determine whether the machine has the current "set" of 
applications and changes.

Since we install over 120 applications the entire update process from build 
to finish can take some time.  Some of our slower machines can take 
hours.  Some of our application installs from AFS are merely file copies 
and we don't use compression, so this can extend the time for the install.

The process of installing and maintaining change-management of application 
software and system settings on an enterprise network of machines isn't for 
the feeble at heart.  You can take shortcuts but you will pay dearly.  Of 
course all of these methods highly depend on whether your users are Admins 
on their own machines, or whether you are controlling a locked down 
environment.  Most organizations don't have the experience necessary to 
manage applications on locked down user machines.  The tools necessary to 
do so have evolved over the years.  Many of our tools are home grown, some 
are off the shelf, and some tools come with XP or the resource kit.

In the University environment of which I am a part, all this complexity is 
magnified because of the sheer number of applications we need to manage 
every semester.  Most lab machines need all the apps.  Sometimes you can 
"zone" certain applications to certain machines and not others.  Our parent 
update scripts gets the list of applications to install on a machine from 
our Oracle database at the time the script is run.  So our Oracle database 
determines which apps should go where.

There are many companies, Microsoft included, who would like to make these 
management processes simplified.  But take every companies product with a 
grain of salt.  Many times these products are just snake oil, or they only 
get you half way to where you need to be.  Like I said before, many of them 
are "just tools".  You will still end up scripting many of your application 
installs.  We took the approach of setting up an entire structured 
application update process that we can control completely.  We micro-manage 
every application install using our update procedure.

AFS really has little to do with the setup of applications, except in that 
you use AFS for the distribution files.  In our case, some of our 
applications run on the local machine, while others run out of AFS when the 
user clicks on them.  In this case you might need to manage AFS ACLs to 
prevent access to applications that some users shouldn't get to.  But this 
is the case even if you used CIFS, NFS, or Novell NCP.

Depending on your own IT environment the above process may be over 
complicated.  There's obviously a ton of information I haven't written 
about here.  All I can say is good luck to you.

Rodney

Rodney M. Dyer
Windows Systems Programmer
Mosaic Computing Group
William States Lee College of Engineering
University of North Carolina at Charlotte
Email: rmdyer@uncc.edu
Web: http://www.coe.uncc.edu/~rmdyer
Phone: (704)687-3518
Help Desk Line: (704)687-3150
FAX: (704)687-2352
Office:  Cameron Applied Research Center, Room 232