[OpenAFS-devel] Website "bugs"

Thu, 7 Jun 2012 08:20:29 -0600

On Thu, Jun 7, 2012 at 7:25 AM, Jeff Blaine <jblaine@kickflop.net> wrote:
> Let's assume for the sake of argument, based on historical
> progress, dead efforts to that effect, and more pressing issues,
> that the website is never going to get completely redone.
> Based on that assumption, can we have the existing setup
> briefly documented? I've come to understand that there is
> some "web tools" package and process stuff.
>

Believe it or not, I'm slowly working on a website rewrite, and I've
got some code published at https://gitorious.org/ktdreyer/openafs-web.

Here's my notes on how to set up www.openafs.org on your own computer.

--------------------

I needed to set up my own mirror of the contents *and* configuration
of OpenAFS.org. Here's how I did it. I'm using CentOS 5, but any Linux
distro should work with little or no adjustments. I think the
production openafs.org web server is Debian.

== Set up a local Apache virtual host ==
I'm going to call my local copy of the site
"merge.openafs.ktdreyer.com". I need to define an Apache virtual host
called "merge.openafs.ktdreyer.com", and create it's area on the
filesystem.

On my Apache systems, I store my virtual hosts' basic definitions in a
file called vhosts.conf, which I include from my main httpd.conf. You
could just add this to httpd.conf directly, but I like to keep things
separate for cleanliness. Here's what I added to vhosts.conf:

<VirtualHost *:80>
        Include sites/merge.openafs.ktdreyer.com.conf
</VirtualHost>

This is the standard way that I add a new vhost to my config. (If
you're interested, the reason I do this is so I can also make the
vhost listen on :443 with minimal config duplication). Next, I need to
actually define the contents of this merge.openafs.ktdreyer.com.conf
file:

# merge.openafs.ktdreyer.com

# Corresponds to /afs/grand.central.org/www/www.openafs.org/
DocumentRoot /var/www/merge.openafs/openafs-web
ServerName merge.openafs.ktdreyer.com
ErrorLog logs/merge.openafs-error_log
CustomLog logs/merge.openafs-access_log combined

<Directory "/var/www/merge.openafs/openafs-web">
    Options Indexes FollowSymLinks Includes
</Directory>

It's a pretty basic vhost setup. The logging will certainly help
troubleshoot errors along the way.

mkdir /var/www/merge.openafs
chown kdreyer /var/www/merge.openafs

== Getting a local copy of the site contents with Git ==

Exit out of root's shell, and log in with my normal user account, and
pull down the data from git:

cd /var/www/merge.openafs
git clone git://git.openafs.org/openafs-web.git
scp -p -P 29418 gerrit.openafs.org:hooks/commit-msg openafs-web/.git/hooks/

The openafs-web directory appears. That will be the DocumentRoot for
the site. This is also exactly how the production site is updated; it
simply another git clone (that happens to live in AFS.)

It's also a good idea to grab a local clone of the "tools" repository.
Nothing that I've seen in Apache itself references this, but it's
useful to have a local clone in order to look at the code.

cd /var/www/merge.openafs
git clone git://git.openafs.org/tools.git

This creates a "tools" directory alongside "openafs-web".

== Mirroring the Apache configuration of openafs.org ==

OpenAFS.org has some funky SSI things going on, and it's important to
duplicate that exactly on my local mirror. It is difficult to tell
exactly how it all works, particularly the "fset" and "fset-root"
stuff, without seeing the config yourself, so we'll need to grab that
from the production site.

The OpenAFS.org website is hosted within the grand.central.org cell.
If you're curious why this is, read some history in the mailing list
archives: eg. https://lists.openafs.org/pipermail/openafs-devel/2001-June/006211.html

After hunting around g.c.o for a bit, I found some config files in
/afs/grand.central.org/local/src/httpd/. openafs-common looks
promising. I'm not sure if this config file is up-to-date (timestamp's
May 2006), but it seems to do the trick.
mkdir /var/www/merge.openafs/conf
cd /var/www/merge.openafs/conf
cp -p /afs/grand.central.org/local/src/httpd/openafs-common .

It's important that I track my changes if I'm going to modify this
file. Poking around the directory in AFS, it seems like this file was
at one time tracked by CVS. I don't have much CVS experience...
although I'm not really a git pro either! At any rate, I'm going with
git for now so I'll have *some* tracking going on.

git init
git add openafs-common
git commit -m 'Initial import'

Now we need to add a line to merge.openafs.ktdreyer.com.conf, to
actually pull this file in:
Include /var/www/merge.openafs/conf/openafs-common

And lastly, sanity-check Apache:
service httpd configtest

== Config Tweak for fset ==
If you read through that openafs-common file, you'll see the
OpenAFS.org site uses this "fset" command to parse all its pages. When
you load www.openafs.org/windows.html, mod_rewrite turns that into
openafs.org/fset.shtml/windows.html. Sneaky!

In order to do this, the site uses an Apache setting called
AcceptPathInfo.  I couldn't find where this setting is explictly
enabled within any of the openafs.org config files in AFS, so I'm
guessing it's set on the Debian host directly. Maybe this is a
difference between CentOS and Debian.

To enable AcceptPathInfo on my local site, I added the following to
merge.openafs.ktdreyer.com.conf, below the Include line from the
previous step:

<Files "*.shtml">
        AcceptPathInfo On
</Files>

service httpd restart

== Symlinks ==
Now's a good time to look at the "tools" git repo we cloned earlier.
Specifically, tools/openafs/export_htdocs. This is a command that runs
each time a git commit is "pushed" through Gerrit.

>From reading export_htdocs, there are a few symlink operations yet to
be done inside the openafs-web directory.

The first operation is to generate the release index pages, using a
script called "relindex". This script requires the HTML::Element Perl
module (perl-HTML-Tree package in EPEL 5).

cd /var/www/merge.openafs/openafs-web
/var/www/merge.openafs/tools/openafs/relindex release /release openafs
OpenAFS < release/index-template.html > release/index.html

The second operation is to symlink index.html to main.html.

ln -s main.html index.html

Be careful to avoid accidentally committing either of these symlinks
to git. Once http://gerrit.openafs.org/5573 goes through, this won't
be a problem.

== Done ==
At this point, the entire site should load from your local mirror.