[OpenAFS] Web Server and Tokens

[Cees de Groot]

Victor <victord@paid.com> said:
>If I have apache running, does that mean that after 24 hours, it will not be
>able to get files from /afs/cell/www/htdocs/blah.html?
>
I have a setup for Apache+related stuff (in this case, an appserver -
Intershop - running in the same logical group of daemons) running off
AFS in a way that I really like. [long post]

The basic idea is:
- Apache runs under a special AFS user that has read ACL's on all the webserver
  stuff (and the appserver ditto, with some write ACL's added for some content
  management features)
- The whole stuff is started in a separate PAG
- One of the processes of the PAG is a job that renews authorization every 6
  hours.

Now, I fooled around for a long time with custom scripts, various incarnations
of reauth, but I never really got a satisfactory solution for things like
automatic restarts (where all the processes in the group, including the reauth
process, must be taken down). I also had the feeling that I was reinventing
wheels. 

Enter daemontools (http://cr.yp.to/daemontools.html). If you like the
Unix philosophy of 'do one thing, and do it good', you'll like Bernstein's
software. If not, stop reading here. 

Daemontools consists of a superserver (a sort of init, but rather more
flexible) that monitors a directory - every directory in that directory
describes a service, you only need to supply a 'run' script to start the
service. The usual setup is to have one superserver monitor /service, and with
one of the daemontools utilities you can say 'svc -t /service/httpd' to send a
SIGTERM to whatever is running as that service (the service-specific service
monitor started for each service by the superserver will then immediately
restart the service). 

What I did with daemontools is to start a second superserver, by
specifying the appropriate command-line to do that in the service's
'run' script. This superserver is started inside a PAG; it monitors
*its* service subdirectory (/service/intershop/service), and inside
that I have three services: 
- httpd - starts apache 
- is - starts the Intershop appserver 
- reauth - reauthorizes, then sleeps 6 hours. The script then exits, which is
  noted by the service's monitoring daemon which immediately restarts it. 

Now I can do:

% svc -t /service/intershop

which kills the superserver and everything below, restarts the whole
stuff in a new PAG.

% svc -t /service/intershop/service/httpd

kills just Apache, restarts it and it'll run in the correct PAG.

Etcetera ('svc' has arguments for downing, upping, sending various signals,
it's really a nice interface).

The biggest trick is to make sure that no-one 'escapes' from its process
group. In the philosophy of 'do one thing, and do it good' there is
something to be said against server processes detaching themselves,
always with slightly varying code, etcetera - daemontools does away
with the necessity (the associated uspi-tcp suite even does away with
the necessity of opening a server socket in a network daemon), but some
servers are hard learners. You would need to patch Apache to stop it
from detaching, but (code below) you can also insert a small dynamic
link library with LD_PRELOAD if you're on an ELF system (tested on Linux
only) that intercepts the nastiest calls. Both the Intershop appserver
and Apache seem to be happy with this hack.

Since I switched to this setup, the software has been running for around a
month now without giving me any headaches whatsoever, and the longest script I
needed to write for it was a three-liner (ok, and this C code, but that's fun
and I wanted it anyway for daemontools, so it doesn't count ;-)). 

/*
 *  $Id: nodaemon.c,v 1.2 2002/01/11 21:26:57 cg Exp $
 *
 *  Simple hack to prevent processes from making themselves a daemon:
 *  - setsid(), setpgrp() are faked.
 *  - the first fork is faked.
 *
 *  Compile (with "cc -o", "cc -shared -o") to a shared lib and
 *  put it in LD_PRELOAD, e.g.: 'LD_PRELOAD=nodaemon.so httpd'
 *
 *  Cobbled up by Cees de Groot <cg@cdegroot.com>. Public domain.
 */
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <asm/unistd.h>
#include <errno.h>

__pid_t setsid(void) {
    fprintf(stderr, "suppressing setsid\n");
    return getpid();
}
int setpgrp(void) {
    fprintf(stderr, "suppressing setpgrp\n");
    return 0;
}

/* Go out of our way to make sure that fork() is only suppressed once
   in a collection of processes, otherwise bad things happen */
static int _first = 1;
static int _getfirst() {
    char *p;
    p = getenv("FORK_FIRST_HACK");
    if (p)
        return 0;
    else
        return _first;
}
static void _setfirst() {
    setenv("FORK_FIRST_HACK", "1", 1);
    _first = 0;
}

__pid_t fork(void) {
    if (_getfirst()) {
        fprintf(stderr, "Forking first time - skip\n");
        _setfirst();
        return 0;
    }
    else {
        long res;
        fprintf(stderr, "Ok, safe now\n");
        __asm__ volatile ("int $0x80" \
            : "=a" (res) \
            : "0" (__NR_fork));
        if ((unsigned long) res > (unsigned long) -125) {
            errno = -res;
            res = -1;
        }
        return (__pid_t) res;
    }
}
-- 
Cees de Groot               http://www.cdegroot.com     <cg@cdegroot.com>
GnuPG 1024D/E0989E8B 0016 F679 F38D 5946 4ECD  1986 F303 937F E098 9E8B