[OpenAFS] Re: Avoiding NTFS and FAT

[Mike Polek]

I guess I would be a member of the "really determined" club.

For anyone interested, I run diskless clients using only OpenAFS.
The root filesystem is actually a tmpfs filesystem, so that it
can handle everything in /dev, but you could probably do it
the other way around and just mount a tmpfs filesystem for /dev
and a couple of other places that need it. The servers PXE
boot from the network and require no local disk storage.

Using mount --bind makes it pretty easy to put things where
you need them:

[root@host1 /]# df
Filesystem           1k-blocks      Used Available Use% Mounted on
tmpfs                    92160     23456     68704  26% /
none                   1032348         0   1032348   0% /dev/shm

-- The tmpfs filesystem only takes about 25M of memory.

[root@host1 /]# cat /proc/mounts
rootfs / rootfs rw 0 0
tmpfs / tmpfs rw 0 0
AFS /afs afs rw 0 0
AFS /bin afs rw 0 0
AFS /lib afs rw 0 0
AFS /opt afs rw 0 0
AFS /sbin afs rw 0 0
AFS /usr afs rw 0 0
AFS /var/lib/rpm afs rw 0 0
/proc /proc proc rw 0 0
usbdevfs /proc/bus/usb usbdevfs rw 0 0
none /dev/pts devpts rw 0 0
none /dev/shm tmpfs rw 0 0

I have it set up so that most of the OS is
in AFS and readonly. The root filesystem is in
memory and is mostly /dev, /etc, /root, /tmp, and /var
(except /var/lib/rpm, which is pretty big and can be
readonly). As long as you don't really need to store
much stuff locally, this scenario can work pretty well.
We replicate all of the RO volumes to four servers,
so the diskless clients are not dependent on any particular
AFS server being up and running.

If you want to have RW storage not in memory, you could also
mount /home and/or /var to some RW volume someplace for
the individual machine.

Currently our diskless clients are still running RedHat 7.3.
We use them for apache servers, image processors, servlet
runners, etc. As Jeff mentions, it does take some work to
get things going, and there lots of little niggly details
to work out, but once things are set up, it makes bringing
up a new machine a snap. We use blades for image processors
to save on power and footprint.

Also, it's even possible to set it up where you have afs disk
servers that get their OS from the network, so 100% of the disk
space is available for the /vicepX partitions. As long as a few
core servers boot from a local disk and have the volumes needed
to load/serve the OS to other servers, you still get the
reliability/redundancy benefits, and you don't have to do
much configuration to add a new disk server to the cell.

(I haven't done it yet, but having network booting disk servers
  could make upgrading the OS much easier in the future.
  Right now, I'd have to move all the volumes off to upgrade
  because the current partitions aren't really big enough.
  I'm considering bypassing all of that and putting the
  newer OS in AFS and just switching most of the AFS servers
  to network bootable.)


Mike Polek
Pictage, Inc.

 >-------------------------------------------------------------------
 >Message: 9
 >Date: Fri, 28 Jan 2005 14:51:38 -0500
 >From: "Jeffrey B. Woodward" <Jeffrey.B.Woodward@Dartmouth.EDU> To: 
 >penafs-info@openafs.org
 >Subject: Re: [OpenAFS] Avoiding NTFS and FAT
 >
 >Derek Atkins wrote:
 >
 >>>ajpearce <ajpearceuk@gmail.com> writes:
 >>>
 >>>
 >>>
 >>
 >>>>>I thought NTFS was in play because OpenAFS wouldn't let me use a ext3
 >>>>>drive mounted using a IFS driver (ext2anywhere).
 >>>>>
 >>>>>"AFS's permissions model is neither that
 >>>>>of Unix nor that of Windows NTFS.  (Under certain circumstances, that 
 >makes
 >>>>>it the worst of both worlds...)"
 >>>>>
 >>>>>^ so I see that AFS isn't the answer to serving UNIX compatible
 >>>>>filesystems to my linux thinclient. In other words, I can't store all
 >>>>>types of unix files on an OpenAFS volume. I couldn't, for example use
 >>>>>OpenAFS to serve Linux ThinClients - re: ltsp.org ?
 >>>>>
 >>>>>
 >>
 >>>
 >>>I dont understand this statement..  What exactly are you trying to
 >>>"serve" to your thinclients?  I don't understand what you mean by
 >>>"UNIX compatible filesystems".
 >>>
 >>>Are you trying to serve a root filesystem?  Are you trying to serve
 >>>the /usr filesystem?  Are you trying to server applications and/or
 >>>homedirectory filesystems?  Any (but the first) is certainly possible
 >>>using AFS.  But it's still very unclear what you're trying to do and
 >>>what you mean by the buzzwords you're throwing around.
 >>>
 >>>-derek
 >>>
 >
 >
 >
 >Actually, for the really determined, you can even use OpenAFS for your
 >root filesystem...takes a little work, but it has been done.
 >
 >-Jeff
 >