[OpenAFS] mail spool on AFS

[Nathan Rawling]

> > > Any solution with a distributed /var/spool/mail just will not
> > scale.
> > 
> > Why would it not scale?  I was toying with the idea of making each users
> > mail file in that directory a volume and then load balancing across multiple
> > servers, and being able to enforce a different quota for mail than for the
> > standard files.
> 
> Because you cannot have replicated read-write volumes, basically.  You
> are adding the complexity of AFS for zero (or near-zero) gain.

Even with NFS, I would expect that /var/spool/mail is on a different
partition from the user home directories (i.e. you can do separate quotas
on it anyhow). My understanding is that M$ Exchange allows you to do some
sort of load balancing across a SAN-enabled filestore. (Uck)

> How it works at mit: Each user is assigned a particular Mail Server.
> All mail gets delivered to the centralized set of mail servers and
> then gets deposited locally on the mail server for the particular
> receipient.  The recipient uses one of many mail clients to pull the
> mail off the mail server and deposits it into their AFS home
> directory.

By adding LDAP, or another directory service in front of this system (ex,
X.500 similar to Umich) you can easily migrate users between servers as
well. It could be implemented using flat files and DNS only, but it would
probably be a real hassle to maintain.

> The Mail Server provides a quota for how large the mail spool can be
> for a particular user.  Similarly, each user has an AFS quota.  The
> two are not necessarily related.
> 
> > So something like ~/mail/INBOX would be a workable solution?  Then having to
> > tweak and or recompile pop3/imap/etc?
> 
> Well, it depends -- who puts the mail into ~/mail/INBOX?  If the mail
> server, then no, I do not consider it workable.  If the mail client
> (pine, GNUS, your-favorite-mail-client, etc) pulls the mail off the
> mail server and stores it in ~/mail/INBOX, yes, that is perfectly
> workable.

Well, from past experience at an AFS-aware institution that used mail
delivery into AFS, it is workable. However, I don't think it's worth
it. IMHO, distributed locking through any filesystem just *does not work*,
whether it be AFS, NFS, or <shudder> something else. 

With MH-style mailboxes, we had decent mileage from the system, but
Berkeley-style (standard) mailboxes used to get corrupted all the time
from locking issues. We told users that they couldn't use certain
mailreaders because of locking issues, but if it didn't give them any
trouble in the first five minutes, they guessed it would be fine. Six
months later, they would page me at 3:00 AM wondering where their mail
went.

I even had a user using a mail notification program similar to
"xbiff" called "xbuffy" that could eat his mail spool, and corrupt the AFS
cache on the machine.

MUAs simply cannot be trusted to lock (and honor locks) reliably. Add that
to a filesystem that will calmly allow you to shoot yourself in the foot
and you get a support nightmare.

> > Excerpt from "afs-faq" on faqs.org: (not the whole section)
> [snip]
> > Seems to recommend NOT using a dedicated mail machine, but putting into
> > $home/mail or someplace, and indicates that the dedicated mail server
> > doesn't scale very well.
> 
> This FAQ is old and incomplete.  They also leave out the ability to
> access mail via POP/IMAP/etc.  Going back to the MIT solution, this is
> a modified scenario 1 however users never log into the mail servers;
> they access them via mail clients.

Yes, I actually believe that portion of the FAQ was written before IMAP
became popular (or perhaps even was written). It isn't too hard to scrape
together fetchmail or something to grab mail off of a mail server and
stuff in into your $HOME for use by your mailreader; I use this method to
limit my contact with M$ Exchange mailservers. 

For a time, I seem to remember that AFS mail delivery was popular at a
number of places, including at least the University of Massachusetts and a
few corporate places I know of. Everyone I've spoken with in the past two
years who was doing this either expressed their intention to move away
from that setup, or already have given it up.

Local locking seems to be a good deal more reliable than distributed
locking, I did not experience too many difficulties with this setup even
when $HOME was in AFS.

Nathan