[OpenAFS] Mail Storage in OpenAFS ( Was Failover )

[Robert Banz]

> Sure, a bunch of clients talking to the same directory has scalability
> problems, but if I've got a mailbox that is that is huge enough to have 
> these problems, it's not something I'm going to be able to effectively read
> anyway. Heck, my imap client (backened by afs) only checks mail every 5
> minutes anyway.

That is true.  However, in a careful design of the backend mail storage 
format -- and how you have your "clients" situated, can mitigate the 
callback issue.

First, if you're storing your mail in a directory-based structure, the 
times that would trigger a change in the directory (and hense a callback 
break) would be:

	* When new mail is delivered
	* When status on a mail message changes
	* When mail is copied to a folder
	* When mail is removed from a folder

All in all, even on a busy mailbox, these things don't happen "all that 
often"

In our environment, our IMAP/POP servers (that make up for 99% of the 
mail clients -- not many people use 'pine' anymore) are behind an IP 
load balancer with client IP connection affinity enabled, so, all of one 
person's IMAP/POP sessions from a particular machine tend to go against 
the same server over and over -- really taking advantage of the local 
AFS cache on the machines.  (We had recently discovered that our 
performance bottleneck on the mail reading machines was the performance 
of the local disk, and have moved to a very large in-memory cache). 
We've seen our IMAP/POP servers be able to handle 700-800 imap reading 
sessions; so far the real limiter is available memory.

We also don't see an extraordinary amount of load on the fileservers.

Other than having to maintain the code for the "odd" maildir c-client 
driver we've written, the only big drawback for us so far is that it 
increases the # of individual files on our fileservers, which, increase 
the salvage time in the instance of a crash -- which shouldn't happen 
anyhow ;)

If I went to a 'monolithic' architecture such as cyrus, I'd be 
frustrated with the inability to "manage" my storage as I can do with 
AFS.  Being able to move users' data volumes from server to server to 
balance load, space usage, and perform maintenance ... well, I'm pretty 
addicted to it.

-rob