[OpenAFS] ZFS-on-Linux on production fileservers?

[Ken Dreyer]

On Sun, Oct 6, 2013 at 12:36 AM, Dirk Heinrichs <dirk.heinrichs@altum.de> wrote:
> Am Samstag 05 Oktober 2013, 18:16:54 schrieb Ken Dreyer:
>
>> The reason I have advocated against ZFS-on-Linux at work for our
>> fileservers is that out-of-tree modules on Linux are such a hassle.
>
> Hmm, not on Debian derivatives, thanks to DKMS.

Sure, I don't mean to imply that there aren't tools available to try
to make the process user-friendly. But I still find them painful.

For example, DKMS would require me to install the compiler and various
development libraries on every single AFS (or ZFS) system. That may be
acceptable for production systems, but it's not optimal for large
production server deployments. Moreover, it's not a good use of
resources to rebuild the same software over and over across all our
production systems. I'll grant that both of those are relatively minor
since disk space and CPU resources are getting cheaper, but at the
same time, the number of virtual machines in our environment is always
increasing.

The third reason I find DKMS painful is the issue of repeatable
builds. At work we use a binary-based OS distribution because we want
to reduce as much as possible the category of risk associated with
builds that can differ slightly from system to system. It's been my
experience that DKMS had the tendency to work or fail depending on
subtle configuration drift across systems. When considering production
servers, DKMS packages just don't have the same risk profile as a
regular binary package. From my experience, DKMS is one more area of
possible configuration entropy that I want to avoid.

I haven't use Debian enough to know how well DKMS works there. And on
Red Hat, I haven't used DKMS for a couple years. Please feel free to
take what I'm saying with a grain of salt :)

Of the the four options that I know:

1. Rebuild entirely new binaries for every new kernel version
2. DKMS
3. Use and trust Red Hat's kABI
4. Just avoid out-of-tree modules

each one has pros and cons, and none give me a warm fuzzy feeling,
particularly for something as critical as a file system.

- Ken