[OpenAFS] buildbot and packages

David Boyes dboyes@sinenomine.net
Fri, 14 Sep 2012 11:12:38 -0500

> > All this talk about 'reliable code for our users' is total BS until
> > 'make check' actually does some realisitic functionality tests.
> > If you can't write an automated test for a feature, they I would
> > request we consider disabling that feature.

I'm not sure this is a realistic goal in a single machine environment. For =
a realistic testing environment, you need at least 10 system images (and th=
e ability to create a lot more would be very desirable for some of the subt=
ler bugs), and the ability to control time and replay multiple streams of e=
vents in a repeatable way. It involves a separate Kerberos infrastructure, =
and a lot of other moving parts, plus a lot of scripting to build the envir=
onment , run the test, and then reset the environment for the next run. You=
 also need different types of systems, different OS levels, etc which compl=
icate the test even further.
It's not impossible, but I can say that it cost a fair amount (in the mid-t=
o-high 5 digit range) to build that environment here.=20