[OpenAFS] buildbot and packages
Jeffrey Altman
jaltman@your-file-system.com
Fri, 14 Sep 2012 17:22:36 -0400
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig6DDB93938A7729D300AA0A00
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
On 9/14/2012 12:12 PM, David Boyes wrote:
>>> All this talk about 'reliable code for our users' is total BS until
>>> 'make check' actually does some realisitic functionality tests.
>>> If you can't write an automated test for a feature, they I would
>>> request we consider disabling that feature.
>=20
> I'm not sure this is a realistic goal in a single machine environment. =
For a realistic testing environment, you need at least 10 system images (=
and the ability to create a lot more would be very desirable for some of =
the subtler bugs), and the ability to control time and replay multiple st=
reams of events in a repeatable way. It involves a separate Kerberos infr=
astructure, and a lot of other moving parts, plus a lot of scripting to b=
uild the environment , run the test, and then reset the environment for t=
he next run. You also need different types of systems, different OS level=
s, etc which complicate the test even further.
> =20
> It's not impossible, but I can say that it cost a fair amount (in the m=
id-to-high 5 digit range) to build that environment here.=20
David,
In this case I think you are low-balling the estimate. To do it right
it isn't sufficient to test one build against itself. You need to test
new clients against a range of old servers and vice versa in a
constrained environment. It is necessary to be able to identify when a
change has an adverse performance impact as well as accuracy. There is
a need to be able to introduce intentional errors at various points in
the protocol. Just the hardware costs are mid 5 digits and the software
development is significantly more than that.
Jeffrey Altman
--------------enig6DDB93938A7729D300AA0A00
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
iQEcBAEBAgAGBQJQU6AfAAoJENxm1CNJffh4WBcH/0VRDx/Vdxj4Q77gM8oDVZKS
SAjhFUnhhJnG/DIH0frmQr7vmlOMFJkNHhFASTqVcWdHc0mCVm70bSJfSvpau7Ij
kvbkV65V+zBD4pNfm4wLGXlJko+/m8rdJI455ZlGls33W0iBODEPo9mxxtdASWdd
mOWlaUo/4zFjzQtixvHbq7SL+FpaQAvMLH/R4Vii6nzCzEI5kAFWrzyGxYSoDSiY
MyGc7aS2QtAVuyNAgh+oNtcaC13yQda4nkysZWD5+WQtMRdSTHQmeVnApWKY7itm
8psRaiqRhYoDaEJdaQPKvmv0Yq/yZezd1DZl3ld59QglgM2fIB0W3sGE+jwyCGc=
=CFcK
-----END PGP SIGNATURE-----
--------------enig6DDB93938A7729D300AA0A00--