[OpenAFS] buildbot and packages

[Doug Hirsch]

Troy,

If you set this up, I'm willing to be your guinea pig.  It'll cost you
enough support and/or documentation to get me over initial learning
curve.

Doug

On 9/15/12, Troy Benjegerdes <hozer@hozed.org> wrote:
> Sometimes I think we get hung up on 'good testing' vs having *something*.
>
> The last time I worked for someone else, it was writing test code for
> Cray's
> supercomputer systems. You don't get much more complex than a machine
> with 30,000 cores in which 'acceptable' performance is defined as 'pushing
> the system to the point right before it collapses into an unusable heap',
> and it's got to run a workload of hundreds of thousands of the world's most
> complex and numerically sensitive computational codes.
>
> And I'd hazard a guess that 3/4 of the system problems were with the
> filesystem
> (Lustre most often). I've also heard a pretty good argument that the reason
>
> Cray went bankrupt a couple of times is they over-tested. If you did get a
> machine back in the YMP days, it was very well tested, but the price showed
>
> it, and clusters ate their market.
>
>
> Maybe we don't have money.. But how many users of AFS are there. I'm not
> talking
> companies, I'm talking people.. specifically, bored college students. How
> many
> people have used AFS at a major university, and might help us out doing
> manual
> testing if we give them a framework?
>
> To paraphrase the .. well.. chief cat herder .. of the most widely deployed
> operating system ever (Linux),
> "With enough QA testers, all bugs are shallow"
>
> On Fri, Sep 14, 2012 at 04:42:37PM -0500, David Boyes wrote:
>> > In this case I think you are low-balling the estimate.  To do it right
>> > it isn't
>> > sufficient to test one build against itself.  You need to test new
>> > clients
>> > against a range of old servers and vice versa in a constrained
>> > environment.
>> > It is necessary to be able to identify when a change has an adverse
>> > performance impact as well as accuracy.  There is a need to be able to
>> > introduce intentional errors at various points in the protocol.  Just
>> > the
>> > hardware costs are mid 5 digits and the software development is
>> > significantly more than that.
>>
>>  I agree --  if you were starting from scratch, you're probably right.
>>
>> But, a) I wasn't starting from scratch, so the additional equipment for
>> adding the AFS framework stuff was about what I quoted, and b) I was
>> discussing our tooling and test setup, not the general case.
>> We reused existing tooling in a number of places, and layered the AFS
>> component onto that. We do this kind of thing for other software, so we
>> had a decent baseline to start from.
>>
>> Solid QA infrastructure -- especially for complex systems -- isn't simple
>> or cheap; there we agree wholeheartedly.
>>
>>
>>
>> :??
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>