[OpenAFS] buildbot and packages

Troy Benjegerdes hozer@hozed.org
Sun, 16 Sep 2012 11:28:13 -0500 

Here's my thought:

Spend your 5 start-up hours either re-installing or upgrading your Debian
system to squeeze (the current debian stable). Try doing 'apt-get install
openafs-fileserver', and then, if it works, please edit
https://bitbucket.org/dahozer/tfs/wiki/Home saying so, or if it does not,
create an issue: https://bitbucket.org/dahozer/tfs/issues/new

Also, spend a few minutes looking at time tracking tools (maybe one from
http://lifehacker.com/5362829/five-best-time+tracking-applications or try
http://rescuetime.com ), and let's see what you actually spend. 

If you can spend 2-4 hours a week installing a weekly build on both a debian
linux server, and a client on a MacOS X laptop, and then occasionally try
accessing your server and using disconnected operation, my personal opinion is
that's a huge benefit.

(Yes, I'm hand-waving over some things like needing multiple DB servers and
kerberos right now.. we'll get to that next week, or I'll just create a
principal for you on my HOZED.ORG realm)


I'd also like to ask the openafs-info list if it would be appropriate
to create a 'openafs-test' list so we can have a more focused discussion.


Some more technical in the weeds stuff:

Can you leave your debian box powered on, with a real ip, or is it behind a
NAT? My server has a real IP that might change, so I have some VPN tunnels to a
'cloud' virtual private server with a static IP.  I can probably explain how to
set up OpenVPN, but I got irritated enough setting that up that I'd rather
spend time setting up IPv6 tunnels and hacking on v6 support for AFS than
dealing with certificate creation again.


I think the biggest reason I'm leading a crusade for IPv6 support is that I
have a use case a lot like Doug, and if I can get v6 support with fully krb5
authenticated/encrypted transports, then I can forget about ever having to 
utter the word 'vpn' or create another damn openssl cert ever again. If,
for some reason, I *do* have to mess around with ssl, I'll have the directions
and documentation and all my certs in my globally accessible (and protected)
AFS directory, so I can find it.





On Sat, Sep 15, 2012 at 11:21:33PM -0700, Doug Hirsch wrote:
> Troy,
> 
> I'm unclear I've offered you anything you can actually use.  Mostly,
> I'm offering you the reality check of a non-programmer, a Macbook with
> me on the road and a stale Debian box powered down back at home.
> You'll have to steer me through downloading, installing and using
> anything that's not on a stock Mac running Mac OS 10.6.8, or bringing
> the Linux box up to whatever environment you want once I get home.
> Most of my other machines run Windows, although I have a couple of G4
> Mac mini's hanging around for fun.  Your average college student will
> not have much more to offer you, so I'm offering a chance for you to
> define what you could actually accomplish harnessing thousands of us
> "amicable zombies" with limited time, experience and resources.  If it
> will help, I'm willing to install some virtualization package on the
> Macbook, but will need guidance.  I also need to keep a lid on my time
> commitment, so assume no more than 5 hours a week from me, with an
> extra 5 hours this week to start up.  If you can make use of that, let
> me know and I'll wander over to bitbucket.org.  What I want is someone
> to talk me through getting OpenAFS going in my personal environment.
> I'm unclear how much value you'll get out of me on just five hours a
> week and 1.5 machines.  I've written proposals and defended engineers
> building test environments, among other things, but I haven't gotten
> my hands into code for many years, so I'm sure you'll be surprised by
> what cultural assumptions you discover I don't know.  I see and
> appreciate your energy and optimism, while I think you're
> underestimating what you're asking.  But if you can make something
> work with limited commitments from others, I'm happy to go along to
> see what we can contribute to the community together.
> 
> Doug
> 
> On Sat, Sep 15, 2012 at 10:34 PM, Troy Benjegerdes <hozer@hozed.org> wrote:
> > I'll buy that for a few emails.
> >
> > Let's start by having you take a look at:
> >
> > https://bitbucket.org/dahozer/tfs
> >
> > There are tabs for issues & wikis, so sign up for a bitbucket account and
> > ask some questions there, so we don't spam the -devel list with lots of
> > 'how do I xyz' questions
> >
> > For the openafs-devel list, please let the list know what resources/
> > platforms you have for testing, and I'd like to hear from the list what
> > could I write some tests for that could utilize those resources.
> >
> >
> > On Sat, Sep 15, 2012 at 09:44:07PM -0700, Doug Hirsch wrote:
> >> Troy,
> >>
> >> If you set this up, I'm willing to be your guinea pig.  It'll cost you
> >> enough support and/or documentation to get me over initial learning
> >> curve.
> >>
> >> Doug
> >>
> >> On 9/15/12, Troy Benjegerdes <hozer@hozed.org> wrote:
> >> > Sometimes I think we get hung up on 'good testing' vs having *something*.
> >> >
> >> > The last time I worked for someone else, it was writing test code for
> >> > Cray's
> >> > supercomputer systems. You don't get much more complex than a machine
> >> > with 30,000 cores in which 'acceptable' performance is defined as 'pushing
> >> > the system to the point right before it collapses into an unusable heap',
> >> > and it's got to run a workload of hundreds of thousands of the world's most
> >> > complex and numerically sensitive computational codes.
> >> >
> >> > And I'd hazard a guess that 3/4 of the system problems were with the
> >> > filesystem
> >> > (Lustre most often). I've also heard a pretty good argument that the reason
> >> >
> >> > Cray went bankrupt a couple of times is they over-tested. If you did get a
> >> > machine back in the YMP days, it was very well tested, but the price showed
> >> >
> >> > it, and clusters ate their market.
> >> >
> >> >
> >> > Maybe we don't have money.. But how many users of AFS are there. I'm not
> >> > talking
> >> > companies, I'm talking people.. specifically, bored college students. How
> >> > many
> >> > people have used AFS at a major university, and might help us out doing
> >> > manual
> >> > testing if we give them a framework?
> >> >
> >> > To paraphrase the .. well.. chief cat herder .. of the most widely deployed
> >> > operating system ever (Linux),
> >> > "With enough QA testers, all bugs are shallow"
> >> >
> >> > On Fri, Sep 14, 2012 at 04:42:37PM -0500, David Boyes wrote:
> >> >> > In this case I think you are low-balling the estimate.  To do it right
> >> >> > it isn't
> >> >> > sufficient to test one build against itself.  You need to test new
> >> >> > clients
> >> >> > against a range of old servers and vice versa in a constrained
> >> >> > environment.
> >> >> > It is necessary to be able to identify when a change has an adverse
> >> >> > performance impact as well as accuracy.  There is a need to be able to
> >> >> > introduce intentional errors at various points in the protocol.  Just
> >> >> > the
> >> >> > hardware costs are mid 5 digits and the software development is
> >> >> > significantly more than that.
> >> >>
> >> >>  I agree --  if you were starting from scratch, you're probably right.
> >> >>
> >> >> But, a) I wasn't starting from scratch, so the additional equipment for
> >> >> adding the AFS framework stuff was about what I quoted, and b) I was
> >> >> discussing our tooling and test setup, not the general case.
> >> >> We reused existing tooling in a number of places, and layered the AFS
> >> >> component onto that. We do this kind of thing for other software, so we
> >> >> had a decent baseline to start from.
> >> >>
> >> >> Solid QA infrastructure -- especially for complex systems -- isn't simple
> >> >> or cheap; there we agree wholeheartedly.
> >> >>
> >> >>
> >> >>
> >> >> :??
> >> > _______________________________________________
> >> > OpenAFS-info mailing list
> >> > OpenAFS-info@openafs.org
> >> > https://lists.openafs.org/mailman/listinfo/openafs-info
> >> >
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info