[OpenAFS-devel] Re: Adding utility to do a volume dump table of contents...

[Jeffrey Hutzelman]

On Fri, 10 May 2002, Mitch Collinsworth wrote:

>
> > >I'm sorry to say we wrote tocvol last summer, though we called it list.
> > >But here's the interesting part.  We recently started testing our code
> > >on our production AFS servers and found that for large volume sets of
> > >20-30 GB and lots of files, "list" can an inordinate amount of time.
> > >For most of our volume sets it's not too bad, but in our worst case it
> > >continues to run for a few hours after the actual dump has finished.
> > >We've been looking at our code to try to find out why but haven't yet
> > >found an obvious problem.  The programmer who did our coding is right
> > >now looking at tocvol to see if he can find any obvious algorithmical
> > >difference, in hopes that you may have solved our problem.  :-)
> >
> > I have noticed in dumptool (since you say "list" is based off of it) there
> > are plenty of places for performance enhancements (I know it doesn't perform
> > that well with larger volume dumps).  It's been on my "list" (ha ha) to
> > look at, but you know how that goes ...
>
> Certainly do..  The note I received from our programmer says that
> Marc's code looks cleaner but that there are no obvious algorithmical
> differences other than that tocvol prints its list as it goes, while
> list waits and prints its list after if finishes traversing the dump.
> He tested them both on a ~ 1 GB volume and they took roughly the same
> time, ~ 58 seconds.
>
> In order to avoid adding any further delay to our efforts to collaborate
> on this, I've put our source tree where you can get it from either of
> the following:
>
> /afs/msc.cornell.edu/common/ftp/pub/amanda-afs
> ftp://ftp.ccmr.cornell.edu/pub/amanda-afs
>
> There's plenty yet to be done in documenting our work better, but for
> now go ahead and look at it and ask me questions about anything that's
> non-obvious.

<sigh>  I suppose I should get around to formally releasing my dumpscan
tools.  Dumpscan is a library for parsing and generating AFS volume dumps,
and a variety of tools that use them, including a dump lister, a tool to
extract dump contents into the filesystem, and the beginnings of a tool to
make systematic changes to a dump.  I expect to add other tools in the
future.

As I said, I need to get around to doing a formal release, but in the
meantime the 1.00 version can be found in my sandbox in
/afs/cs.cmu.edu/project/systems-jhutz/dumpscan

-- Jeffrey T. Hutzelman (N3NHS) <jhutz+@cmu.edu>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA