[OpenAFS-devel] Ideas on vos dump slowness

[Neulinger, Nathan R.]

As near as I can tell, the key is in dumpstuff.c in volser. Basically, the
dump is calling iod_Write for each tiny little piece of data that is
retrieved from the volume. Each iod_Write in turn immediately calls an
rx_Write. I tried bumping up the block size for the read in DumpFile, which
might help for a volume containing really large files, however, it doesn't
help for volumes containing lots of tiny files.

It appears to me that this generates tons of rx calls (as the code appears
to name them) in the rx portion of the code, probably each requiring
bidirectional communication of some sort. 

What I wonder is - what if iod_Write were to buffer? i.e. have iod_Write
keep a chunk of memory around to buffer up a 100k or so before it calls
rx_Write. Add a iod_Flush routine that would force iod_Write to dump out
it's buffer (this could be called at the end of the dump to insure
everything has been sent.) Whenever it fills up the buffer, go ahead and
dump that out. 

I don't know enough about rx to know if this makes sense or not, but given
that use of iod_Write appears to be isolated to dumpstuff.c I think I may
give it a shot and see what happens. 

One negative is that write errors will be delayed till when the buffer
actually gets flushed instead of returning immediately, but given that the
code doesn't distinguish between error locations currently, that doesn't
appear like a horrible problem. 

-- Nathan

------------------------------------------------------------
Nathan Neulinger                       EMail:  nneul@umr.edu
University of Missouri - Rolla         Phone: (573) 341-4841
Computing Services                       Fax: (573) 341-4216