[OpenAFS-devel] vos dump/volserver slowness/lack of buffering?

[Troy Benjegerdes]

On Mon, Jun 27, 2005 at 05:49:40PM -0400, Jeffrey Hutzelman wrote:
> 
> 
> On Monday, June 27, 2005 04:03:10 PM -0500 Troy Benjegerdes 
> <hozer@hozed.org> wrote:
> 
> >Has anyone looked into making volserver/vos dump go any faster? Is there
> >any way to coalesce multiple small files into one RPC call, or
> >pre-buffer files from disk?
> 
> This is not about multiple RPC's.  A volume dump is transferred as part of 
> one large RPC.

Is there any analog to the '-chunksize' argument to afsd? I'd guess not
if it's one big rpc.

> >I'm seeing around an average of 10MB/sec, and performance goes in the
> >toilet when the volume has lots of small files.
> 
> Volume dumps are created on the fly.  Especially with lots of small files, 
> much of the data is metadata emitted by the volserver based on the contents 
> of the vnode index, rather than actual file data.  There will also be 
> delays related to opening files and reading their contents, which will be 
> more prominent when lots of small files are involved.

The test volume was approximately 1GB.. which fits fully in the buffer
cache on the (relatively unloaded) server machine I was testing, so this
is in no way I/O bound.

I'd like to get to somewhere near 50MB/sec (which is about half of the
throughput gigabit ethernet can support).. I'm guessing it's time for
some oprofile runs..

> >This has basically prevented vos backup/etc from ever being able to go
> >fast enough to get my DLT8000 drive to stream, and I think probably
> >prematurely wore out one of my drives.
> 
> While volume dump performance is slow and certainly something we'd like to 
> improve, it is unreasonable to expect the volserver to keep up a transfer 
> rate sufficient for tape streaming given variability in network 
> performance, fileserver load, and other factors.  You should consider a 
> backup system which spools volume dumps to disk, and writes spooled dumps 
> to tape as part of a separate process which is not tightly coupled to the 
> volume dump process.

How is amanda-afs integration these days? When I first set up my cell I 
used the integrated openafs backup system because it didn't appear anything
else had a good way to deal with AFS ACL's, and I didn't feel like paying for
some big and complicated enterprise backup system that I didn't
understand.


--------------------------------------------------------------------------
Troy Benjegerdes                'da hozer'                hozer@hozed.org  

Somone asked my why I work on this free (http://www.fsf.org/philosophy/)
software stuff and not get a real job. Charles Shultz had the best answer:

"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's why
I draw cartoons. It's my life." -- Charles Shultz