[OpenAFS] Re: Speed up pts actions

[Andrew Deason]

On Thu, 18 Nov 2010 16:03:50 +0100 (CET)
Harald Barth <haba@kth.se> wrote:

> > It seems like it commits each change to all protection servers
> > before continuing. 
> 
> As the ptservers together form a distributed database, I curently can
> not imagine (please correct me if I am wrong) how this could be done
> without implementing _a_lot_ more "database stuff" like checkpoints
> and rollback.

ubik already has enough functionality itself to do this (you can
rollback by aborting the ubik transaction), but we don't have any RPCs
that let you update so many entries at once. A split RPC to do some kind
of 'bulk update' mechanism doesn't sound difficult, but I haven't really
thought it through.

As for current existing solutions, though, if you can afford some
downtime and want to just add a bunch of entries at once, you could
populate the ptdb offline with the pt_util utility. It operates directly
on the ptdb database file, so you should be able to copy the ptdb, run
'pt_util -w -datafile /foo/bar -prdb /copy/of/db', and shutdown the
ptserver, copy the updated db back in place, and turn the ptserver back
on.

Using pt_util to modify the database isn't the most-used or rigorously
tested tool, though, so I'd test it to make sure it does what you want,
first. I also don't remember the exact format it wants for input, but I
think it matches how prdb outputs data (without the -w flag).

That would certainly be faster than populating the db over RPCs. You
could also just only have one dbserver while you are populating the
database, which would be faster since it does not need to propagate
updates.

There's also some pthreaded ubik enhancements that _might_ make this
faster, but they are new, and really intended for better read
performance. So I don't know how much that would really help you.

-- 
Andrew Deason
adeason@sinenomine.net