[OpenAFS-devel] vos move takes ages (almost)

[Rainer Toebbicke]

Harald Barth wrote:

> 
> All these volumes are > 3GB, so they are big enough to cancel out any
> start-stop effects. I looked at the first two volumes: The 599KB/sec
> volume has 4GB in 72441 files and the 15MB/sec volume has 9GB in 696
> files. I think I see a pattern here. Calculating backwards I get 10
> files per second for the volume with many files and about 1 file per
> second for the volume with few files. 15MB/sec is probably dominated
> by data move time and 10 files per second by file create and delete
> times. 15MB/sec is not good but 10 files per second is just horrible.
> 

You didn't say which AFS that is.

The modern, multi-threaded RX has a "pacing" bug where every now and 
then one of the parties waits for roughly 0.3 seconds for a missing 
acknowledgement. The bug must be there in the lwp-version as well but 
somehow it doesn't slow things down enough. I mention this as last 
year I ran into this RX-tracing a memory-to-memory RX application and 
bypassed the problem by completely and horribly changing the ACK 
algorithm, just to show that >110MB/s is possible using RX.

On the volume with the 70k files you suffer mainly from another 
effect: the first clone increments all the link counts and fsync()s 
the link count file for every one of them. Depending on your RAID you 
won't do many more than 100-200 per second of those. After the move, 
it decrements all the link counts twice (for the clone and the 
original) fsyncing all the time, again you can calculate how long this 
will take.

In the inode fileserver there is not much you can do about that, for 
the namei fileserver I've got a patch which a few people are already 
running that batches the fsyncs - speedup factor >200 on a volume with 
1 million 30-bytes files that used to take almost 24 hours to move. 
Although we're using it on several servers now I still consider it 
experimental - let me know if you're interested.




-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Rainer Toebbicke
European Laboratory for Particle Physics(CERN) - Geneva, Switzerland
Phone: +41 22 767 8985       Fax: +41 22 767 7155