[OpenAFS] Re: bonnie++ on OpenAFS

[Hartmut Reuter]

Achim Gsell wrote:
>
> On Nov 23, 2010, at 12:15 AM, Simon Wilkinson wrote:
>
>>
>> On 22 Nov 2010, at 23:06, Achim Gsell wrote:
>>>
>>> 3.) But if I first open 8 files and - after this is done - start writing
>>> to these files sequentially, the problem occurs. The difference to 1.)
>>> and 2.) is, that I have these 8 open files while the test is running.
>>> This simulates the "putc-test" of bonnie++ more or less:
>>
>> AFS is a write-on-close filesystem, so holding all of these files open
>> means that it is trying really hard not to flush any data back to the
>> fileserver. However, at some point the cache fills, and it has to start
>> writing data back. In 1.4, we make some really bad choices about which data
>> to write back, and so we end up thrashing the cache. With Marc Dionne's
>> work in 1.5, we at least have the ability to make better choices, but
>> nobody has really looked in detail at what happens when the cache fills, as
>> the best solution is to avoid it happening in the first place!
>
> Sounds reasonable. But I have the same problem with a 9GB disk-cache, a 1GB
> disk-cache, 1GB mem-cache and a 256kB mem-cache: I can write 6 GB pretty fast
> then performance drops to<  3MB/s ...

We are always using memcache with only 64 or 256 MB, but I have seen this 
problem, too. I think it's on the server side: Today's server have a lot of 
memory and the data are written into the buffers first. Only when the buffers 
rach the limit the operating system starts to really sync them out to the disks. 
And with this huge amount of buffers you regularly see for some time the 
performance going down. I suppose that during the sync the fileserver's writes 
are hanging.

Hartmut

>
> So long
>
> Achim
>
>
> _______________________________________________ OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info


-- 
-----------------------------------------------------------------
Hartmut Reuter                  e-mail 		reuter@rzg.mpg.de
			   	phone 		 +49-89-3299-1328
			   	fax   		 +49-89-3299-1301
RZG (Rechenzentrum Garching)   	web    http://www.rzg.mpg.de/~hwr
Computing Center of the Max-Planck-Gesellschaft (MPG) and the
Institut fuer Plasmaphysik (IPP)
-----------------------------------------------------------------