[OpenAFS] humungous disk caches
Bryan Bayerdorffer
bryan.bayerdorffer@spd.analog.com
Thu, 13 Mar 2003 12:57:44 -0600
Derek Atkins wrote:
> Honestly, I think you'll run into a bunch of other hashtable
> limitations when you get that big. I also think you'll find that the
> cache (as it's currently implemented) works better if it's smaller.
>
> How big are these files that you're dealing with?
1 - 10GB and there are many of them accessed in an unpredictable pattern. They
occupy a total space far larger than the 128G I have allocated for cache. There
are also thousands of smaller files of O(1KB).
I've thought of just rsyncing the files on demand at the application level, but
it's much more appealing to do so in the filesystem if I can get comparable or
better performance. With rsync I may end up transferring parts of files that I
don't need, and I have to worry about expiring files from the cache.
>>Our apps are characterized by very large files that are read often and modified
>>only rarely, so I'd like to see if a large disk cache (128GB) will improve
>>performance.
--
.. ..-. ..- -.-. .- -. .-. . .- -.. - .... .. ... --. . - .- .-.. .. ..-. .
Bryan Bayerdorffer bryan@meatspace.net bryan@spd.analog.com
(Wit's End Computation Center) (Analog Devices)
"Sing, O goddess, the anger of Achilles son of Peleus, that brought countless
ills upon the Achaeans." -- Homer
"Donuts---is there anything they can't do?" -- Homer