[OpenAFS] humungous disk caches

[Bryan Bayerdorffer]

Derek Atkins wrote:
> Honestly, I think you'll run into a bunch of other hashtable
> limitations when you get that big.  I also think you'll find that the
> cache (as it's currently implemented) works better if it's smaller.
> 
> How big are these files that you're dealing with?

1 - 10GB  and there are many of them accessed in an unpredictable pattern.  They 
occupy a total space far larger than the 128G I have allocated for cache. There 
are also thousands of smaller files of O(1KB).

I've thought of just rsyncing the files on demand at the application level, but 
it's much more appealing to do so in the filesystem if I can get comparable or 
better performance.  With rsync I may end up transferring parts of files that I 
don't need, and I have to worry about expiring files from the cache.

>>Our apps are characterized by very large files that are read often and modified
>>only rarely, so I'd like to see if a large disk cache (128GB) will improve
>>performance.


-- 
  .. ..-. ..- -.-. .- -. .-. . .- -.. - .... .. ... --. . - .- .-.. .. ..-. .
Bryan Bayerdorffer       bryan@meatspace.net              bryan@spd.analog.com
                    (Wit's End Computation Center)           (Analog Devices)

"Sing, O goddess, the anger of Achilles son of Peleus, that brought countless
ills upon the Achaeans."  -- Homer

"Donuts---is there anything they can't do?"  -- Homer