[OpenAFS] maximum number of files per directory??

Todd Lewis utoddl@email.unc.edu
Wed, 11 Aug 2010 10:09:52 -0400


On 08/11/2010 05:39 AM, Simon Wilkinson sent:
> But that really depends on your "real world" - we have a number of directories here that have 60,000 or so objects in them. In fact, to hit a limit as low as 16,000 entires, every filename would need to take up 4 slots, which means every name would have to be more than 80 characters long.

I once worked on a project that had ~800,000 files with names between 120
and 199 characters each. (Not in AFS, and not in the same directory!) The
first 8 characters of the files' names were held in common between large
groups of files, and performance was horrific. It wasn't until after the
project was over that we discovered that the hierarchical storage system
we were using employed a hash table to manage file names. It didn't hash
the entire file name though; to improve performance of the hashing
operation itself, it only hashed the first 6 characters. That gave us
about 200 hash buckets for 800,000 files.

Not that you should care about any of this. It's just to illustrate that
knowing (or not knowing in our case) the characteristics of your storage
system may have huge impacts on how effectively you use it.

Good luck with that.
-- 
    +--------------------------------------------------------------+
   / Todd_Lewis@unc.edu  919-445-9302  http://www.unc.edu/~utoddl /
  /       "I have never killed a man, but I have read many       /
 /      obituaries with great pleasure." - Clarence Darrow      /
+--------------------------------------------------------------+