[OpenAFS] Maximum number of directory created under a volume?

[Jeffrey Hutzelman]

On Tuesday, May 10, 2005 09:36:33 AM +0800 Alvin Chan 
<alvin.chan@primecreation.com> wrote:

> Hi all,
>
> I would like to ask if there is a maximum number of directory that can be
> created under a volume. I've tried to created directories under a volume
> and the last one that can be created is the 64435th. And after that an
> error mesage of "Cannot create directory '/xxxx' : File too large". But
> there's certainly still lots of space left. So is 64435 the number limit
> and why is it so?

There is a limit to the number of entries that can be created in any given 
directory.  The limit depends on the length of the filenames; if they are 
all sufficiently short, the limit is around 64K; longer filenames result in 
lower limits.

This limit is imposed by the on-disk and on-the-wire structure of AFS 
directories.  Allowing a larger number of entries would require a change to 
these structures, which would render them incompatible with existing 
fileservers and clients.  A transition to a new format is possible, but 
will be a significant undertaking requiring that care be taken to maintain 
as much backward compatibility as possible.  We'll get there, eventually, 
but it will take time.


I would suggest that if you have a directory containing on the order of 64K 
entries, it is time to stop and reconsider your filesystem layout.  While 
there are certainly legitimate reasons to need that number of files in a 
single directory, they are quite uncommon, and usually it is appropriate to 
consider a deeper structure.  Bear in mind that while AFS's directories are 
hash tables, most other filesystems use linear directories, in which the 
time required to locate, insert, or delete an entry in a directory that 
size can be quite significant.

-- Jeffrey T. Hutzelman (N3NHS) <jhutz+@cmu.edu>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA