[OpenAFS] Re: Introduction and quick reference question

[Andrew Deason]

On Thu, 27 Sep 2012 15:41:35 -0700
Timothy Balcer <timothy@telmate.com> wrote:

> 1) How many files per volume?

Non-directories are stored in an "index" file, where each entry takes up
64 bytes. In previous versions we had issues when that file grew larger
than 2GiB, due to how we were storing file offsets. So that would be...
33,554,432 files, I think? So, with older versions of the OpenAFS
fileserver, you may hit issues when you store more than that many files.
With OpenAFS 1.6.1, you should no longer be limited by that, since we
should be handling index file offsets correctly (using 64 bits).

For directories, the situation is the same, but each entry takes 256
bytes in a separate index file, so the previous limit was 8,388,608
directories. And now you should no longer be limited.

The next limit you may hit is that the vnode number for any file is
32-bits wide, and every directory or non-directory in a volume must have
a vnode number. Currently odd vnode numbers are directories and even
vnode numbers are non-directories, so you are limited to 2^31
directories, and 2^31 non-directories. There may be other limits you hit
before that, though.

Non-directories are regular files, symlinks, and volume mountpoints
(even though volume mountpoints appear as directories to the client);
everything except for directories.

> 2) Any limits to how many volumes can be mounted on a client under
> other volumes? (aka, /afs/foo.com/volume/1,2,3...N i.e. how wide can a
> directory be?

I'm not sure what you mean by this. Do you mean how many levels deep can
you go with /afs/foo.com/volume/1/2/3, where 'volume', '1', '2', and '3'
are all mountpoints to volumes?

If so, no limit, I don't think. But you are limited to 2^32 volumes in a
single cell, though you'll probably hit issues sooner than that. Some
things have had trouble with volume IDs over 2^31-1, but those bugs I
think are all gone by now. Volume IDs do not start at 0, and I assume
something will break when you try to roll over the volume ID counter.
However, when you create a volume, you can manually specify your own
volume ID.

> Also how deep can a volume be (how many levels of directories)?

No limit, I think, aside from total dir limits.

> 3) How many directories per directory (excluding volume limits, if not
> applicable),

I'm not sure how this differs from the questions above. This is the same
as the 64k/32k/etc limit you mentioned above.

> I'm planning a fairly rigorous application and I need to know upper
> limits on details like these. Where can I find this info? Or did I
> just miss it when I RTFM'd?

The directory entry limit is documented because it's an arbitrary limit
we impose due to the directory storage format. The other limits you're
talking about don't have any arbitrary limits (aside from a 32-bit wide
vnode), so we may not be sure what the limits are, since whatever limits
were not chosen deliberately. So, that's why some of them can be hard to
find.

-- 
Andrew Deason
adeason@sinenomine.net