[OpenAFS] Number of files in an OpenAFS volume...

[Jeffrey Altman]

This is a cryptographically signed message in MIME format.

--------------ms080204020803010706030502
Content-Type: multipart/mixed;
 boundary="------------BB9668270E43EDDB3BE1CECA"

This is a multi-part message in MIME format.
--------------BB9668270E43EDDB3BE1CECA
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 7/4/2016 6:55 AM, Matthew Lowy wrote:
> Hello,
>=20
> We have a number of OpenAFS volumes that serve as storage for (public)
> mirrors and one of them is misbehaving when updated from upstream - the=

> error indicates we've reached the limit of file names allowed in a volu=
me.

In a volume or a directory?

The theoretical limit of directories in a volume is 2^30 and
non-directories in a volume is 2^30.  There have been incomplete efforts
to raise those limits by treating signed values as unsigned values but I
wouldn't count on them.

> The limit I am seeing is not compatible with my understanding of how
> OpenAFS handles file names in a directory. I've seen in the mail list
> archives the statements about how many file names can fit, that there
> are 64k slots and a file name < 16 in length occupies one slot, a file
> name from 16 to 32 characters long occupies two slots and so on. The
> earliest reference I've found is at
> http://lists.openafs.org/pipermail/openafs-info/2002-September/005812.h=
tml

Your understanding is roughly correct except that the numbers Todd
specified are wrong.   The actual number are determined by this function:=


/* Find out how many entries are required to store a name. */
int
afs_dir_NameBlobs(char *name)
{
    int i;
    i =3D strlen(name) + 1;
    return 1 + ((i + 15) >> 5);
}

A slot contains per file metadata followed by name data.  When multiple
slots all of the space in the 2nd and subsequent slots are used for file
name data.

> However...
>=20
> The directory concerned has more than 21,000 files in it, almost all of=

> them have names exceeding 52 characters... as at today there are
> 1,220,000 characters in filenames in that directory. Even assuming they=

> pack down perfectly into directory name slots that's over 76,000
> slots... and working them out using the rule above indicates that the
> directory is using over 87,000 slots. These are both significantly abov=
e
> 64k.
>=20
> I don't know if I'm misinterpreting the information in the OpenAFS
> archive or if the information is out of date - but I've not found
> anything that fundamentally is different from the information in the
> archive and I'm looking at a volume that seems to break the limits.=20

The AFS3 directory format is part of the wire protocol as it is shared
by both the file server and the clients.

> I'd really benefit from understanding what's going on ... how we appear=
 to
> be getting more file name information into a directory than should be
> possible.
>=20
> /mirror.ox.ac.uk/sites/archive.ubuntu.com/ubuntu/pool/main/l/linux$ ls
> |wc -l
> 21731

This number is within the existing limits.

> /mirror.ox.ac.uk/sites/archive.ubuntu.com/ubuntu/pool/main/l/linux$ ls
> |wc -c
> 1250894

File names of length 52 through 70 characters require three slots.  If
all of the file names are of length 60 and are perfectly packed they
would require 62545 slots which is very close to the limit.

> This is one directory in a mirror of archive.ubuntu.com so you can see
> the contents from (e.g)
> https://launchpad.net/ubuntu/+mirror/mirror.ox.ac.uk-archive which
> points to the presentation of our mirror. The number of files has
> recently gone up because of upstream changes.

The directory size restrictions are one of the reasons that /afs cannot
be used for a large number of applications.  The AuriStor File System
implements a new directory format which is understood only by AuriStor
clients.  This format permits directories to grow to store an unlimited
number of entries.  However, the AuriStor file servers currently apply
an artificial limit of approximately 20 million entries.

More details on the AuriStor File System can be obtained at

  https://www.auristor.com/openafs/migrate-to-auristor/

Jeffrey Altman



--------------BB9668270E43EDDB3BE1CECA
Content-Type: text/x-vcard; charset=utf-8;
 name="jaltman.vcf"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="jaltman.vcf"

begin:vcard
fn:Jeffrey Altman
n:Altman;Jeffrey
org:AuriStor, Inc.
adr:Suite 6B;;255 West 94Th Street;New York;New York;10025-6985;United St=
ates
email;internet:jaltman@auristor.com
title:Founder and CEO
tel;work:+1-212-769-9018
note;quoted-printable:LinkedIn: https://www.linkedin.com/in/jeffreyaltman=
=3D0D=3D0A=3D
	Skype: jeffrey.e.altman=3D0D=3D0A=3D
=09
url:https://www.auristor.com/
version:2.1
end:vcard


--------------BB9668270E43EDDB3BE1CECA--

--------------ms080204020803010706030502
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC
DEkwggYFMIIE7aADAgECAhAxSdDYnMC0s7K+ddH7ywANMA0GCSqGSIb3DQEBCwUAMIGmMQsw
CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNVBAsTFlN5
bWFudGVjIFRydXN0IE5ldHdvcmsxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3
MDUGA1UEAxMuU3ltYW50ZWMgQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBH
NTAeFw0xNTExMDEwMDAwMDBaFw0xNjExMDEyMzU5NTlaMIGnMS4wLAYDVQQDDCVQZXJzb25h
IE5vdCBWYWxpZGF0ZWQgLSAxNDQ2NDA0MDI1NjI1MSMwIQYJKoZIhvcNAQkBFhRqYWx0bWFu
QGF1cmlzdG9yLmNvbTEPMA0GA1UECwwGUy9NSU1FMR4wHAYDVQQLDBVQZXJzb25hIE5vdCBW
YWxpZGF0ZWQxHzAdBgNVBAsMFlN5bWFudGVjIFRydXN0IE5ldHdvcmswggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQC2cv6bENZULsb1FNyEBI47G2kA7Rogocg5u0qnQuDMCCNH
DnXkI62H2z/464AS8AGp4FcZIdvCPYp0POFlOl2XEiyA59FEDi+s3b33nLrnVOa4esw2NFBi
ZvwHvniUjgwWSUcS5V5+VhXeIKBL1U1NtMtB2XEVnc72RiQZB3KqzlS9GUvtSTdmCc6ULD/t
P009yCsBqY6NR/nug5NtUja2N+xUC2l8coYU1ingj/M4+fT8KcSq5t8laj0E+3X9ZPYlN9in
L364hXB1b+EHfxp0F0wZdsCapkEKV2VN16S1Ee5rccJIMaAJxrybK7NdI36Es/XqWlQzSeVN
wUYdwejrAgMBAAGjggIqMIICJjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIFoDAgBgNV
HSUBAf8EFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwHQYDVR0OBBYEFInhroYG7TrlBeOJmwRc
a0rfA3jtMB8GA1UdEQQYMBaBFGphbHRtYW5AYXVyaXN0b3IuY29tMGwGA1UdIARlMGMwYQYL
YIZIAYb4RQEHFwEwUjAmBggrBgEFBQcCARYaaHR0cDovL3d3dy5zeW1hdXRoLmNvbS9jcHMw
KAYIKwYBBQUHAgIwHBoaaHR0cDovL3d3dy5zeW1hdXRoLmNvbS9ycGEwXQYDVR0fBFYwVDBS
oFCgToZMaHR0cDovL3BraS1jcmwuc3ltYXV0aC5jb20vY2FfNTdkZTdhMjM4ZDQ1ZDhkNGZj
NzFmOGE1YzZiODFjOTMvTGF0ZXN0Q1JMLmNybDBOBggrBgEFBQcBAQRCMEAwPgYIKwYBBQUH
MAKGMmh0dHA6Ly9jYWNlci5zeW1hdXRoLmNvbS9tcGtpL3N5bWNjMWluZHN1YmNhZzUuY3J0
MB8GA1UdIwQYMBaAFGcZtj2lebszYNgtU9OMCT0HrBhwMCsGCmCGSAGG+EUBEAMEHTAbBhJg
hkgBhvhFARABAgIEAa3ukhMWBTEwOTIyMDkGCmCGSAGG+EUBEAUEKzApAgEAFiRhSFIwY0hN
Nkx5OXdhMmt0Y21FdWMzbHRZWFYwYUM1amIyMD0wDQYJKoZIhvcNAQELBQADggEBAHhpoe3z
j0uxbWFz4Q7F2KyRJTgREbQ+imVv3ibbd2uvnckJ+vTNJuFoaORXKTt3B8TUT8rBTwXm35D5
K4DOrKMVS0iyR9PDobLpjQM8FcIGUdbGWeZZq/1zoWhi3RtFNzZaSKXjwiQUUWYTZUE7rUmj
qu7fiACksPAAdyG12FJtk1pdVByqmaFC75/z2Qs/jVjyySpy2SRg8wlpqM2h8tl9SGska/BT
gXLHJMhKrHQvBi+9Xo3MrmJA5Z3f5vcheoOAoM5/ScHBUXWDG0+NnEIb340By0adU823pF9c
K4gxI0ZJNID+eaT0XCqg47QFOZYZUlm3s4rDi1l1iPIHekwwggY8MIIFJKADAgECAhAHAqIa
hbhLZZ4YCm7m9aNlMA0GCSqGSIb3DQEBCwUAMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMO
VmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNV
BAsTMShjKSAxOTk5IFZlcmlTaWduLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkx
RTBDBgNVBAMTPFZlcmlTaWduIENsYXNzIDEgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlv
biBBdXRob3JpdHkgLSBHMzAeFw0xNTEwMDEwMDAwMDBaFw0yNTA5MzAyMzU5NTlaMIGmMQsw
CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNVBAsTFlN5
bWFudGVjIFRydXN0IE5ldHdvcmsxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3
MDUGA1UEAxMuU3ltYW50ZWMgQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBH
NTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANb7DTFJIsAMuu9OLRe4tDDlNNKs
Ce7JiQBd9K4TEbMTjlXPIhioIAam/SXLaJS0oIKQDm6WkkTuFbm07PrPeT1p1Jwf3CCaDijV
DbXGnjrx1GUZwIQzTY7aivHWJ7kDoNH6KJyCk2z3DYV9W+lOmWL+k0LS7j6zcVeGl0bL3w3h
xoRasw2P9fQQigVdn2hG7AiwWEKC9r4tEEamJAsn/pgUU4OTgtvqwD9PolhhtUtyaRJfM1n2
+bNMAGTOhcWGkgxuHOsoz3GpkKl0mXQk60jhDl1oEqgBZujumrIv+D3Nt3gkzqVgfOgWPUnx
B7ozvjIrwmejFsdvwNJalATCa0UCAwEAAaOCAj4wggI6MDcGCCsGAQUFBwEBBCswKTAnBggr
BgEFBQcwAYYbaHR0cDovL3BraS1vY3NwLnN5bWF1dGguY29tMBIGA1UdEwEB/wQIMAYBAf8C
AQAwbAYDVR0gBGUwYzBhBgtghkgBhvhFAQcXATBSMCYGCCsGAQUFBwIBFhpodHRwOi8vd3d3
LnN5bWF1dGguY29tL2NwczAoBggrBgEFBQcCAjAcGhpodHRwOi8vd3d3LnN5bWF1dGguY29t
L3JwYTAvBgNVHR8EKDAmMCSgIqAghh5odHRwOi8vcy5zeW1jYi5jb20vcGNhMS1nMy5jcmww
DgYDVR0PAQH/BAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFTeW1hbnRlY1BLSS0y
LTIxNzAdBgNVHQ4EFgQUZxm2PaV5uzNg2C1T04wJPQesGHAwgfEGA1UdIwSB6TCB5qGB0KSB
zTCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMTk5OSBWZXJpU2lnbiwgSW5j
LiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJpU2lnbiBDbGFzcyAx
IFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzOCEQCLW3VWhFSF
CwDPrzhIzrGkMA0GCSqGSIb3DQEBCwUAA4IBAQBGGeQndTu+r+LaohRgjx5EkIFpK2NJNY3p
drqfmI8TgdfL/GUb+A5j3pQodPvjb9jJKnoOVKaDrilK9itR/T04ErvdY0X7ZMw+VIZ/TkJt
I7cdC/36zA2OkzXK5UX5zy9/eT1jGMdHI0r2qRQArX5ZVRqJJ9uUoJE4xv5AlaNg9l24yMUW
7ZxmaRRGEErKcCpv0VDgJhrTUrRHcotF0r0DuaXc2QjzkKt0cKvKoE7wwE7k4L5PkBFgJwwr
HN/nbMp1tCXnkUiqkrRRdV8pm0cXHL3J6s91rXUjz/LF31qrt2vKu7heq9WjcNRo8xd6mwug
FDz76IFWaOjPXXxzuC68MYIEYjCCBF4CAQEwgbswgaYxCzAJBgNVBAYTAlVTMR0wGwYDVQQK
ExRTeW1hbnRlYyBDb3Jwb3JhdGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0d29y
azEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBD
bGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEc1AhAxSdDYnMC0s7K+ddH7ywAN
MA0GCWCGSAFlAwQCAQUAoIICdzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3
DQEJBTEPFw0xNjA3MDQxNDI3NDlaMC8GCSqGSIb3DQEJBDEiBCCSNVZ0C1uALfjHvwsNncO6
Sh1KjAYfJyXMU9nrS32X6jBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjALBglghkgB
ZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsO
AwIHMA0GCCqGSIb3DQMCAgEoMIHMBgkrBgEEAYI3EAQxgb4wgbswgaYxCzAJBgNVBAYTAlVT
MR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3JhdGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1
c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5T
eW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEc1AhAxSdDYnMC0
s7K+ddH7ywANMIHOBgsqhkiG9w0BCRACCzGBvqCBuzCBpjELMAkGA1UEBhMCVVMxHTAbBgNV
BAoTFFN5bWFudGVjIENvcnBvcmF0aW9uMR8wHQYDVQQLExZTeW1hbnRlYyBUcnVzdCBOZXR3
b3JrMR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlN5bWFudGVj
IENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzUCEDFJ0NicwLSzsr510fvL
AA0wDQYJKoZIhvcNAQEBBQAEggEAIIVK/UBgpDyh+BFzIUG/910diYU/pIu7J60vAym7hng4
PK4KBe5wV3HJ97NCSiu3OrE8hLihrtA7ejoOoT9iZru57YgIMx4jltVK9X3f/jVUhy5QXAPw
Tjs5ZeSBQWmTf2+IsHO+BsA/LmPHHyflvKOcbCcs6b0DcQzW6QBGQXJI8MTSgfdv3UICvjLu
/knO57Z0MrdJ01bORVv7O9kwTifRrTjbRyLj+A3bxMlib2ydOyKLq0D9Zh02hWdrWVKu7kOY
3+2TOgbv5Y0N2TpEJgCtFhqrmc0AYbQYOLEY8J9NiAREOb5ERrQPpF1Dj537vApilOwYIf7F
8QTjGn7QAAAAAAAAAA==
--------------ms080204020803010706030502--