[OpenAFS] No space left on device error -- despite partition being only 33% full

[Ken Elkabany]

Hi all,

My afs clients have begun receiving a "No space left on device" error
when attempting to write to or delete from a specific afs volume. The
volume in question called "jobs" has both a RW and RO site on the same
partition, vicepa.

vos partinfo [server]
Free space on partition /vicepa: 3292236 K blocks out of total 4898432

fs listquota jobs
Volume Name                   Quota      Used %Used   Partition
jobs                        3000000    701444   23%         33%

I have also verified via "df" that on our afs fileserver, the mount
for /vicepa is only 33% full.

The one peculiarity to our system is that our jobs volume has many
folders. On the 1st/top level there are 50 folders, on the second
level there are 1000 folders for each folder on the top level. For
each of the 1000*50=50,000 second level directories, there is a single
third level directory which contains a small collection of files.
Could this directory structure be causing any problems? Any other
ideas?

Thanks in advance,

Ken