[OpenAFS] About partition size limitations again

[Serge Torres]

Hi,

 From previous messages in the list I had the notion that partition size
was only limited by the operating system capabilities.

In fact this is not always a correct assumption. Playing around with our
brand new disk array, we have created a 4 Tb partition on a (Debian) 
Linux box with XFS, created volumes in it with openAFS 1.4.1 and even 
mounted them on client machines. So far, so good.

But when we try to write something to a volume we get a "no space left
on device" message.

Using the "fs diskfree" command returns:
Volume Name             kbytes      used     avail %used
test.volume2            97975296 731912516-633937220  747%<< <<WARNING



The "vos partinfo" command returns:

Free space on partition /vicepa: -633937220 K blocks out of total 97975296

We checked and rechecked, deleted, recreated, salvaged... no joy.

A 2 Tb partition worked without a glitch.

Eventually we went back to the sources in tar.gz form, run "configure"
and poke around the code. In src/vol/partition.h, we found that struct:

  struct DiskPartition {
      struct DiskPartition *next;
      char *name;                 /* Mounted partition name */
      char *devName;              /* Device mounted on */
      Device device;              /* device number */
      int lock_fd;                /* File descriptor of this partition
                                     if locked; otherwise -1;
                                   * Not used by the file server */
      int free;                   /* Total number of blocks (1K) presumed
                                   * available on this partition
                                     (accounting
                                   * for the minfree parameter for the
                                   * partition).  This is adjusted
                                   * approximately by the sizes of files
                                   * and directories read/written, and
                                   * periodically the superblock is read
                                     and
                                   * this is recomputed.  This number can
                                   * be negative, if the partition starts
                                   * out too full */
      int totalUsable;            /* Total number of blocks available on
                                     this
                                   * partition, taking into account the
                                     minfree
                                   * parameter for the partition (see the
                                   * 4.2bsd command tunefs, but note
                                     that the
                                   * bug mentioned there--that the
                                     superblock
                                   * is not reread--does not apply here.
                                     The
                                   * superblock is re-read periodically
                                     by
                                   * VSetPartitionDiskUsage().) */
      int minFree;                /* Number blocks to be kept free, as
                                     last read
                                   * from the superblock */
      int flags;
      int f_files;                /* total number of files in this
                                     partition */
  };

As you can notice various data (free, totalUsable...) are coded as ints
so we suspect that, on 32 bits platforms, the corresponding capacity is 
stuck to 2 Tb (2 Giga x 1 kb blocks).

We did not look any further into the code but the same kind of
definition is probably found in many different place. We did not have 
the time/capacity to fix throughout such a huge piece of code as the 60+
Mb of openAFS. After all, is it worth it since we are not advocating for
such big partitions as 4 Tb: waiting for a fsck of such a beast will
take whole coffee crop of Costa Rica (or maybe that of Brazil). But if 
you, as we are, are foolish enough to play this game, be warned: it does 
not work, at least on Linux x86.

Best regards,

Serge Torres

LIP
Ecole normale superieure de Lyon