[OpenAFS-devel] Re: Java VM native libs call flock() with l_len=0x7ffffffffffffffe

[Adam Megacz]

[ correction, the value is actually 0x7ffffffffffffffe, which is ]
[ OFFSET_MAX+(-1)                                                ]

Derrick J Brashear <shadow@dementia.org> writes:
> ideally, submit it with a simple test program that demonstrates the
> problem. hopefully we will have some better testing in place in the
> future to try things like this.

(example below, patch updated and tested)

It appears that Sun's JVM (and perhaps others) have only one low-level
C routine for requesting locks from the OS.  That routine is a
byte-range locking routine.  To implement the higher-level "lock this
entire file" API (FileChannel.tryLock()), the libraries (written in
Java) invoke the low-level C routine with the file length set to -1.
This constant is provided as a *Java long*, which is always 64 signed
bits regardless of OS or CPU.

Anyways, the ultimate result is that AFS winds up getting a lock
request like this:

   l_start  = 0
   l_whence = 0
   l_len    = 0x7ffffffffffffffe

Clearly this is a Java bug, but Sun is unlikely to ever do anything
about it.  Since it's also unlikely that any file is ever going to be
anywhere close to being (unsigned)0x7ffffffffffffffe bytes long (more
than 2^63), the patch below has OpenAFS interpret such requests as
whole-file locks.

Also available from:

  /afs/megacz.com/debian/patches/make-openafs-and-java-cooperate2.patch

Test case (cd /afs/foo; javac FlockDemo.java; java -cp . FlockDemo)

  import java.io.*;
  import java.nio.*;
  
  public class FlockDemo {
      public static void main(String[] s) throws Exception {
          System.err.println("flocking foo.lock");
          File file = new File("foo.lock");
          new RandomAccessFile(file, "rw")
              .getChannel()
              .tryLock();
      }
  }
  
Patch:

--- openafs-1.4.2/src/afs/VNOPS/afs_vnop_flock.c	2007-02-13 20:25:18.000000000 -0800
+++ openafs-1.4.2/src/afs/VNOPS/afs_vnop_flock.c	2007-02-13 20:18:44.000000000 -0800
@@ -554,6 +554,10 @@
 	af->l_len = 0;		/* since some systems indicate it as EOF */
 #endif
 #endif
+    /* Java VMs ask for l_len=(long)-1 regardless of OS/CPU; bottom 32 bits
+     * sometimes get masked off by OS */
+    if (af->l_len == OFFSET_MAX-1)
+	af->l_len = 0;
     /* next line makes byte range locks always succeed,
      * even when they should block */
     if (af->l_whence != 0 || af->l_start != 0 || af->l_len != 0) {