[OpenAFS] Re: stackable file system above OpenAFS

[Andrew Deason]

On Thu, 2 Oct 2014 15:41:32 +0200 (CEST)
Erik Braun <erik.braun@uni-jena.de> wrote:

> On Thu, 25 Sep 2014, Andrew Deason wrote:
> 
> > It's been a while since I looked at this, so I may have things wrong;
> > if things still hang, just give us a backtrace again.
> 
> Thank you for the patch – unfortunately it was not sufficient
> (OpenAFS 1.6.9, Linux 3.16.3 as shipped with Debian Testing).
> The corresponding backtrace is this:

Yes, my suggestion was perhaps a bit naive, and I hadn't really
remembered the issue fully.

> When changing osi_VM_FlushPages() as follows, Aufs and OpenAFS seem to
> run. I guess, that I don't need the locking, because the AFS volume is
> accessed readonly and will certainly not change in my case.

Yes, and that's probably fine for you, but of course we can't do that
for everyone. The patch you showed will avoid this, but it violates the
locking requirements for Linux's VFS structures, so it could cause any
kinds of problems if accessed outside of AUFS.

My understanding of what's going on:

AUFS tries to lock the inode so it can't change while it reads the
inode. The purpose of this is for normal filesystems is (I think) to
make anything like a write() from another process block, so AUFS can get
a consistent 'snapshot' of the file.

But with AFS, you cannot prevent someone else from changing the file
(even for RO volumes, the volume could be changed and released). So when
the AFS cache manager detects that the file has changed, it locks the
inode so we can tell Linux that the file contents have changed.

As you said, in your situation nothing is actually changing; the reason
this still happens is due to some details of how AFS tracks if a file
has "changed". That is probably possible to avoid, but I'd need to think
about it. But even solving that doesn't get rid of this deadlock, it's
just a "sorta works for you" workaround; we could still deadlock if the
files ever change after you've accessed them once.

Of course, if AUFS would just lock the inode after the open() call, that
would also avoid this in a similar "sorta works" way. I can talk to them
and ask if they'd do such a thing and submit a patch or something. But
keep in mind the deadlock can still occur; the only way I can think of
to avoid that is if AUFS has some kind of mode that says "the underlying
file can indeed change, and there's nothing I can do about it".

Anyway, you can run with that patch for now. It's not really "safe", but
it's how openafs ran for a long time, and for you specifically, I don't
expect you'll notice a problem.

-- 
Andrew Deason
adeason@sinenomine.net