[OpenAFS-devel] Cache inconsistency in client 1.4.8 and above

[Felix Frank]

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--579669762-867124517-1239884731=:7689
Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
X-MIME-Autoconverted: from 8bit to quoted-printable by znsun1.ifh.de id n3GCPWqa009573

On Wed, 15 Apr 2009, Derrick Brashear wrote:

> On Wed, Apr 15, 2009 at 5:44 AM, Felix Frank <Felix.Frank@desy.de> wrot=
e:
>> On a hunch, I applied this to 1.4.8:
>>
>> --- src/afs/LINUX/osi_vm.c.orig 2009-04-15 11:37:49.000000000 +0200
>> +++ src/afs/LINUX/osi_vm.c =A0 =A0 =A02009-04-15 11:38:56.000000000 +0=
200
>> @@ -102,11 +102,6 @@ osi_VM_StoreAllSegments(struct vcache *a
>> =A0{
>> =A0 =A0 struct inode *ip =3D AFSTOV(avc);
>>
>> - =A0 =A0if (!avc->states & CPageWrite)
>> - =A0 =A0 =A0 avc->states |=3D CPageWrite;
>> - =A0 =A0else - =A0 =A0 =A0 return; /* someone already writing */
>> -
>> =A0#if LINUX_VERSION_CODE >=3D KERNEL_VERSION(2,4,5)
>> =A0 =A0 /* filemap_fdatasync() only exported in 2.4.5 and above */
>> =A0 =A0 ReleaseWriteLock(&avc->lock);
>> @@ -120,7 +115,6 @@ osi_VM_StoreAllSegments(struct vcache *a
>> =A0 =A0 AFS_GLOCK();
>> =A0 =A0 ObtainWriteLock(&avc->lock, 121);
>> =A0#endif
>> - =A0 =A0avc->states &=3D ~CPageWrite;
>> =A0}
>>
>> =A0/* Purge VM for a file when its callback is revoked.
>>
>>
>> This apparently solved the problem for 1.4.8 w/ disk cache. Will try 1=
.4.10
>> as well. BCC'ing openafs-bugs now.
>
> The problem without that is a deadlock as described in RT 120491,
> which means either this or that needs to be solved in another way.
>
> Looking through my local pile of things to deal with, I see Chaskiel
> commented thus:
> "What's there seems like it will prevent recursion, but in a silly
> way. The whole point of calling filemap_fdatawrite
> is for the kernel to call writepage() on all the dirty pages. But
> since osi_VM_StoreAllSegments always sets CPageWrite and CPageWrite
> means writepage always returns WRITEPAGE_ACTIVATE, there's no point.
> Wouldn't it be better
> for a DoPartialWrite-driven StoreAllSegments to not call
> osi_VM_StoreAllSegments (and restore the latter to usefulness)?"
>
> If you wish to/can look, please do, otherwise I will as soon as I can.

I'm stuck. More tests showed that linux-mmap-antirecursion-20081020 will=20
lead to faulty mmap behaviour *in any case* on Linux (disk cache, don't=20
even get me started on memcache), not only when making=20
changes after the file close. As to why, this has me puzzled:

I fstraced the run of the attached program with and without antirecursion=
,=20
and the traces differ like this

...
Open
Iwrite
Iupdatepage ... code 99999
Write
Iupdatepage ... code 2
Iwrite
Gn_map
+ Iupdatepage ... code 99999
+ Write
+ Iupdatepage ... code 2
StoreAll
...

Where the lines marked "+" show up only in the run where antirecursion is=
=20
short-circuited. They are the only lines associated to a pid that shows u=
p=20
as [pdflush] in ps.

What I don't get is why setting CPageWrite prevents
afs_linux_writepage_sync from being called (?), as CPageWrite is checked=20
inside it, and only after the afs_Trace4(). Iupdatepage with code 99999
should therefore even show up with working antirecursion, as far as I can=
=20
understand it.

Sorry if I'm being gratuitous in a request for offline advice. Any=20
pointers, though?

Sincerely
Felix
--579669762-867124517-1239884731=:7689
Content-Type: TEXT/PLAIN; charset=US-ASCII; name=behave.c
Content-Description: 
Content-Disposition: attachment; filename=behave.c
Content-Transfer-Encoding: BASE64

I2luY2x1ZGUgPHN0ZGlvLmg+DQojaW5jbHVkZSA8c3lzL3R5cGVzLmg+DQoj
aW5jbHVkZSA8c3lzL3N0YXQuaD4NCiNpbmNsdWRlIDxzeXMvbW1hbi5oPg0K
I2luY2x1ZGUgPHVuaXN0ZC5oPg0KI2luY2x1ZGUgPGZjbnRsLmg+DQoNCmlu
dCBtYWluKGludCBhcmdjLCBjaGFyICoqYXJndikNCnsNCiAgICBjaGFyICpm
aWxlID0gIm1hcHBlZC1maWxlLmJpbiI7DQogICAgY2hhciAqbWFwID0gTlVM
TDsNCiAgICBpbnQgZmQ7DQoNCiAgICBpZiAoIGFyZ2MgPiAxICkNCglmaWxl
ID0gYXJndlsxXTsNCg0KICAgIHByaW50ZigiVXNpbmcgZmlsZSAlcy4uLlxu
IiwgZmlsZSk7DQoNCiAgICBmZCA9IG9wZW4oZmlsZSwgT19SRFdSIHwgT19D
UkVBVCk7DQogICAgaWYgKCBmZCA9PSAtMSApIHsNCglwZXJyb3IoZmlsZSk7
DQoJcmV0dXJuIDE7DQogICAgfQ0KDQogICAgd3JpdGUoZmQsICIxXG4iLCAy
KTsNCg0KICAgIGlmICggKG1hcCA9IChjaGFyKiltbWFwKE5VTEwsIDEsIFBS
T1RfUkVBRHxQUk9UX1dSSVRFLCBNQVBfU0hBUkVELCBmZCwgMCkpIA0KCQkJ
PT0gKGNoYXIqKSAtMSApIHsNCglwZXJyb3IoIm1tYXAiKTsNCglyZXR1cm4g
MTsNCiAgICB9DQoNCiAgICBtYXBbMF0rKzsNCg0KICAgIHByaW50ZigiQ2hh
bmdlZCBmaXJzdCBieXRlIHRvICV1LCB1bm1hcHBpbmcgYW5kIGNsb3Npbmcu
Li5cbiIsIG1hcFswXSk7DQoNCiAgICBtdW5tYXAoKHZvaWQqKW1hcCwgMSk7
DQoNCiAgICBjbG9zZShmZCk7DQoNCiAgICByZXR1cm4gMDsNCn0NCg==

--579669762-867124517-1239884731=:7689--