[OpenAFS] [ OpenAFS - Cache FileSystem ]
Shawn McKee
smckee@umich.edu
Thu, 3 Oct 2013 09:10:26 -0400
--047d7b6da1cad0341804e7d5e817
Content-Type: text/plain; charset=ISO-8859-1
Hi Arne,
Yes, sorry for the confusion. I missed the main word in the thread of
"cache"!
Our clients are primarily using ext4 so my comments don't really apply.
Shawn
On Thu, Oct 3, 2013 at 9:07 AM, Arne Wiebalck <Arne.Wiebalck@cern.ch> wrote:
> Hi Shawn,
>
> From what I understood, you use ZFS on the servers, not the clients, right?
>
> Cheers,
> Arne
>
>
> On Oct 3, 2013, at 3:00 PM, Shawn McKee <smckee@umich.edu>
> wrote:
>
> Hi Nicolas,
>
> Just as an FYI we are running our AFS cell (atlas.umich.edu) over ZFS
> v0.6.2 on Scientific Linux 6.4 64-bit without a problem.
>
> Shawn
>
>
> On Wed, Oct 2, 2013 at 3:48 PM, nicolas prochazka <
> prochazka.nicolas@gmail.com> wrote:
>
>> Hello,
>> For your intention,
>> In Faq, we can read :
>>
>> The OpenAFS cache manager will detect an unsupported filesystem and
>> refuse to start.
>>
>> The following file systems have been reported to work for the AFS client
>> cache:
>>
>> ext2
>> ext3
>> hfs (HP-UX)
>> xfs (at least on IRIX 6.5)
>> ufs (Solaris, ?Tru64Unix)
>>
>>
>> But if I configure cache on zfs on linux ( zfsonlinux.org) ,
>> i got kernel panic :
>>
>> [114328.841466] Starting AFS cache scan...
>> [114349.618208] openafs: Inconsistent file handles within cache
>> [114349.618238] ------------[ cut here ]------------
>> [114349.618926] kernel BUG at
>> /tmp/openafs/src/libafs/MODLOAD-3.11.2-MP/osi_file.c:129!
>> [114349.620004] invalid opcode: 0000 [#1] SMP
>> [114349.620004] Modules linked in: libafs(PO) zfs(PO) zunicode(PO)
>> zavl(PO) zcommon(PO) znvpair(PO) spl(O) kvm_intel kvm
>> [114349.620004] CPU: 0 PID: 32590 Comm: afsd Tainted: P O
>> 3.11.2 #1
>> [114349.620004] Hardware name: NEC Express5800/120Rj-2
>> [N8100-1407E]/MS-9192-01S, BIOS 1.0.4S31 03/05/2010
>> [114349.620004] task: ffff88076e5f2000 ti: ffff880559c6c000 task.ti:
>> ffff880559c6c000
>> [114349.620004] RIP: 0010:[<ffffffffa02971e8>] [<ffffffffa02971e8>]
>> osi_get_fh+0xc8/0xe0 [libafs]
>> [114349.620004] RSP: 0018:ffff880559c6dc38 EFLAGS: 00010296
>> [114349.620004] RAX: 000000000000002f RBX: 0000000000000007 RCX:
>> 0000000000000000
>> [114349.620004] RDX: ffff88082fc0fa88 RSI: ffff88082fc0e178 RDI:
>> 0000000000000246
>> [114349.620004] RBP: ffff880559c6dc48 R08: 0000000000000400 R09:
>> ffffffff82716488
>> [114349.635745] R10: 000000000000041c R11: 000000000000041b R12:
>> 0000000000000000
>> [114349.635745] R13: 000000000062dd80 R14: ffff88077ad03800 R15:
>> 0000000000000001
>> [114349.635745] FS: 00007f2afb49c700(0000) GS:ffff88082fc00000(0000)
>> knlGS:0000000000000000
>> [114349.635745] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [114349.635745] CR2: 000000007398600c CR3: 00000005583ef000 CR4:
>> 00000000000027e0
>> [114349.635745] Stack:
>> [114349.635745] ffff88077ad03800 0000000300000001 ffff880559c6dc68
>> ffffffffa02975c0
>> [114349.635745] ffff880559c6dc88 ffff88051fdb6900 ffff880559c6dca8
>> ffffffffa025ddfa
>> [114349.635745] ffff880559c6dc98 ffffffff81064565 ffff880559c6dca8
>> ffffffff81f182cd
>> [114349.635745] Call Trace:
>> [114349.635745] [<ffffffffa02975c0>] osi_InitCacheInfo+0x40/0x80 [libafs]
>> [114349.635745] [<ffffffffa025ddfa>] afs_InitCacheInfo+0x2a/0x130
>> [libafs]
>> [114349.635745] [<ffffffff81064565>] ? ns_capable+0x35/0x60
>> [114349.635745] [<ffffffff81f182cd>] ? mutex_lock+0x1d/0x50
>> [114349.635745] [<ffffffffa02a28de>] afs_syscall_call+0x105e/0x1bf0
>> [libafs]
>> [114349.635745] [<ffffffff8117a2ab>] ? do_last.isra.57+0x11b/0xc60
>> [114349.635745] [<ffffffff811775a8>] ? inode_permission+0x18/0x50
>> [114349.635745] [<ffffffff81177a8d>] ? link_path_walk+0x23d/0x8e0
>> [114349.635745] [<ffffffff8108538a>] ? lg_local_unlock+0x1a/0x20
>> [114349.635745] [<ffffffff81189e79>] ? mntput_no_expire+0x49/0x160
>> [114349.635745] [<ffffffffa022a9ec>] afs_syscall+0x17c/0x650 [libafs]
>> [114349.635745] [<ffffffffa02421f9>] afs_unlocked_ioctl+0x69/0xd0
>> [libafs]
>> [114349.635745] [<ffffffff811cf86f>] proc_reg_unlocked_ioctl+0x3f/0x70
>> [114349.635745] [<ffffffff8117d5fb>] do_vfs_ioctl+0x8b/0x4f0
>> [114349.635745] [<ffffffff81177376>] ? final_putname+0x26/0x50
>> [114349.635745] [<ffffffff8117dab0>] SyS_ioctl+0x50/0x90
>> [114349.635745] [<ffffffff81f1bb99>] system_call_fastpath+0x16/0x1b
>> [114349.635745] Code: 8b 0d 75 2c 02 00 89 05 73 2c 02 00 89 c2 85 c9
>> 79 ac 8b 4d fc 89 0d 60 2c 02 00 eb a1 48 c7 c7 08 42 2b a0 31 c0 e8
>> 3a 40 c7 e1 <0f> 0b 48 c7 c7 e0 41 2b a0 31 c0 e8 2a 40 c7 e1 0f 0b 66
>> 0f 1f
>> [114349.635745] RIP [<ffffffffa02971e8>] osi_get_fh+0xc8/0xe0 [libafs]
>> [114349.635745] RSP <ffff880559c6dc38>
>> [114351.110672] ---[ end trace b1f29ac8defa1347 ]---
>> _______________________________________________
>> OpenAFS-info mailing list
>> OpenAFS-info@openafs.org
>> https://lists.openafs.org/mailman/listinfo/openafs-info
>>
>
>
>
--047d7b6da1cad0341804e7d5e817
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Hi Arne,<div><br></div><div>Yes, sorry for the confusion. =
=A0 I missed the main word in the thread of "cache"!</div><div><b=
r></div><div>Our clients are primarily using ext4 so my comments don't =
really apply.</div>
<div><br></div><div>Shawn</div></div><div class=3D"gmail_extra"><br><br><di=
v class=3D"gmail_quote">On Thu, Oct 3, 2013 at 9:07 AM, Arne Wiebalck <span=
dir=3D"ltr"><<a href=3D"mailto:Arne.Wiebalck@cern.ch" target=3D"_blank"=
>Arne.Wiebalck@cern.ch</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div style=3D"word-wrap:break-word">Hi Shawn=
,<div><br></div><div>From what I understood, you use ZFS on the servers, no=
t the clients, right?</div>
<div><br></div><div>Cheers,</div><div>=A0Arne</div><div><br></div><div><br>=
<div><div>On Oct 3, 2013, at 3:00 PM, Shawn McKee <<a href=3D"mailto:smc=
kee@umich.edu" target=3D"_blank">smckee@umich.edu</a>></div><div>=A0wrot=
e:</div>
<div><div class=3D"h5"><br><blockquote type=3D"cite"><div dir=3D"ltr">Hi Ni=
colas,<div><br></div><div>Just as an FYI we are running our AFS cell (<a hr=
ef=3D"http://atlas.umich.edu/" target=3D"_blank">atlas.umich.edu</a>) over =
ZFS =A0v0.6.2 on Scientific Linux 6.4 64-bit without a problem.</div>
<div><br>
</div><div>Shawn</div></div><div class=3D"gmail_extra"><br><br><div class=
=3D"gmail_quote">On Wed, Oct 2, 2013 at 3:48 PM, nicolas prochazka <span di=
r=3D"ltr"><<a href=3D"mailto:prochazka.nicolas@gmail.com" target=3D"_bla=
nk">prochazka.nicolas@gmail.com</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Hello,<br>
For your intention,<br>
In Faq, we can read :<br>
<br>
The OpenAFS cache manager will detect an unsupported filesystem and<br>
refuse to start.<br>
<br>
The following file systems have been reported to work for the AFS client ca=
che:<br>
<br>
ext2<br>
ext3<br>
hfs (HP-UX)<br>
xfs (at least on IRIX 6.5)<br>
ufs (Solaris, ?Tru64Unix)<br>
<br>
<br>
But if I configure cache on zfs on linux ( <a href=3D"http://zfsonlinux.org=
/" target=3D"_blank">zfsonlinux.org</a>) ,<br>
i got kernel panic :<br>
<br>
<a href=3D"tel:%5B114328.841466" value=3D"+14328841466" target=3D"_blank">[=
114328.841466</a>] Starting AFS cache scan...<br>
<a href=3D"tel:%5B114349.618208" value=3D"+14349618208" target=3D"_blank">[=
114349.618208</a>] openafs: Inconsistent file handles within cache<br>
<a href=3D"tel:%5B114349.618238" value=3D"+14349618238" target=3D"_blank">[=
114349.618238</a>] ------------[ cut here ]------------<br>
<a href=3D"tel:%5B114349.618926" value=3D"+14349618926" target=3D"_blank">[=
114349.618926</a>] kernel BUG at<br>
/tmp/openafs/src/libafs/MODLOAD-3.11.2-MP/osi_file.c:129!<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] invalid opcode: 0000 [#1] SMP<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] Modules linked in: libafs(PO) zfs(PO) zunicode(PO)<br>
zavl(PO) zcommon(PO) znvpair(PO) spl(O) kvm_intel kvm<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] CPU: 0 PID: 32590 Comm: afsd Tainted: P =A0 =A0 =A0 =A0 =
=A0 O 3.11.2 #1<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] Hardware name: NEC Express5800/120Rj-2<br>
[N8100-1407E]/MS-9192-01S, BIOS 1.0.4S31 03/05/2010<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] task: ffff88076e5f2000 ti: ffff880559c6c000 task.ti:<br>
ffff880559c6c000<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] RIP: 0010:[<ffffffffa02971e8>] =A0[<ffffffffa02=
971e8>]<br>
osi_get_fh+0xc8/0xe0 [libafs]<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] RSP: 0018:ffff880559c6dc38 =A0EFLAGS: 00010296<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] RAX: 000000000000002f RBX: 0000000000000007 RCX:<br>
0000000000000000<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] RDX: ffff88082fc0fa88 RSI: ffff88082fc0e178 RDI:<br>
0000000000000246<br>
<a href=3D"tel:%5B114349.620004" value=3D"+14349620004" target=3D"_blank">[=
114349.620004</a>] RBP: ffff880559c6dc48 R08: 0000000000000400 R09:<br>
ffffffff82716488<br>
<a href=3D"tel:%5B114349.635745" value=3D"+14349635745" target=3D"_blank">[=
114349.635745</a>] R10: 000000000000041c R11: 000000000000041b R12:<br>
0000000000000000<br>
<a href=3D"tel:%5B114349.635745" value=3D"+14349635745" target=3D"_blank">[=
114349.635745</a>] R13: 000000000062dd80 R14: ffff88077ad03800 R15:<br>
0000000000000001<br>
<a href=3D"tel:%5B114349.635745" value=3D"+14349635745" target=3D"_blank">[=
114349.635745</a>] FS: =A000007f2afb49c700(0000) GS:ffff88082fc00000(0000)<=
br>
knlGS:0000000000000000<br>
<a href=3D"tel:%5B114349.635745" value=3D"+14349635745" target=3D"_blank">[=
114349.635745</a>] CS: =A00010 DS: 0000 ES: 0000 CR0: 0000000080050033<br>
<a href=3D"tel:%5B114349.635745" value=3D"+14349635745" target=3D"_blank">[=
114349.635745</a>] CR2: 000000007398600c CR3: 00000005583ef000 CR4:<br>
00000000000027e0<br>
[114349.635745] Stack:<br>
[114349.635745] =A0ffff88077ad03800 0000000300000001 ffff880559c6dc68<br>
ffffffffa02975c0<br>
[114349.635745] =A0ffff880559c6dc88 ffff88051fdb6900 ffff880559c6dca8<br>
ffffffffa025ddfa<br>
[114349.635745] =A0ffff880559c6dc98 ffffffff81064565 ffff880559c6dca8<br>
ffffffff81f182cd<br>
[114349.635745] Call Trace:<br>
[114349.635745] =A0[<ffffffffa02975c0>] osi_InitCacheInfo+0x40/0x80 [=
libafs]<br>
[114349.635745] =A0[<ffffffffa025ddfa>] afs_InitCacheInfo+0x2a/0x130 =
[libafs]<br>
[114349.635745] =A0[<ffffffff81064565>] ? ns_capable+0x35/0x60<br>
[114349.635745] =A0[<ffffffff81f182cd>] ? mutex_lock+0x1d/0x50<br>
[114349.635745] =A0[<ffffffffa02a28de>] afs_syscall_call+0x105e/0x1bf=
0 [libafs]<br>
[114349.635745] =A0[<ffffffff8117a2ab>] ? do_last.isra.57+0x11b/0xc60=
<br>
[114349.635745] =A0[<ffffffff811775a8>] ? inode_permission+0x18/0x50<=
br>
[114349.635745] =A0[<ffffffff81177a8d>] ? link_path_walk+0x23d/0x8e0<=
br>
[114349.635745] =A0[<ffffffff8108538a>] ? lg_local_unlock+0x1a/0x20<b=
r>
[114349.635745] =A0[<ffffffff81189e79>] ? mntput_no_expire+0x49/0x160=
<br>
[114349.635745] =A0[<ffffffffa022a9ec>] afs_syscall+0x17c/0x650 [liba=
fs]<br>
[114349.635745] =A0[<ffffffffa02421f9>] afs_unlocked_ioctl+0x69/0xd0 =
[libafs]<br>
[114349.635745] =A0[<ffffffff811cf86f>] proc_reg_unlocked_ioctl+0x3f/=
0x70<br>
[114349.635745] =A0[<ffffffff8117d5fb>] do_vfs_ioctl+0x8b/0x4f0<br>
[114349.635745] =A0[<ffffffff81177376>] ? final_putname+0x26/0x50<br>
[114349.635745] =A0[<ffffffff8117dab0>] SyS_ioctl+0x50/0x90<br>
[114349.635745] =A0[<ffffffff81f1bb99>] system_call_fastpath+0x16/0x1=
b<br>
[114349.635745] Code: 8b 0d 75 2c 02 00 89 05 73 2c 02 00 89 c2 85 c9<br>
79 ac 8b 4d fc 89 0d 60 2c 02 00 eb a1 48 c7 c7 08 42 2b a0 31 c0 e8<br>
3a 40 c7 e1 <0f> 0b 48 c7 c7 e0 41 2b a0 31 c0 e8 2a 40 c7 e1 0f 0b 6=
6<br>
0f 1f<br>
[114349.635745] RIP =A0[<ffffffffa02971e8>] osi_get_fh+0xc8/0xe0 [lib=
afs]<br>
[114349.635745] =A0RSP <ffff880559c6dc38><br>
[114351.110672] ---[ end trace b1f29ac8defa1347 ]---<br>
_______________________________________________<br>
OpenAFS-info mailing list<br>
<a href=3D"mailto:OpenAFS-info@openafs.org" target=3D"_blank">OpenAFS-info@=
openafs.org</a><br>
<a href=3D"https://lists.openafs.org/mailman/listinfo/openafs-info" target=
=3D"_blank">https://lists.openafs.org/mailman/listinfo/openafs-info</a><br>
</blockquote></div><br></div>
</blockquote></div></div></div><br></div></div></blockquote></div><br></div=
>
--047d7b6da1cad0341804e7d5e817--