[OpenAFS] kernel panic in 1.4.14.1 on rhel6
Aaron Knister
aaronk@umbc.edu
Wed, 27 Jul 2011 13:27:24 -0400
--90e6ba4fc3aebf255804a9105d5e
Content-Type: text/plain; charset=ISO-8859-1
I'm running openafs 1.4.14.1 on a RHEL6 system and under heavy load when
accessing files in AFS via apache I'm able to consistently trigger a kernel
panic. Below is the output from the "crash" command and includes a stack
back-trace.
Please let me know if/what I can do to help further troubleshoot this.
Thanks!
SYSTEM MAP: /boot/System.map-2.6.32-131.6.1.el6.x86_64
DEBUG KERNEL: /usr/lib/debug/lib/modules/2.6.32-131.6.1.el6.x86_64/vmlinux
(2.6.32-131.6.1.el6.x86_64)
DUMPFILE: vmcore [PARTIAL DUMP]
CPUS: 2
DATE: Wed Jul 27 13:17:04 2011
UPTIME: 00:04:12
LOAD AVERAGE: 22.70, 5.09, 1.67
TASKS: 369
NODENAME: jabba-www-test1
RELEASE: 2.6.32-131.6.1.el6.x86_64
VERSION: #1 SMP Mon Jun 20 14:15:38 EDT 2011
MACHINE: x86_64 (2593 Mhz)
MEMORY: 2 GB
PANIC: ""
PID: 3573
COMMAND: "httpd"
TASK: ffff88006fc174c0 [THREAD_INFO: ffff8800610bc000]
CPU: 0
STATE: TASK_RUNNING (PANIC)
crash> bt
PID: 3573 TASK: ffff88006fc174c0 CPU: 0 COMMAND: "httpd"
#0 [ffff8800610bd850] machine_kexec at ffffffff810310cb
#1 [ffff8800610bd8b0] crash_kexec at ffffffff810b6312
#2 [ffff8800610bd980] oops_end at ffffffff814de540
#3 [ffff8800610bd9b0] die at ffffffff8100f2eb
#4 [ffff8800610bd9e0] do_general_protection at ffffffff814de0d2
#5 [ffff8800610bda10] general_protection at ffffffff814dd8a5
[exception RIP: mutex_lock+33]
RIP: ffffffff814dbfd1 RSP: ffff8800610bdac8 RFLAGS: 00010246
RAX: 0000000000000000 RBX: 28ec2299000000e7 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 28ec2299000000e7
RBP: ffff8800610bdad8 R8: ffff88006f8b6148 R9: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 28ec2299000000df
R13: 0000000000000001 R14: 28ec2299000000e7 R15: ffff8800610bdc08
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#6 [ffff8800610bdae0] afs_mutex_enter at ffffffffa02b8ec6 [openafs]
#7 [ffff8800610bdb00] rxi_ResetCall at ffffffffa02af261 [openafs]
#8 [ffff8800610bdb30] rxi_NewCall at ffffffffa02b4804 [openafs]
#9 [ffff8800610bdb60] rx_NewCall at ffffffffa02b5290 [openafs]
#10 [ffff8800610bdbd0] RXAFS_FetchStatus at ffffffffa02a8ab7 [openafs]
#11 [ffff8800610bdca0] afs_FetchStatus at ffffffffa028b752 [openafs]
#12 [ffff8800610bdd70] afs_GetAccessBits at ffffffffa028ffa6 [openafs]
#13 [ffff8800610bde00] afs_AccessOK at ffffffffa02900d6 [openafs]
#14 [ffff8800610bde60] afs_access at ffffffffa02905a1 [openafs]
#15 [ffff8800610bdef0] afs_linux_permission at ffffffffa02cb41c [openafs]
#16 [ffff8800610bdf20] inode_permission at ffffffff8117ee1e
#17 [ffff8800610bdf40] sys_chdir at ffffffff8117061a
#18 [ffff8800610bdf80] system_call_fastpath at ffffffff8100b172
RIP: 00007f814bc78dd7 RSP: 00007fff529e5a38 RFLAGS: 00010202
RAX: 0000000000000050 RBX: ffffffff8100b172 RCX: 00007f814bc78eaa
RDX: 0000000000000000 RSI: 00007f814ebaacc2 RDI: 00007fff529e59a0
RBP: 00007fff529e5a30 R8: 75702f7564652e63 R9: 7777772f63696c62
R10: 666e6978696e752f R11: 0000000000000246 R12: 00007f814ebaaca0
R13: 0000000000000022 R14: ffff88007ab27b40 R15: ffff880037099c80
ORIG_RAX: 0000000000000050 CS: 0033 SS: 002b
--
Aaron Knister
Systems Administrator
Division of Information Technology
University of Maryland, Baltimore County
aaronk@umbc.edu
--90e6ba4fc3aebf255804a9105d5e
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<div>I'm running openafs 1.4.14.1 on a RHEL6 system and under heavy loa=
d when accessing files in AFS via apache I'm able to consistently trigg=
er a kernel panic. Below is the output from the "crash" command a=
nd includes a stack back-trace.</div>
<div><br></div><div>Please let me know if/what I can do to help further tro=
ubleshoot this.</div><div><br></div><div>Thanks!</div><div><br></div><div>=
=A0 SYSTEM MAP: /boot/System.map-2.6.32-131.6.1.el6.x86_64 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0=A0</div>
<div>DEBUG KERNEL: /usr/lib/debug/lib/modules/2.6.32-131.6.1.el6.x86_64/vml=
inux (2.6.32-131.6.1.el6.x86_64)</div><div>=A0 =A0 DUMPFILE: vmcore =A0[PAR=
TIAL DUMP]</div><div>=A0 =A0 =A0 =A0 CPUS: 2</div><div>=A0 =A0 =A0 =A0 DATE=
: Wed Jul 27 13:17:04 2011</div>
<div>=A0 =A0 =A0 UPTIME: 00:04:12</div><div>LOAD AVERAGE: 22.70, 5.09, 1.67=
</div><div>=A0 =A0 =A0 =A0TASKS: 369</div><div>=A0 =A0 NODENAME: jabba-www-=
test1</div><div>=A0 =A0 =A0RELEASE: 2.6.32-131.6.1.el6.x86_64</div><div>=A0=
=A0 =A0VERSION: #1 SMP Mon Jun 20 14:15:38 EDT 2011</div>
<div>=A0 =A0 =A0MACHINE: x86_64 =A0(2593 Mhz)</div><div>=A0 =A0 =A0 MEMORY:=
2 GB</div><div>=A0 =A0 =A0 =A0PANIC: ""</div><div>=A0 =A0 =A0 =
=A0 =A0PID: 3573</div><div>=A0 =A0 =A0COMMAND: "httpd"</div><div>=
=A0 =A0 =A0 =A0 TASK: ffff88006fc174c0 =A0[THREAD_INFO: ffff8800610bc000]</=
div>
<div>=A0 =A0 =A0 =A0 =A0CPU: 0</div><div>=A0 =A0 =A0 =A0STATE: TASK_RUNNING=
(PANIC)</div><div><br></div><div>crash> bt</div><div>PID: 3573 =A0 TASK=
: ffff88006fc174c0 =A0CPU: 0 =A0 COMMAND: "httpd"</div><div>=A0#0=
[ffff8800610bd850] machine_kexec at ffffffff810310cb</div>
<div>=A0#1 [ffff8800610bd8b0] crash_kexec at ffffffff810b6312</div><div>=A0=
#2 [ffff8800610bd980] oops_end at ffffffff814de540</div><div>=A0#3 [ffff880=
0610bd9b0] die at ffffffff8100f2eb</div><div>=A0#4 [ffff8800610bd9e0] do_ge=
neral_protection at ffffffff814de0d2</div>
<div>=A0#5 [ffff8800610bda10] general_protection at ffffffff814dd8a5</div><=
div>=A0 =A0 [exception RIP: mutex_lock+33]</div><div>=A0 =A0 RIP: ffffffff8=
14dbfd1 =A0RSP: ffff8800610bdac8 =A0RFLAGS: 00010246</div><div>=A0 =A0 RAX:=
0000000000000000 =A0RBX: 28ec2299000000e7 =A0RCX: 0000000000000001</div>
<div>=A0 =A0 RDX: 0000000000000000 =A0RSI: 0000000000000001 =A0RDI: 28ec229=
9000000e7</div><div>=A0 =A0 RBP: ffff8800610bdad8 =A0 R8: ffff88006f8b6148 =
=A0 R9: 0000000000000000</div><div>=A0 =A0 R10: 0000000000000000 =A0R11: 00=
00000000000000 =A0R12: 28ec2299000000df</div>
<div>=A0 =A0 R13: 0000000000000001 =A0R14: 28ec2299000000e7 =A0R15: ffff880=
0610bdc08</div><div>=A0 =A0 ORIG_RAX: ffffffffffffffff =A0CS: 0010 =A0SS: 0=
018</div><div>=A0#6 [ffff8800610bdae0] afs_mutex_enter at ffffffffa02b8ec6 =
[openafs]</div>
<div>=A0#7 [ffff8800610bdb00] rxi_ResetCall at ffffffffa02af261 [openafs]</=
div><div>=A0#8 [ffff8800610bdb30] rxi_NewCall at ffffffffa02b4804 [openafs]=
</div><div>=A0#9 [ffff8800610bdb60] rx_NewCall at ffffffffa02b5290 [openafs=
]</div>
<div>#10 [ffff8800610bdbd0] RXAFS_FetchStatus at ffffffffa02a8ab7 [openafs]=
</div><div>#11 [ffff8800610bdca0] afs_FetchStatus at ffffffffa028b752 [open=
afs]</div><div>#12 [ffff8800610bdd70] afs_GetAccessBits at ffffffffa028ffa6=
[openafs]</div>
<div>#13 [ffff8800610bde00] afs_AccessOK at ffffffffa02900d6 [openafs]</div=
><div>#14 [ffff8800610bde60] afs_access at ffffffffa02905a1 [openafs]</div>=
<div>#15 [ffff8800610bdef0] afs_linux_permission at ffffffffa02cb41c [opena=
fs]</div>
<div>#16 [ffff8800610bdf20] inode_permission at ffffffff8117ee1e</div><div>=
#17 [ffff8800610bdf40] sys_chdir at ffffffff8117061a</div><div>#18 [ffff880=
0610bdf80] system_call_fastpath at ffffffff8100b172</div><div>=A0 =A0 RIP: =
00007f814bc78dd7 =A0RSP: 00007fff529e5a38 =A0RFLAGS: 00010202</div>
<div>=A0 =A0 RAX: 0000000000000050 =A0RBX: ffffffff8100b172 =A0RCX: 00007f8=
14bc78eaa</div><div>=A0 =A0 RDX: 0000000000000000 =A0RSI: 00007f814ebaacc2 =
=A0RDI: 00007fff529e59a0</div><div>=A0 =A0 RBP: 00007fff529e5a30 =A0 R8: 75=
702f7564652e63 =A0 R9: 7777772f63696c62</div>
<div>=A0 =A0 R10: 666e6978696e752f =A0R11: 0000000000000246 =A0R12: 00007f8=
14ebaaca0</div><div>=A0 =A0 R13: 0000000000000022 =A0R14: ffff88007ab27b40 =
=A0R15: ffff880037099c80</div><div>=A0 =A0 ORIG_RAX: 0000000000000050 =A0CS=
: 0033 =A0SS: 002b</div>
<div><br></div><br>-- <br>Aaron Knister<br>Systems Administrator<br>Divisio=
n of Information Technology<br>University of Maryland, Baltimore County<br>=
<a href=3D"mailto:aaronk@umbc.edu" target=3D"_blank">aaronk@umbc.edu</a><br=
>
--90e6ba4fc3aebf255804a9105d5e--