[OpenAFS-devel] 1.2.9a kernel crash on Irix 6.5.20
Martin MOKREJŠ
mmokrejs@natur.cuni.cz
Wed, 9 Jul 2003 12:50:42 +0200 (CEST)
On Wed, 9 Jul 2003, Martin MOKREJŠ wrote:
> Hi,
> I use the IP22 no nfs module on my Indy machine. When linking the kernel
> I got:
I should say I can boot with IP22 no-nfs module from IBM 3.6_2.38 the client and
server enabled.
savecore: Created log Tue Jul 8 16:21:34 2003
Dump Header Information
-------------------------------------------------------
uname: IRIX nmrindy 6.5 04100802 IP22
physical mem: 160 megabytes
phys start: 0x8000000
page size: 4096 bytes
dir page size: 0 bytes
dump version: 7
dump size: 77204 k
crash time: Tue Jul 8 16:21:34 2003
panic string: PANIC: stack underflow/overflow
kernel putbuf:
pb 0: <6>IRIX Release 6.5 IP22 Version 04100802 System V
pb 1: Copyright 1987-2003 Silicon Graphics, Inc.
pb 2: All Rights Reserved.
pb 3:
pb 4: <5>NOTICE: Start mounting filesystem: /
pb 5: <5>NOTICE: Starting XFS recovery on filesystem: / (dev: 0/74)
pb 6: <5>NOTICE: Ending XFS recovery for filesystem: / (/hw/node/io/gio/hpc/scsi_ctlr/0/target/5/lun/0/disk/partition/0/block)
pb 7: <5>NOTICE: Start mounting filesystem: /vicepc
pb 8: <5>NOTICE: Ending clean XFS mount for filesystem: /vicepc
pb 9: <5>NOTICE: Start mounting filesystem: /vicepb
pb 10: <5>NOTICE: Ending clean XFS mount for filesystem: /vicepb
pb 11: <5>NOTICE: Start mounting filesystem: /vicepa
pb 12: <5>NOTICE: Ending clean XFS mount for filesystem: /vicepa
pb 13: <6>Starting AFS cache scan...<6>found 0 non-empty cache files (0%).
pb 14: <6>Kernel/Interrupt Stack Overflow @0x88118dd8 sp:0xffffae08 k1:0xffffaf48
pb 15: ^M<6>ra:0x88105620 stkflag:1
pb 16:
pb 17: <0>PANIC: stack underflow/overflow
pb 18: <6>
pb 19: Dumping to /hw/node/io/gio/hpc/scsi_ctlr/0/target/5/lun/0/disk/partition/1/block at block 0, space: 0x8000 pages
/var/adm/crash# icrash unix.0 vmcore.0.comp
corefile = vmcore.0.comp, namelist = unix.0, outfile = stdout
Please wait.....................
Dumpheader version 7, processor type IP22, running in M-mode
>> trace
===============================================================================
STACK TRACE FOR UTHREAD 0x8c772000 (afsd, PID=692):
1 dumpsys[../os/vmdump.c: 528, 0x881757e0]
2 syncreboot[../os/printf.c: 1677, 0x8814cfb4]
3 icmn_err_tag[../os/printf.c: 593, 0x8814bac4]
4 panic[../os/printf.c: 795, 0x8814bf3c]
===============================================================================
>> trace -a
UTHREAD STACK TRACES:
[cut]
===============================================================================
STACK TRACE FOR UTHREAD 0x8cde6000 (bosserver, PID=670):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 sv_queue[../os/ksync/mutex.c: 1595, 0x880dcdf8]
4 sv_timedwait_sig[../os/ksync/mutex.c: 2286, 0x880ddc14]
5 sv_wait_sig[../os/ksync/mutex.c: 1406, 0x880dcb64]
6 dopoll[../sgi/select.c: 526, 0x880a52a0]
7 k_select[../sgi/select.c: 698, 0x880a5964]
8 select[../sgi/select.c: 766, 0x880a5b94]
9 syscall[../os/trap.c: 2832, 0x880f5294]
10 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:00000000100abecc r2/v0:000000000000044c
r3/v1:000000001006e860 r4/a0:0000000000000004 r5/a1:000000001006e760
r6/a2:0000000000000000 r7/a3:0000000000000000 r8/a4:000000001006c118
r9/a5:0000000000000000 r10/a6:ffffffffffffffff r11/a7:000000000fb353e8
r12/t0:0000000000000000 r13/t1:000000003f0ad36e r14/t2:000000003f0ad36e
r15/t3:0000000000000000 r16/s0:0000000000000004 r17/s1:000000001006c118
r18/s2:0000000000000000 r19/s3:000000007fff2fc0 r20/s4:0000000000000000
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:00000000100390c0 r26/k0:0000000000000000
r27/k1:000000000037619a r28/gp:000000000fb3fdfc r29/sp:000000001007f430
r30/s8:0000000000000000 r31/ra:000000000fa2e1e8 EPC:000000000fa315ac
CAUSE=4, SR=2400ff33, BADVADDR=100abed0
===============================================================================
STACK TRACE FOR UTHREAD 0x88b3d000 (afsd, PID=677):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 semawait[../os/ksync/sema.c: 633, 0x880d8be4]
4 psema[../os/ksync/sema.c: 900, 0x880d8fa0]
5 biowait[../sgi/fs_bio.c: 2458, 0x880a8a08]
6 read_buf_targ[../sgi/fs_bio.c: 635, 0x880a6528]
7 xfs_trans_read_buf[../fs/xfs/xfs_trans_buf.c: 346, 0x881eb6e4]
8 xfs_itobp[../fs/xfs/xfs_inode.c: 422, 0x881e45a4]
9 xfs_iread[../fs/xfs/xfs_inode.c: 836, 0x881e4fc0]
10 xfs_iget[../fs/xfs/xfs_iget.c: 280, 0x881e333c]
11 xfs_dir_lookup_int[../fs/xfs/xfs_utils.c: 219, 0x881f85c8]
12 xfs_lookup[../fs/xfs/xfs_vnodeops.c: 2353, 0x881f1ab8]
13 lookuppn[../os/lookup.c: 223, 0x88155404]
14 lookupname[../os/lookup.c: 70, 0x88155130]
15 cmount[../os/vfs.c: 133, 0x8812d1f8]
16 mount[../os/vfs.c: 113, 0x8812d1ac]
17 syscall[../os/trap.c: 2832, 0x880f5294]
18 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:0000000000000000 r2/v0:00000000000003fd
r3/v1:0000000000000088 r4/a0:0000000010059358 r5/a1:000000001005b310
r6/a2:0000000000000002 r7/a3:0000000010059358 r8/a4:0000000000000000
r9/a5:0000000000000000 r10/a6:000000000fb377c8 r11/a7:0000000000000001
r12/t0:0000000000000074 r13/t1:0000000000000000 r14/t2:0000000000000000
r15/t3:000000001000c08c r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:00000000000008ba r25/t9:ffffffffffffffff r26/k0:0000000000000000
r27/k1:000000000022079a r28/gp:00000000100608c8 r29/sp:000000007ffefa00
r30/s8:0000000000000000 r31/ra:000000001000c08c EPC:000000000fa31118
CAUSE=ffffffff80000004, SR=2400ff33, BADVADDR=100589b0
===============================================================================
STACK TRACE FOR UTHREAD 0x8c78a600 (vlserver, PID=688):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 qswtch[../os/swtch.c: 174, 0x88169fb8]
3 kpswtch[../os/swtch.c: 238, 0x8816a110]
4 mutex_wake[../os/ksync/mutex.c: 1148, 0x880dc750]
5 mutex_unlock[../os/ksync/mutex.c: 408, 0x880dbccc]
6 page_zero[../os/page.c: 5576, 0x880f1c34]
7 pas_vfault[../os/as/fault.c: 3215, 0x881089ec]
8 pas_fault[../os/as/pas.c: 119, 0x8818b49c]
9 vfault[../ksys/as.h: 1068, 0x88135b14]
10 tlbmiss[../os/trap.c: 3308, 0x880f5964]
11 VEC_tlbmiss[../ml/LOCORE/vec_tlbmiss.s: 41, 0x8801b740]
r0/zero:0000000000000000 r1/at:0000000000000001 r2/v0:0000000000000001
r3/v1:000000000000c000 r4/a0:0000000000000001 r5/a1:000000000000c000
r6/a2:0000000000003628 r7/a3:0000000000003628 r8/a4:00000000100f9000
r9/a5:00000000100f59d8 r10/a6:00000000100f59d8 r11/a7:00000000100c7e80
r12/t0:000000000fb3fdfc r13/t1:0000000000000051 r14/t2:0000000000000000
r15/t3:0000000000000009 r16/s0:00000000100ce6e8 r17/s1:000000007fff2fa4
r18/s2:0000000000001b5b r19/s3:000000007fff2fc0 r20/s4:0000000000000000
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000620 r25/t9:0000000010060300 r26/k0:0000000000000000
r27/k1:000000000038361a r28/gp:000000001008c3f0 r29/sp:000000007fff2a10
r30/s8:0000000000000000 r31/ra:000000001005ea6c EPC:0000000010060348
CAUSE=c, SR=400ff33, BADVADDR=100f9000
===============================================================================
STACK TRACE FOR UTHREAD 0x9024c000 (afsd, PID=689):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 sv_queue[../os/ksync/mutex.c: 1595, 0x880dcdf8]
4 sv_timedwait_sig[../os/ksync/mutex.c: 2286, 0x880ddc14]
5 sv_wait_sig[../os/ksync/mutex.c: 1406, 0x880dcb64]
6 sbunlock_wait[../bsd/socket/uipc_socket2.c: 365, 0x88078134]
7 soreceive[../bsd/socket/uipc_socket.c: 986, 0x8807af70]
8 osi_NetReceive[/var/tmp/openafs-1.2.9/src/libafs/rx/rx_knet.c: 73, 0x88322580]
9 rxk_ReadPacket[/var/tmp/openafs-1.2.9/src/libafs/rx/rx_kcommon.c: 966, 0x88340dc4]
10 rxk_Listener[/var/tmp/openafs-1.2.9/src/libafs/rx/rx_kcommon.c: 1072, 0x883411e4]
11 afs_syscall_call[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 383, 0x883170b4]
12 Afs_syscall[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 1023, 0x8831a45c]
13 syscall[../os/trap.c: 2832, 0x880f5294]
14 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:00000000000457c0 r2/v0:0000000000000431
r3/v1:000000000000002c r4/a0:0000000000000030 r5/a1:0000000000000802
r6/a2:0000000000000000 r7/a3:0000000000000000 r8/a4:0000000000000000
r9/a5:0000000000000006 r10/a6:0000000000000000 r11/a7:0000000000000000
r12/t0:0000000000000000 r13/t1:0000000000000001 r14/t2:000000000fb73924
r15/t3:0000000000000008 r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:000000000fa8324c r26/k0:0000000000000000
r27/k1:000000000037c11a r28/gp:00000000100608c8 r29/sp:000000007ffef9b0
r30/s8:0000000000000000 r31/ra:000000001000c6b4 EPC:0000000010013d80
CAUSE=4, SR=400ff33, BADVADDR=7fff2e58
===============================================================================
STACK TRACE FOR UTHREAD 0x8b07f600 (afsd, PID=690):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 sv_queue[../os/ksync/mutex.c: 1595, 0x880dcdf8]
4 sv_timedwait[../os/ksync/mutex.c: 2205, 0x880dda6c]
5 sv_wait[../os/ksync/mutex.c: 1392, 0x880dcb40]
6 rx_GetCall[/var/tmp/openafs-1.2.9/src/libafs/rx/rx.c: 1528, 0x882dbeb8]
7 rxi_ServerProc[/var/tmp/openafs-1.2.9/src/libafs/rx/rx.c: 1293, 0x882dae9c]
8 rx_ServerProc[/var/tmp/openafs-1.2.9/src/libafs/rx/rx_kcommon.c: 257, 0x883403fc]
9 afs_RXCallBackServer[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_callback.c: 830, 0x882ff76c]
10 afs_syscall_call[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 356, 0x88316f80]
11 Afs_syscall[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 1023, 0x8831a45c]
12 syscall[../os/trap.c: 2832, 0x880f5294]
13 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:00000000000457c0 r2/v0:0000000000000431
r3/v1:000000000000002c r4/a0:0000000000000000 r5/a1:0000000000000802
r6/a2:0000000000000000 r7/a3:0000000000000000 r8/a4:0000000000000000
r9/a5:0000000000000006 r10/a6:0000000000000000 r11/a7:0000000000000000
r12/t0:0000000000000000 r13/t1:0000000000000001 r14/t2:000000000fb73924
r15/t3:0000000000000008 r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:000000000fa8324c r26/k0:0000000000000000
r27/k1:00000000003a731a r28/gp:00000000100608c8 r29/sp:000000007ffef9b0
r30/s8:0000000000000000 r31/ra:000000001000c6b4 EPC:0000000010013d80
CAUSE=4, SR=400ff33, BADVADDR=7fff2e58
===============================================================================
STACK TRACE FOR UTHREAD 0x8ae29000 (afsd, PID=691):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 sv_queue[../os/ksync/mutex.c: 1595, 0x880dcdf8]
4 sv_timedwait[../os/ksync/mutex.c: 2205, 0x880dda6c]
5 osi_TimedSleep[/var/tmp/openafs-1.2.9/src/libafs/afs/osi_sleep.c: 174, 0x883512c4]
6 afs_osi_Wait[/var/tmp/openafs-1.2.9/src/libafs/afs/osi_sleep.c: 62, 0x88350c54]
7 afs_rxevent_daemon[/var/tmp/openafs-1.2.9/src/libafs/rx/rx_kcommon.c: 917, 0x88340bd0]
8 afs_syscall_call[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 450, 0x883174c4]
9 Afs_syscall[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 1023, 0x8831a45c]
10 syscall[../os/trap.c: 2832, 0x880f5294]
11 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:00000000000457c0 r2/v0:0000000000000431
r3/v1:000000000000002c r4/a0:0000000000000013 r5/a1:0000000000000002
r6/a2:000000007fff2e30 r7/a3:0000000000000000 r8/a4:0000000000000000
r9/a5:0000000000000006 r10/a6:0000000000000000 r11/a7:0000000000000000
r12/t0:0000000000000000 r13/t1:0000000000000001 r14/t2:000000000fb73924
r15/t3:0000000000000008 r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:000000000fa8324c r26/k0:0000000000000000
r27/k1:00000000003a819a r28/gp:00000000100608c8 r29/sp:000000007ffef9b0
r30/s8:0000000000000000 r31/ra:000000001000c6b4 EPC:0000000010013d80
CAUSE=4, SR=400ff33, BADVADDR=7fff2e58
===============================================================================
STACK TRACE FOR UTHREAD 0x8c772000 (afsd, PID=692):
1 dumpsys[../os/vmdump.c: 528, 0x881757e0]
2 syncreboot[../os/printf.c: 1677, 0x8814cfb4]
3 icmn_err_tag[../os/printf.c: 593, 0x8814bac4]
4 panic[../os/printf.c: 795, 0x8814bf3c]
===============================================================================
STACK TRACE FOR UTHREAD 0x8b767000 (afsd, PID=693):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 mutex_wait[../os/ksync/mutex.c: 955, 0x880dc460]
4 mutex_lock[../os/ksync/mutex.c: 516, 0x880dbd84]
5 afs_osi_Sleep[/var/tmp/openafs-1.2.9/src/libafs/afs/osi_sleep.c: 141, 0x88351008]
6 afs_CheckServerDaemon[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_daemons.c: 64, 0x88344480]
7 afs_syscall_call[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 409, 0x8831723c]
8 Afs_syscall[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 1023, 0x8831a45c]
9 syscall[../os/trap.c: 2832, 0x880f5294]
10 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:00000000000457c0 r2/v0:0000000000000431
r3/v1:000000000000002c r4/a0:0000000000000004 r5/a1:0000000000000000
r6/a2:0000000000000000 r7/a3:0000000000000000 r8/a4:0000000000000000
r9/a5:0000000000000006 r10/a6:0000000000000000 r11/a7:0000000000000000
r12/t0:0000000000000000 r13/t1:0000000000000001 r14/t2:000000000fb73924
r15/t3:0000000000000008 r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:000000000fa8324c r26/k0:0000000000000000
r27/k1:00000000003a889a r28/gp:00000000100608c8 r29/sp:000000007ffef9b0
r30/s8:0000000000000000 r31/ra:000000001000c6b4 EPC:0000000010013d80
CAUSE=4, SR=400ff33, BADVADDR=7fff2e58
===============================================================================
STACK TRACE FOR UTHREAD 0x8c9ba000 (afsd, PID=694):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 sv_queue[../os/ksync/mutex.c: 1595, 0x880dcdf8]
4 sv_timedwait[../os/ksync/mutex.c: 2205, 0x880dda6c]
5 sv_wait[../os/ksync/mutex.c: 1392, 0x880dcb40]
6 afs_osi_Sleep[/var/tmp/openafs-1.2.9/src/libafs/afs/osi_sleep.c: 141, 0x88350f70]
7 afs_BackgroundDaemon[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_daemons.c: 1284, 0x883465f0]
8 afs_syscall_call[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 429, 0x88317334]
9 Afs_syscall[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 1023, 0x8831a45c]
10 syscall[../os/trap.c: 2832, 0x880f5294]
11 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:00000000000457c0 r2/v0:0000000000000431
r3/v1:000000000000002c r4/a0:0000000000000002 r5/a1:fffffffffffffff8
r6/a2:0000000000000000 r7/a3:0000000000000000 r8/a4:0000000000000000
r9/a5:0000000000000006 r10/a6:0000000000000000 r11/a7:0000000000000000
r12/t0:0000000000000000 r13/t1:0000000000000001 r14/t2:000000000fb73924
r15/t3:0000000000000008 r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:000000000fa8324c r26/k0:0000000000000000
r27/k1:00000000003a851a r28/gp:00000000100608c8 r29/sp:000000007ffef9b0
r30/s8:0000000000000000 r31/ra:000000001000c6b4 EPC:0000000010013d80
CAUSE=4, SR=400ff33, BADVADDR=7fff2e58
===============================================================================
STACK TRACE FOR UTHREAD 0x8e9e6000 (afsd, PID=695):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 sv_queue[../os/ksync/mutex.c: 1595, 0x880dcdf8]
4 sv_timedwait[../os/ksync/mutex.c: 2205, 0x880dda6c]
5 sv_wait[../os/ksync/mutex.c: 1392, 0x880dcb40]
6 afs_osi_Sleep[/var/tmp/openafs-1.2.9/src/libafs/afs/osi_sleep.c: 141, 0x88350f70]
7 afs_BackgroundDaemon[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_daemons.c: 1284, 0x883465f0]
8 afs_syscall_call[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 429, 0x88317334]
9 Afs_syscall[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 1023, 0x8831a45c]
10 syscall[../os/trap.c: 2832, 0x880f5294]
11 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:00000000000457c0 r2/v0:0000000000000431
r3/v1:000000000000002c r4/a0:0000000000000002 r5/a1:fffffffffffffff8
r6/a2:0000000000000000 r7/a3:0000000000000000 r8/a4:0000000000000001
r9/a5:0000000000000003 r10/a6:0000000000000001 r11/a7:0000000000000000
r12/t0:0000000000000000 r13/t1:0000000000000001 r14/t2:000000000fb73924
r15/t3:0000000000000008 r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:000000000fa8324c r26/k0:0000000000000000
r27/k1:00000000003a791a r28/gp:00000000100608c8 r29/sp:000000007ffef9b0
r30/s8:0000000000000000 r31/ra:000000001000c6b4 EPC:0000000010013d80
CAUSE=4, SR=400ff33, BADVADDR=7fff2e58
===============================================================================
STACK TRACE FOR UTHREAD 0x8ea18000 (afsd, PID=696):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 thread_block[../os/ksync/mutex.c: 178, 0x880dba20]
3 sv_queue[../os/ksync/mutex.c: 1595, 0x880dcdf8]
4 sv_timedwait[../os/ksync/mutex.c: 2205, 0x880dda6c]
5 sv_wait[../os/ksync/mutex.c: 1392, 0x880dcb40]
6 afs_osi_Sleep[/var/tmp/openafs-1.2.9/src/libafs/afs/osi_sleep.c: 141, 0x88350f70]
7 afs_BackgroundDaemon[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_daemons.c: 1284, 0x883465f0]
8 afs_syscall_call[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 429, 0x88317334]
9 Afs_syscall[/var/tmp/openafs-1.2.9/src/libafs/afs/afs_call.c: 1023, 0x8831a45c]
10 syscall[../os/trap.c: 2832, 0x880f5294]
11 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:00000000000457c0 r2/v0:0000000000000431
r3/v1:000000000000002c r4/a0:0000000000000002 r5/a1:fffffffffffffff8
r6/a2:0000000000000000 r7/a3:0000000000000000 r8/a4:0000000000000001
r9/a5:0000000000000003 r10/a6:0000000000000002 r11/a7:0000000000000000
r12/t0:0000000000000000 r13/t1:0000000000000001 r14/t2:000000000fb73924
r15/t3:0000000000000008 r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:000000000fa8324c r26/k0:0000000000000000
r27/k1:00000000003a869a r28/gp:00000000100608c8 r29/sp:000000007ffef9b0
r30/s8:0000000000000000 r31/ra:000000001000c6b4 EPC:0000000010013d80
CAUSE=4, SR=400ff33, BADVADDR=7fff2e58
===============================================================================
STACK TRACE FOR UTHREAD 0x88b49000 (bosserver, PID=699):
1 swtch[../os/swtch.c: 1161, 0x8816a7a4]
2 qswtch[../os/swtch.c: 174, 0x88169fb8]
3 kpswtch[../os/swtch.c: 238, 0x8816a110]
4 VEC_int[../ml/LOCORE/vec_int.s: 442, 0x88004584]
r0/zero:0000000000000000 r1/at:0000000000015376 r2/v0:ffffffff883dcd18
r3/v1:0000000000000000 r4/a0:ffffffff8aab3480 r5/a1:0000000000000000
r6/a2:0000000000000019 r7/a3:ffffffff80000002 r8/a4:ffffffff88b490cc
r9/a5:0000000000000000 r10/a6:0000000000000001 r11/a7:0000000000000003
r12/t0:0000000000000000 r13/t1:0000000000000000 r14/t2:0000000000000001
r15/t3:0000000000000001 r16/s0:ffffffff8aab3480 r17/s1:ffffffff88b49000
r18/s2:ffffffff904f86f0 r19/s3:ffffffffffffcd30 r20/s4:ffffffff8b65c880
r21/s5:0000000000000008 r22/s6:0000000000000010 r23/s7:0000000000000000
r24/t8:0000000000000000 r25/t9:0000000000000007 r26/k0:0000000000004fea
r27/k1:ffffffffffffcb50 r28/gp:0000000000000000 r29/sp:ffffffffffffcab0
r30/s8:ffffffff904f8000 r31/ra:ffffffff881944a4 EPC:ffffffff881334c0
CAUSE=8000, SR=ff03, BADVADDR=ffffffffc01e6000
5 vn_rele[../os/vnode.c: 1734, 0x881334c0]
6 remove_proc[../os/exec.c: 1023, 0x8819449c]
7 elf2exec[../os/elf.c: 501, 0x8819bc04]
8 elfexec[../os/elf.c: 446, 0x8819bb58]
9 gexec[../os/exec.c: 478, 0x881939e8]
10 iexec[../os/exec.c: 221, 0x88193338]
11 exece[../os/exec.c: 129, 0x881931bc]
12 syscall[../os/trap.c: 2832, 0x880f5294]
13 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:0000000000000000 r2/v0:0000000000000423
r3/v1:00000000000001c0 r4/a0:00000000100b8778 r5/a1:000000001008b420
r6/a2:000000007fff2fac r7/a3:000000007fff2fac r8/a4:0000000000000004
r9/a5:0000000000000000 r10/a6:0000000010073380 r11/a7:000000001008b668
r12/t0:0000000000000000 r13/t1:0000000000000000 r14/t2:0000000000000065
r15/t3:0000000010053708 r16/s0:0000000000000001 r17/s1:000000007fff2fa4
r18/s2:000000007fff2fac r19/s3:000000007fff2fc0 r20/s4:0000000000000000
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:000000000000053b r25/t9:000000000fa811f0 r26/k0:0000000000000000
r27/k1:0000000000224d9a r28/gp:000000000fb3fdfc r29/sp:000000001008b398
r30/s8:0000000000000000 r31/ra:000000000fa8121c EPC:000000000fa81248
CAUSE=8, SR=2400ff33, BADVADDR=fa811f0
===============================================================================
STACK TRACE FOR UTHREAD 0x8d233600 (afsd, PID=703):
1 thread_save[../os/fork.c: 1124, 0x88179d70]
2 procfork[../os/fork.c: 900, 0x88179a68]
3 dofork[../os/fork.c: 773, 0x88179770]
4 fork[../os/fork.c: 154, 0x88178dc4]
5 syscall[../os/trap.c: 2832, 0x880f5294]
6 systrap[../ml/LOCORE/systrap.s: 315, 0x88015a70]
r0/zero:0000000000000000 r1/at:0000000000000001 r2/v0:00000000000003ea
r3/v1:000000000fa466bc r4/a0:0000000000000001 r5/a1:0000000000000000
r6/a2:0000000000000000 r7/a3:0000000000046654 r8/a4:0000000000000000
r9/a5:0000000000000000 r10/a6:000000000fb377c8 r11/a7:0000000000000001
r12/t0:0000000000000061 r13/t1:0000000000000000 r14/t2:0000000000000000
r15/t3:0000000000000000 r16/s0:0000000010061560 r17/s1:fffffffffffffff7
r18/s2:0000000000000000 r19/s3:0000000010061060 r20/s4:0000000010061178
r21/s5:0000000000000000 r22/s6:0000000000000000 r23/s7:0000000000000000
r24/t8:000000007ffeee71 r25/t9:000000000fa3b2a8 r26/k0:0000000000000000
r27/k1:0000000000387c9a r28/gp:000000000fb3fdfc r29/sp:000000007ffef9d0
r30/s8:0000000000000000 r31/ra:000000000fa43b38 EPC:000000000fa43b68
CAUSE=ffffffff80000004, SR=400ff33, BADVADDR=fb3ea38
===============================================================================
44 uthread traces found
[cut]
kthread
ACTIVE STHREADS:
[cut]
ACTIVE UTHREADS:
KTHREAD TYPE ID WCHAN NAME
=============================================================================
887dc000 3 100000020 887dc0a0 init
887c8000 3 10000003d 8894155c rc2
88c1a000 3 10000006a 88b3f518 syslogd
8c74a600 3 100000080 8c74a6a0 eventmond
8c73e600 3 10000007f 8c73e9b0 eventmond
8c3e7600 3 10000007e 8c3e79b0 eventmond
8c3e1600 3 10000007d 8c748060 eventmond
8c726600 3 10000007c 8c7269b0 eventmond
8c3dd600 3 10000007b 885d9f60 eventmond
8c69e600 3 10000007a 885d9e60 eventmond
8c2e4600 3 100000079 8c2e49b0 eventmond
8bf6f600 3 1000000d9 8bf6f6a0 nsd
88d3c000 3 1000000e7 88d3c0a0 inetd
8ded3600 3 1000000ee 8ded36a0 snetd
8e914600 3 1000000f5 00000000 perl5.6.1-n32
8aff2000 3 1000000f9 00000000 prngd
8f266600 3 1000000fe 8f2666a0 sshd
8f1a7000 3 100000110 8f1a70a0 sshd
8f152600 3 100000111 8f1526a0 lpd
9024c600 3 10000016d 8e20d55c sh
9059b600 3 10000019b 00000000 ntpd
8ee03000 3 1000001ab 00000000 bash
8e9ea000 3 100000262 8f893a50 cron
8ec55600 3 100000280 8f3eb55c sh
8f482600 3 100000282 912da398 logger
8f61a000 3 100000285 00000000 sgindexAdmin
8b759000 3 1000002b7 8ea2355c sh
8cde6000 3 1000002cc 8cde60a0 bosserver
88b3d000 3 1000002d3 885a08d8 afsd
8c78a600 3 1000002de 00000000 vlserver
9024c000 3 1000002df 912b2260 afsd
8b07f600 3 1000002e0 8fa0aa94 afsd
8ae29000 3 1000002e1 912b2810 afsd
8c772000 3 1000002e2 00000000 afsd
8b767000 3 1000002e3 883fa548 afsd
8c9ba000 3 1000002e4 912b2d10 afsd
8e9e6000 3 1000002e5 912b2d10 afsd
8ea18000 3 1000002e6 912b2d10 afsd
8c8d4000 3 1000002e7 00000000 klogpp
88a4a600 3 1000002e8 00000000 perl5.6.1-n32
8fad5600 3 1000002eb 00000000 prngd
88b3d600 3 1000002ea 00000000 bash
88b49000 3 1000002e9 00000000 bosserver
8d233600 3 1000002ed 00000000 afsd
=============================================================================
44 active uthreads found
[cut]
>> report
=======================
ICRASH CORE FILE REPORT
=======================
SYSTEM:
system name: IRIX
release: 6.5 (6.5.20f)
node name: nmrindy
version: 04100802
machine name: IP22
GENERATED ON:
Wed Jul 9 12:14:12 2003
TIME OF CRASH:
1057674094 Tue Jul 8 16:21:34 2003
PANIC STRING:
PANIC: stack underflow/overflow
NAMELIST:
unix.0 [CREATE TIME: Wed Jul 9 10:27:51 2003]
COREFILE:
vmcore.0.comp [CREATE TIME: Wed Jul 9 10:28:12 2003]
================
COREFILE SUMMARY
================
The system was brought down due to an internal panic.
===========
PUTBUF DUMP
===========
<6>IRIX Release 6.5 IP22 Version 04100802 System V
Copyright 1987-2003 Silicon Graphics, Inc.
All Rights Reserved.
<5>NOTICE: Start mounting filesystem: /
<5>NOTICE: Starting XFS recovery on filesystem: / (dev: 0/74)
<5>NOTICE: Ending XFS recovery for filesystem: / (/hw/node/io/gio/hpc/scsi_ctlr/0/target/5/lun/0/disk/partition/0/block)
<5>NOTICE: Start mounting filesystem: /vicepc
<5>NOTICE: Ending clean XFS mount for filesystem: /vicepc
<5>NOTICE: Start mounting filesystem: /vicepb
<5>NOTICE: Ending clean XFS mount for filesystem: /vicepb
<5>NOTICE: Start mounting filesystem: /vicepa
<5>NOTICE: Ending clean XFS mount for filesystem: /vicepa
<6>Starting AFS cache scan...<6>found 0 non-empty cache files (0%).
<6>Kernel/Interrupt Stack Overflow @0x88118dd8 sp:0xffffae08 k1:0xffffaf48
<6>ra:0x88105620 stkflag:1
<0>PANIC: stack underflow/overflow
<6>
Dumping to /hw/node/io/gio/hpc/scsi_ctlr/0/target/5/lun/0/disk/partition/1/block at block 0, space: 0x8000 pages
<6>Dumping low memory...<6>
<6>Dumping static kernel pages...<6>.<6>.<6>.
===========
CPU SUMMARY
===========
CPU 0 was idle
STACK TRACE:
===============================================================================
STACK TRACE FOR UTHREAD 0x8c772000 (afsd, PID=692):
1 dumpsys[../os/vmdump.c: 528, 0x881757e0]
2 syncreboot[../os/printf.c: 1677, 0x8814cfb4]
3 icmn_err_tag[../os/printf.c: 593, 0x8814bac4]
4 panic[../os/printf.c: 795, 0x8814bf3c]
===============================================================================
=======================
CRASH SUMMARY FOR CPU 0
=======================
The command 'afsd' was running.
1 dumpsys[../os/vmdump.c: 528, 0x881757e0]
2 syncreboot[../os/printf.c: 1677, 0x8814cfb4]
3 icmn_err_tag[../os/printf.c: 593, 0x8814bac4]
4 panic[../os/printf.c: 795, 0x8814bf3c]
>>
And I don't know if this is relevant at all, but I see now on that
machine(with IBM AFS kernel) but rest are OpenAFS 1.2.9a binaries:
# more /usr/afs/logs/*Log*
Wed Jul 9 11:40:55 2003: Server directory access is okay
Wed Jul 9 11:40:55 2003: bosserver: Something is wrong (-1) with the bos config
uration file /usr/afs/local/BosConfig; aborting
...skipping...
Wed Jul 9 11:40:55 2003: Server directory access is okay
Wed Jul 9 11:40:55 2003: bosserver: Something is wrong (-1) with the bos configuration file /usr/afs/local/BosConfig;
aborting
...skipping...
...skipping...
...skipping...
ptserver: problems with host name Ubik init failed
erver: 195.113.59.251
Tue Jul 8 16:02:49 2003 Inconsistent Cell Info on server: 195.113.59.121
...skipping...
bash-2.05b# cat /usr/afs/local/BosConfig
bash-2.05b# ls -la /usr/afs/local/BosConfig
-rw-r--r-- 1 root sys 156 Jul 8 16:21 /usr/afs/local/BosConfig
bash-2.05b# od -c /usr/afs/local/BosConfig
0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
*
0000220 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0000234
bash-2.05b#
I removed the file and retried(the file got regenerated later and
properly):
bash-2.05b# /usr/afs/bin/bosserver
bash-2.05b# bos delete -instance fs -server nmrindy
bos: failed to delete instance 'fs' (no such entity)
bash-2.05b# bos status -server nmrindy -long
Instance ptserver, (type is simple) currently running normally.
Process last started at Wed Jul 9 12:38:25 2003 (1 proc starts)
Command 1 is '/usr/afs/bin/ptserver'
Instance vlserver, (type is simple) temporarily disabled, stopped for too many errors, currently shutdown.
Process last started at Wed Jul 9 12:38:28 2003 (13 proc starts)
Last exit at Wed Jul 9 12:38:28 2003
Last error exit at Wed Jul 9 12:38:28 2003, by exiting with code 2
Command 1 is '/usr/afs/bin/ptserver'
bash-2.05b# bos create nmrindy fs fs /usr/afs/bin/fileserver /usr/afs/bin/volserver /usr/afs/bin/salvager -cell natur.cuni.cz
bash-2.05b# bos status -server nmrindy -long
Instance ptserver, (type is simple) temporarily disabled, stopped for too many errors, currently shutdown.
Process last started at Wed Jul 9 12:38:38 2003 (13 proc starts)
Last exit at Wed Jul 9 12:38:38 2003
Last error exit at Wed Jul 9 12:38:38 2003, by exiting with code 2
Command 1 is '/usr/afs/bin/ptserver'
Instance vlserver, (type is simple) temporarily disabled, stopped for too many errors, currently shutdown.
Process last started at Wed Jul 9 12:38:28 2003 (13 proc starts)
Last exit at Wed Jul 9 12:38:28 2003
Last error exit at Wed Jul 9 12:38:28 2003, by exiting with code 2
Command 1 is '/usr/afs/bin/ptserver'
Instance fs, (type is fs) currently running normally.
Auxiliary status is: file server running.
Process last started at Wed Jul 9 12:38:52 2003 (2 proc starts)
Command 1 is '/usr/afs/bin/fileserver'
Command 2 is '/usr/afs/bin/volserver'
Command 3 is '/usr/afs/bin/salvager'
bash-2.05b# bos status -server nmrindy -long
Instance ptserver, (type is simple) temporarily disabled, stopped for too many errors, currently shutdown.
Process last started at Wed Jul 9 12:38:38 2003 (13 proc starts)
Last exit at Wed Jul 9 12:38:38 2003
Last error exit at Wed Jul 9 12:38:38 2003, by exiting with code 2
Command 1 is '/usr/afs/bin/ptserver'
Instance vlserver, (type is simple) temporarily disabled, stopped for too many errors, currently shutdown.
Process last started at Wed Jul 9 12:38:28 2003 (13 proc starts)
Last exit at Wed Jul 9 12:38:28 2003
Last error exit at Wed Jul 9 12:38:28 2003, by exiting with code 2
Command 1 is '/usr/afs/bin/ptserver'
Instance fs, (type is fs) currently running normally.
Auxiliary status is: file server running.
Process last started at Wed Jul 9 12:38:52 2003 (2 proc starts)
Command 1 is '/usr/afs/bin/fileserver'
Command 2 is '/usr/afs/bin/volserver'
Command 3 is '/usr/afs/bin/salvager'
bash-2.05b# bos status -server nmrindy -long
bos: failed to contact host's bosserver (communications failure (-1)).
bash-2.05b#
bash-2.05b# for f in /usr/afs/logs/*Log*; do echo $f; cat $f; done
/usr/afs/logs/BosLog
Wed Jul 9 12:38:25 2003: Server directory access is okay
Wed Jul 9 12:38:25 2003: vlserver exited with code 2
Wed Jul 9 12:38:26 2003: vlserver exited with code 2
Wed Jul 9 12:38:26 2003: vlserver exited with code 2
Wed Jul 9 12:38:26 2003: vlserver exited with code 2
Wed Jul 9 12:38:26 2003: vlserver exited with code 2
Wed Jul 9 12:38:27 2003: vlserver exited with code 2
Wed Jul 9 12:38:27 2003: vlserver exited with code 2
Wed Jul 9 12:38:27 2003: vlserver exited with code 2
Wed Jul 9 12:38:27 2003: vlserver exited with code 2
Wed Jul 9 12:38:28 2003: vlserver exited with code 2
Wed Jul 9 12:38:28 2003: vlserver exited with code 2
Wed Jul 9 12:38:28 2003: vlserver exited with code 2
Wed Jul 9 12:38:28 2003: BNODE 'vlserver' repeatedly failed to start, perhaps missing executable.
Wed Jul 9 12:38:28 2003: vlserver exited with code 2
Wed Jul 9 12:38:28 2003: BNODE 'vlserver' repeatedly failed to start, perhaps missing executable.
Wed Jul 9 12:38:35 2003: ptserver exited with code 2
Wed Jul 9 12:38:36 2003: ptserver exited with code 2
Wed Jul 9 12:38:36 2003: ptserver exited with code 2
Wed Jul 9 12:38:36 2003: ptserver exited with code 2
Wed Jul 9 12:38:36 2003: ptserver exited with code 2
Wed Jul 9 12:38:36 2003: ptserver exited with code 2
Wed Jul 9 12:38:37 2003: ptserver exited with code 2
Wed Jul 9 12:38:37 2003: ptserver exited with code 2
Wed Jul 9 12:38:37 2003: ptserver exited with code 2
Wed Jul 9 12:38:38 2003: ptserver exited with code 2
Wed Jul 9 12:38:38 2003: ptserver exited with code 2
Wed Jul 9 12:38:38 2003: ptserver exited with code 2
Wed Jul 9 12:38:38 2003: BNODE 'ptserver' repeatedly failed to start, perhaps missing executable.
Wed Jul 9 12:38:38 2003: ptserver exited with code 2
Wed Jul 9 12:38:38 2003: BNODE 'ptserver' repeatedly failed to start, perhaps missing executable.
/usr/afs/logs/BosLog.old
Wed Jul 9 12:36:19 2003: Server directory access is okay
Wed Jul 9 12:36:20 2003: vlserver exited with code 2
Wed Jul 9 12:36:20 2003: vlserver exited with code 2
Wed Jul 9 12:36:20 2003: vlserver exited with code 2
Wed Jul 9 12:36:21 2003: vlserver exited with code 2
Wed Jul 9 12:36:21 2003: vlserver exited with code 2
Wed Jul 9 12:36:21 2003: vlserver exited with code 2
Wed Jul 9 12:36:21 2003: vlserver exited with code 2
/usr/afs/logs/FileLog
Wed Jul 9 12:38:52 2003 XFS/EFS File server starting
Wed Jul 9 12:38:52 2003 /usr/afs/local/sysid: doesn't exist
Wed Jul 9 12:38:52 2003 Creating new SysID file
Wed Jul 9 12:38:53 2003 Set thread id 14 for FSYNC_sync
Wed Jul 9 12:38:53 2003 Partition /vicepc: attached 0 volumes; 0 volumes not attached
Wed Jul 9 12:38:53 2003 Partition /vicepb: attached 0 volumes; 0 volumes not attached
Wed Jul 9 12:38:53 2003 Partition /vicepa: attached 0 volumes; 0 volumes not attached
Wed Jul 9 12:38:53 2003 Set thread id 15 for 'FiveMinuteCheckLWP'
Wed Jul 9 12:38:53 2003 Set thread id 16 for 'HostCheckLWP'
Wed Jul 9 12:38:53 2003 Getting FileServer name...
Wed Jul 9 12:38:53 2003 FileServer host name is 'nmrindy.natur.cuni.cz'
Wed Jul 9 12:38:53 2003 Getting FileServer address...
Wed Jul 9 12:38:53 2003 FileServer nmrindy.natur.cuni.cz has address 195.113.59.111 (0xc3713b6f or 0xc3713b6f in host byte order)
Wed Jul 9 12:38:53 2003 File Server started Wed Jul 9 12:38:53 2003
/usr/afs/logs/PtLog
ptserver: problems with host name Ubik init failed
erver: Wed Jul 9 12:38:38 2003 195.113.59.251 Wed Jul 9 12:38:38 2003
Wed Jul 9 12:38:38 2003 Inconsistent Cell Info on server: Wed Jul 9 12:38:38 2003 195.113.59.121 Wed Jul 9 12:38:38 2003
/usr/afs/logs/PtLog.old
ptserver: problems with host name Ubik init failed
erver: Wed Jul 9 12:38:38 2003 195.113.59.251 Wed Jul 9 12:38:38 2003
Wed Jul 9 12:38:38 2003 Inconsistent Cell Info on server: Wed Jul 9 12:38:38 2003 195.113.59.121 Wed Jul 9 12:38:38 2003
/usr/afs/logs/VolserLog
Wed Jul 9 12:38:55 2003 Starting AFS Volserver 2.0 (/usr/afs/bin/volserver)
bash-2.05b#
It seems because of vlserver exiting also bosserver dies.
--
Martin Mokrejs <mmokrejs@natur.cuni.cz>, <m.mokrejs@gsf.de>
PGP5.0i key is at http://www.natur.cuni.cz/~mmokrejs
MIPS / Institute for Bioinformatics <http://mips.gsf.de>
GSF - National Research Center for Environment and Health
Ingolstaedter Landstrasse 1, D-85764 Neuherberg, Germany
tel.: +49-89-3187 3683 , fax: +49-89-3187 3585