[OpenAFS] Re: 1.6.2 buserver + butc
Eric Sturdivant
sturdiva@umd.edu
Wed, 27 Mar 2013 16:05:06 -0400 (EDT)
On Wed, 27 Mar 2013, Andrew Deason wrote:
> On Wed, 27 Mar 2013 14:53:26 -0400
> Derrick Brashear <shadow@gmail.com> wrote:
>
>> there won't be anything sensitive in just a stack trace, and on
>> solaris that is easily generated with just running pstack and could be
>> shared with impunity. it may or may not be enough information but it's
>> certainly a place to start.
>
> These are LWP, so I was expecting all we'd see is IOMGR. I suppose it's
> _possible_ it could be something useful; worth a shot, at least.
>
>
Info from the buserver and butc:
Core was generated by `/usr/afs/bin/buserver'.
#0 0xff18e4a4 in __pollsys () from /lib/libc.so.1
(gdb) info threads
Id Target Id Frame
* 2 Thread 1 (LWP 1) 0xff18e4a4 in __pollsys () from /lib/libc.so.1
1 LWP 1 0xff18e4a4 in __pollsys () from /lib/libc.so.1
(gdb) thread 1
[Switching to thread 1 (LWP 1 )]
#0 0xff18e4a4 in __pollsys () from /lib/libc.so.1
(gdb) where
#0 0xff18e4a4 in __pollsys () from /lib/libc.so.1
#1 0xff17e190 in _pollsys () from /lib/libc.so.1
#2 0xff12a150 in pselect_large_fdset () from /lib/libc.so.1
#3 0xff12a598 in select_large_fdset () from /lib/libc.so.1
#4 0x0007473c in ?? ()
#5 0x0007473c in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 2
[Switching to thread 2 (Thread 1 (LWP 1))]
#0 0xff18e4a4 in __pollsys () from /lib/libc.so.1
(gdb) where
#0 0xff18e4a4 in __pollsys () from /lib/libc.so.1
#1 0xff17e190 in _pollsys () from /lib/libc.so.1
#2 0xff12a150 in pselect_large_fdset () from /lib/libc.so.1
#3 0xff12a598 in select_large_fdset () from /lib/libc.so.1
#4 0x0007473c in ?? ()
#5 0x0007473c in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
Core was generated by `/usr/afsws/etc/butc 3106 -localauth'.
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) info threads
Id Target Id Frame
16 Thread 8 0xff17ac78 in __lwp_park () from /lib/libc.so.1
15 Thread 7 0xff17ac78 in __lwp_park () from /lib/libc.so.1
14 Thread 6 0xff17ac78 in __lwp_park () from /lib/libc.so.1
13 Thread 5 0xff17ac78 in __lwp_park () from /lib/libc.so.1
12 Thread 4 (LWP 4) 0xff17dacc in ___nanosleep () from /lib/libc.so.1
11 Thread 3 0xff17ac78 in __lwp_park () from /lib/libc.so.1
10 Thread 2 (LWP 2) 0xff17dd64 in _so_recvmsg () from /lib/libc.so.1
* 9 Thread 1 0xff17ac78 in __lwp_park () from /lib/libc.so.1
8 LWP 8 0xff17ac78 in __lwp_park () from /lib/libc.so.1
7 LWP 7 0xff17ac78 in __lwp_park () from /lib/libc.so.1
6 LWP 6 0xff17ac78 in __lwp_park () from /lib/libc.so.1
5 LWP 5 0xff17ac78 in __lwp_park () from /lib/libc.so.1
4 LWP 4 0xff17dacc in ___nanosleep () from /lib/libc.so.1
3 LWP 3 0xff17ac78 in __lwp_park () from /lib/libc.so.1
2 LWP 2 0xff17dd64 in _so_recvmsg () from /lib/libc.so.1
1 LWP 1 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) thread 1
[Switching to thread 1 (LWP 1 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x0007df9c in rx_GetCall ()
#6 0x0007d568 in rxi_ServerProc ()
#7 0x00079e54 in rx_ServerProc ()
#8 0x0007b850 in rx_StartServer ()
#9 0x0002a8e8 in ?? ()
#10 0x0002a8e8 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 2
[Switching to thread 2 (LWP 2 )]
#0 0xff17dd64 in _so_recvmsg () from /lib/libc.so.1
(gdb) where
#0 0xff17dd64 in _so_recvmsg () from /lib/libc.so.1
#1 0x00095b44 in rxi_ReadPacket ()
#2 0x00079c08 in ?? ()
#3 0x00079c08 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 3
[Switching to thread 3 (LWP 3 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff17510c in cond_wait_common () from /lib/libc.so.1
#4 0xff1752a0 in _cond_timedwait () from /lib/libc.so.1
#5 0xff175394 in cond_timedwait () from /lib/libc.so.1
#6 0xff1753d4 in pthread_cond_timedwait () from /lib/libc.so.1
#7 0x00079a14 in ?? ()
#8 0x00079a14 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 4
[Switching to thread 4 (LWP 4 )]
#0 0xff17dacc in ___nanosleep () from /lib/libc.so.1
(gdb) where
#0 0xff17dacc in ___nanosleep () from /lib/libc.so.1
#1 0xff16d408 in sleep () from /lib/libc.so.1
#2 0x0002333c in dbWatcher ()
#3 0xff17abe0 in _lwp_start () from /lib/libc.so.1
#4 0xff17abe0 in _lwp_start () from /lib/libc.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 5
[Switching to thread 5 (LWP 5 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x0007df9c in rx_GetCall ()
#6 0x0007d568 in rxi_ServerProc ()
#7 0x00079e54 in rx_ServerProc ()
#8 0x00079760 in ?? ()
#9 0x00079760 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 6
[Switching to thread 6 (LWP 6 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x0007df9c in rx_GetCall ()
#6 0x0007d568 in rxi_ServerProc ()
#7 0x00079e54 in rx_ServerProc ()
#8 0x00079760 in ?? ()
#9 0x00079760 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 7
[Switching to thread 7 (LWP 7 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x0007df9c in rx_GetCall ()
#6 0x0007d568 in rxi_ServerProc ()
#7 0x00079e54 in rx_ServerProc ()
#8 0x00079760 in ?? ()
#9 0x00079760 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 8
[Switching to thread 8 (LWP 8 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x000989e0 in rxi_ReadProc ()
#6 0x0009c2f4 in ?? ()
#7 0x0009c2f4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 9
[Switching to thread 9 (Thread 1 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x0007df9c in rx_GetCall ()
#6 0x0007d568 in rxi_ServerProc ()
#7 0x00079e54 in rx_ServerProc ()
#8 0x0007b850 in rx_StartServer ()
#9 0x0002a8e8 in ?? ()
#10 0x0002a8e8 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 10
[Switching to thread 10 (Thread 2 (LWP 2))]
#0 0xff17dd64 in _so_recvmsg () from /lib/libc.so.1
(gdb) where
#0 0xff17dd64 in _so_recvmsg () from /lib/libc.so.1
#1 0x00095b44 in rxi_ReadPacket ()
#2 0x00079c08 in ?? ()
#3 0x00079c08 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 11
[Switching to thread 11 (Thread 3 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff17510c in cond_wait_common () from /lib/libc.so.1
#4 0xff1752a0 in _cond_timedwait () from /lib/libc.so.1
#5 0xff175394 in cond_timedwait () from /lib/libc.so.1
#6 0xff1753d4 in pthread_cond_timedwait () from /lib/libc.so.1
#7 0x00079a14 in ?? ()
#8 0x00079a14 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 12
[Switching to thread 12 (Thread 4 (LWP 4))]
#0 0xff17dacc in ___nanosleep () from /lib/libc.so.1
(gdb) where
#0 0xff17dacc in ___nanosleep () from /lib/libc.so.1
#1 0xff16d408 in sleep () from /lib/libc.so.1
#2 0x0002333c in dbWatcher ()
#3 0xff17abe0 in _lwp_start () from /lib/libc.so.1
#4 0xff17abe0 in _lwp_start () from /lib/libc.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 13
[Switching to thread 13 (Thread 5 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x0007df9c in rx_GetCall ()
#6 0x0007d568 in rxi_ServerProc ()
#7 0x00079e54 in rx_ServerProc ()
#8 0x00079760 in ?? ()
#9 0x00079760 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 14
[Switching to thread 14 (Thread 6 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x0007df9c in rx_GetCall ()
#6 0x0007d568 in rxi_ServerProc ()
#7 0x00079e54 in rx_ServerProc ()
#8 0x00079760 in ?? ()
#9 0x00079760 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 15
[Switching to thread 15 (Thread 7 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x0007df9c in rx_GetCall ()
#6 0x0007d568 in rxi_ServerProc ()
#7 0x00079e54 in rx_ServerProc ()
#8 0x00079760 in ?? ()
#9 0x00079760 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) thread 16
[Switching to thread 16 (Thread 8 )]
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
(gdb) where
#0 0xff17ac78 in __lwp_park () from /lib/libc.so.1
#1 0xff174b04 in cond_sleep_queue () from /lib/libc.so.1
#2 0xff174cc8 in cond_wait_queue () from /lib/libc.so.1
#3 0xff175210 in cond_wait () from /lib/libc.so.1
#4 0xff17524c in pthread_cond_wait () from /lib/libc.so.1
#5 0x000989e0 in rxi_ReadProc ()
#6 0x0009c2f4 in ?? ()
#7 0x0009c2f4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
We are going to give it a try with debug turned on, as well as grabbing
tcpdumps from both sides later this evening.
--
Eric Sturdivant
University of Maryland
Division of Information Technology
Enterprise Unix Services