[OpenAFS-devel] afsd-fuse segfaults
Troy Benjegerdes
hozer@hozed.org
Tue, 18 Sep 2012 09:40:04 -0500
On my server, amd64 debian stable, it works.
On my laptop, amd64-debian stable + backports + some testing, segaults.
gdb/backtraces are not particularly informative, either (see end of message)
Is there a clean cross-platform way to install a segfault handler so we can
dump this kind of backtraces out automatically, particularly when doing something
like a 'make check' smoke test?
... some references ...
https://wiki.ubuntu.com/Apport
http://www.cygwin.com/ml/gdb/2007-06/msg00345.html
On Mon, Sep 17, 2012 at 11:39:55PM -0400, Derrick Brashear wrote:
> So. Were you perchance using it on a Mac? Probably a 64 bit Intel mac?
>
> http://gerrit.openafs.org/#change,8132
>
> As nearly as I can tell, this is a very specific problem. The code is fine. The
> circumstances of building afsd.fuse meant it was collateral damage when we
> started using roken, but only on MacOS, and probably only for non-32
> bit pointers,
> because MacOS does something odd with dirent.h
>
> On Mon, Sep 17, 2012 at 1:20 PM, Derrick Brashear <shadow@gmail.com> wrote:
> > On Mon, Sep 17, 2012 at 1:15 PM, Troy Benjegerdes <hozer@hozed.org> wrote:
> >> I'm looking to get all the low-hanging fruit with unskilled testing.
> >> Particularly with regressions like this:
> >>
> >> hozer@six:~/src/openafs-fuse-git/tests/fuse$ /home/hozer/src/openafs-fuse-git/tests/fuse/../../src/afsd/afsd.fuse -dynroot -fakestat -d -confdir /home/hozer/src/openafs-fuse-git/tests/fuse/conf -cachedir /home/hozer/src/openafs-fuse-git/tests/fuse/vcache -mountdir /home/hozer/src/openafs-fuse-git/tests/fuse/mntdir
> >> FUSE library version: 2.8.6
> >> nullpath_ok: 0
> >> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
> >> INIT: 7.17
> >> flags=0x0000047b
> >> max_readahead=0x00020000
> >> Starting AFS cache scan...found 0 non-empty cache files (0%).
> >> afsd: All AFS daemons started.
> >> Segmentation fault
hozer@six:~/src/openafs-fuse-git/tests/fuse$ file ../../src/afsd/afsd.fuse
../../src/afsd/afsd.fuse: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[sha1]=0xd04485c4228faf5d05f75b9f31b37939d54074d7, not stripped
hozer@six:~/src/openafs-fuse-git/tests/fuse$ gdb ../../src/afsd/afsd.fuse
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/hozer/src/openafs-fuse-git/src/afsd/afsd.fuse...done.
(gdb) run -dynroot -fakestat -d -confdir /home/hozer/src/openafs-fuse-git/tests/fuse/conf -cachedir /home/hozer/src/openafs-fuse-git/tests/fuse/vcache -mountdir /home/hozer/src/openafs-fuse-git/tests/fuse/mntdir
Starting program: /home/hozer/src/openafs-fuse-git/src/afsd/afsd.fuse -dynroot -fakestat -d -confdir /home/hozer/src/openafs-fuse-git/tests/fuse/conf -cachedir /home/hozer/src/openafs-fuse-git/tests/fuse/vcache -mountdir /home/hozer/src/openafs-fuse-git/tests/fuse/mntdir
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
FUSE library version: 2.8.6
nullpath_ok: 0
[New Thread 0x7ffff697a700 (LWP 21033)]
[New Thread 0x7ffff6179700 (LWP 21034)]
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
INIT: 7.17
flags=0x0000047b
max_readahead=0x00020000
[New Thread 0x7ffff7f0c700 (LWP 21035)]
[New Thread 0x7ffff7eee700 (LWP 21036)]
[New Thread 0x7ffff7e4f700 (LWP 21037)]
[New Thread 0x7ffff570d700 (LWP 21038)]
[New Thread 0x7ffff56ef700 (LWP 21039)]
[New Thread 0x7ffff5690700 (LWP 21040)]
[New Thread 0x7ffff5631700 (LWP 21041)]
[New Thread 0x7ffff55d2700 (LWP 21042)]
[New Thread 0x7ffff5573700 (LWP 21043)]
Starting AFS cache scan...found 0 non-empty cache files (0%).
afsd: All AFS daemons started.
[New Thread 0x7ffff54d3700 (LWP 21044)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff697a700 (LWP 21033)]
0x0000000000000000 in ?? ()
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x0000000000000000 in ?? ()
(gdb) thread
[Current thread is 2 (Thread 0x7ffff697a700 (LWP 21033))]
(gdb) help thread
Use this command to switch between threads.
The new thread ID must be currently known.
List of thread subcommands:
thread apply -- Apply a command to a list of threads
thread find -- Find threads that match a regular expression
thread name -- Set the current thread's name
Type "help thread" followed by thread subcommand name for full documentation.
Type "apropos word" to search for commands related to "word".
Command name abbreviations are allowed if unambiguous.
(gdb) thread apply gdb
(gdb) thread apply bt
(gdb) thread apply bt all
(gdb) help thread apply
Apply a command to a list of threads.
List of thread apply subcommands:
thread apply all -- Apply a command to all threads
Type "help thread apply" followed by thread apply subcommand name for full documentation.
Type "apropos word" to search for commands related to "word".
Command name abbreviations are allowed if unambiguous.
(gdb) thread apply all bt
Thread 13 (Thread 0x7ffff54d3700 (LWP 21044)):
#0 __memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:385
#1 0x000000000040402f in uafs_InitThread ()
#2 0x0000000000404075 in get_user_struct ()
#3 0x0000000000405c4d in call_syscall ()
#4 0x000000000040e79d in afsd_call_syscall ()
#5 0x000000000040ba6c in call_syscall_thread ()
#6 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#7 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#8 0x0000000000000000 in ?? ()
Thread 12 (Thread 0x7ffff5573700 (LWP 21043)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1 0x0000000000404e81 in afs_osi_Sleep ()
#2 0x0000000000432766 in afs_BackgroundDaemon ()
#3 0x000000000046659f in afs_syscall_call ()
#4 0x0000000000449ef6 in Afs_syscall ()
#5 0x0000000000405c65 in call_syscall ()
#6 0x000000000040e79d in afsd_call_syscall ()
#7 0x000000000040ba07 in afsd_syscall ()
#8 0x000000000040ba27 in daemon_thread ()
#9 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#10 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#11 0x0000000000000000 in ?? ()
Thread 11 (Thread 0x7ffff55d2700 (LWP 21042)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1 0x0000000000404e81 in afs_osi_Sleep ()
#2 0x0000000000432766 in afs_BackgroundDaemon ()
#3 0x000000000046659f in afs_syscall_call ()
#4 0x0000000000449ef6 in Afs_syscall ()
#5 0x0000000000405c65 in call_syscall ()
#6 0x000000000040e79d in afsd_call_syscall ()
#7 0x000000000040ba07 in afsd_syscall ()
#8 0x000000000040ba27 in daemon_thread ()
#9 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#10 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#11 0x0000000000000000 in ?? ()
Thread 10 (Thread 0x7ffff5631700 (LWP 21041)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1 0x0000000000404e81 in afs_osi_Sleep ()
#2 0x0000000000432766 in afs_BackgroundDaemon ()
---Type <return> to continue, or q <return> to quit---
#3 0x000000000046659f in afs_syscall_call ()
#4 0x0000000000449ef6 in Afs_syscall ()
#5 0x0000000000405c65 in call_syscall ()
#6 0x000000000040e79d in afsd_call_syscall ()
#7 0x000000000040ba07 in afsd_syscall ()
#8 0x000000000040ba27 in daemon_thread ()
#9 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#10 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#11 0x0000000000000000 in ?? ()
Thread 9 (Thread 0x7ffff5690700 (LWP 21040)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1 0x0000000000404e81 in afs_osi_Sleep ()
#2 0x0000000000432766 in afs_BackgroundDaemon ()
#3 0x000000000046659f in afs_syscall_call ()
#4 0x0000000000449ef6 in Afs_syscall ()
#5 0x0000000000405c65 in call_syscall ()
#6 0x000000000040e79d in afsd_call_syscall ()
#7 0x000000000040ba07 in afsd_syscall ()
#8 0x000000000040ba27 in daemon_thread ()
#9 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#10 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#11 0x0000000000000000 in ?? ()
Thread 8 (Thread 0x7ffff56ef700 (LWP 21039)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1 0x0000000000404bd6 in afs_osi_Wait ()
#2 0x0000000000431548 in afs_CheckServerDaemon ()
#3 0x0000000000466547 in afs_syscall_call ()
#4 0x0000000000449ef6 in Afs_syscall ()
#5 0x0000000000405c65 in call_syscall ()
#6 0x000000000040e79d in afsd_call_syscall ()
#7 0x000000000040ba6c in call_syscall_thread ()
#8 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#9 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#10 0x0000000000000000 in ?? ()
Thread 7 (Thread 0x7ffff570d700 (LWP 21038)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1 0x0000000000404bd6 in afs_osi_Wait ()
#2 0x0000000000431da2 in afs_Daemon ()
#3 0x0000000000466528 in afs_syscall_call ()
#4 0x0000000000449ef6 in Afs_syscall ()
#5 0x0000000000405c65 in call_syscall ()
---Type <return> to continue, or q <return> to quit---
#6 0x000000000040e79d in afsd_call_syscall ()
#7 0x000000000040ba6c in call_syscall_thread ()
#8 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#9 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#10 0x0000000000000000 in ?? ()
Thread 6 (Thread 0x7ffff7e4f700 (LWP 21037)):
#0 pthread_cond_timedwait@@GLIBC_2.3.2 ()
at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:216
#1 0x000000000041d5ab in afs_rxevent_daemon ()
#2 0x000000000046661b in afs_syscall_call ()
#3 0x0000000000449ef6 in Afs_syscall ()
#4 0x0000000000405c65 in call_syscall ()
#5 0x000000000040e79d in afsd_call_syscall ()
#6 0x000000000040ba6c in call_syscall_thread ()
#7 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#8 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#9 0x0000000000000000 in ?? ()
Thread 5 (Thread 0x7ffff7eee700 (LWP 21036)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1 0x0000000000415cfd in rx_GetCall ()
#2 0x0000000000415f81 in rxi_ServerProc ()
#3 0x000000000041d7ad in rx_ServerProc ()
#4 0x000000000042c033 in afs_RXCallBackServer ()
#5 0x00000000004663a3 in afs_syscall_call ()
#6 0x0000000000449ef6 in Afs_syscall ()
#7 0x0000000000405c65 in call_syscall ()
#8 0x000000000040e79d in afsd_call_syscall ()
#9 0x000000000040ba6c in call_syscall_thread ()
#10 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#11 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#12 0x0000000000000000 in ?? ()
Thread 4 (Thread 0x7ffff7f0c700 (LWP 21035)):
#0 0x00007ffff779f5ad in recvmsg () at ../sysdeps/unix/syscall-template.S:82
#1 0x000000000041dcb3 in rxi_Recvmsg ()
#2 0x00000000004211be in rxi_ReadPacket ()
#3 0x000000000041d690 in rxi_ListenerProc ()
#4 0x000000000041db13 in rxk_Listener ()
#5 0x00000000004664bf in afs_syscall_call ()
#6 0x0000000000449ef6 in Afs_syscall ()
#7 0x0000000000405c65 in call_syscall ()
#8 0x000000000040e79d in afsd_call_syscall ()
---Type <return> to continue, or q <return> to quit---
#9 0x000000000040ba6c in call_syscall_thread ()
#10 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#11 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#12 0x0000000000000000 in ?? ()
Thread 3 (Thread 0x7ffff6179700 (LWP 21034)):
#0 0x00007ffff779f18d in read () at ../sysdeps/unix/syscall-template.S:82
#1 0x00007ffff711fc5d in ?? () from /usr/lib/libfuse.so.2
#2 0x00007ffff71201c6 in ?? () from /usr/lib/libfuse.so.2
#3 0x00007ffff7797b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5 0x0000000000000000 in ?? ()
Thread 2 (Thread 0x7ffff697a700 (LWP 21033)):
#0 0x0000000000000000 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 1 (Thread 0x7ffff7fd4700 (LWP 21028)):
#0 sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:86
#1 0x00007ffff71203f8 in fuse_session_loop_mt () from /usr/lib/libfuse.so.2
#2 0x00007ffff712577d in ?? () from /usr/lib/libfuse.so.2
#3 0x0000000000403cb3 in main (argc=<optimized out>, argv=0x7fffffffe0e8) at ./afsd_fuse.c:578