[OpenAFS-devel] Re: afsd-fuse segfaults
Derrick Brashear
shadow@gmail.com
Tue, 18 Sep 2012 10:41:05 -0400
On Tue, Sep 18, 2012 at 10:40 AM, Troy Benjegerdes <hozer@hozed.org> wrote:
> On my server, amd64 debian stable, it works.
> On my laptop, amd64-debian stable + backports + some testing, segaults.
>
> gdb/backtraces are not particularly informative, either (see end of messa=
ge)
I prefer not to take your word for it. Share the backtrace instead?
> Is there a clean cross-platform way to install a segfault handler so we c=
an
> dump this kind of backtraces out automatically, particularly when doing s=
omething
> like a 'make check' smoke test?
>
> ... some references ...
>
> https://wiki.ubuntu.com/Apport
> http://www.cygwin.com/ml/gdb/2007-06/msg00345.html
>
>
>
> On Mon, Sep 17, 2012 at 11:39:55PM -0400, Derrick Brashear wrote:
>> So. Were you perchance using it on a Mac? Probably a 64 bit Intel mac?
>>
>> http://gerrit.openafs.org/#change,8132
>>
>> As nearly as I can tell, this is a very specific problem. The code is fi=
ne. The
>> circumstances of building afsd.fuse meant it was collateral damage when =
we
>> started using roken, but only on MacOS, and probably only for non-32
>> bit pointers,
>> because MacOS does something odd with dirent.h
>>
>> On Mon, Sep 17, 2012 at 1:20 PM, Derrick Brashear <shadow@gmail.com> wro=
te:
>> > On Mon, Sep 17, 2012 at 1:15 PM, Troy Benjegerdes <hozer@hozed.org> wr=
ote:
>> >> I'm looking to get all the low-hanging fruit with unskilled testing.
>> >> Particularly with regressions like this:
>> >>
>> >> hozer@six:~/src/openafs-fuse-git/tests/fuse$ /home/hozer/src/openafs-=
fuse-git/tests/fuse/../../src/afsd/afsd.fuse -dynroot -fakestat -d -confdir=
/home/hozer/src/openafs-fuse-git/tests/fuse/conf -cachedir /home/hozer/src=
/openafs-fuse-git/tests/fuse/vcache -mountdir /home/hozer/src/openafs-fuse-=
git/tests/fuse/mntdir
>> >> FUSE library version: 2.8.6
>> >> nullpath_ok: 0
>> >> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
>> >> INIT: 7.17
>> >> flags=3D0x0000047b
>> >> max_readahead=3D0x00020000
>> >> Starting AFS cache scan...found 0 non-empty cache files (0%).
>> >> afsd: All AFS daemons started.
>> >> Segmentation fault
>
>
> hozer@six:~/src/openafs-fuse-git/tests/fuse$ file ../../src/afsd/afsd.fus=
e
> ../../src/afsd/afsd.fuse: ELF 64-bit LSB executable, x86-64, version 1 (S=
YSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[=
sha1]=3D0xd04485c4228faf5d05f75b9f31b37939d54074d7, not stripped
> hozer@six:~/src/openafs-fuse-git/tests/fuse$ gdb ../../src/afsd/afsd.fuse
> GNU gdb (GDB) 7.4.1-debian
> Copyright (C) 2012 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.h=
tml>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show copying=
"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from /home/hozer/src/openafs-fuse-git/src/afsd/afsd.fuse.=
..done.
> (gdb) run -dynroot -fakestat -d -confdir /home/hozer/src/openafs-fuse-git=
/tests/fuse/conf -cachedir /home/hozer/src/openafs-fuse-git/tests/fuse/vcac=
he -mountdir /home/hozer/src/openafs-fuse-git/tests/fuse/mntdir
> Starting program: /home/hozer/src/openafs-fuse-git/src/afsd/afsd.fuse -dy=
nroot -fakestat -d -confdir /home/hozer/src/openafs-fuse-git/tests/fuse/con=
f -cachedir /home/hozer/src/openafs-fuse-git/tests/fuse/vcache -mountdir /h=
ome/hozer/src/openafs-fuse-git/tests/fuse/mntdir
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1"=
.
> FUSE library version: 2.8.6
> nullpath_ok: 0
> [New Thread 0x7ffff697a700 (LWP 21033)]
> [New Thread 0x7ffff6179700 (LWP 21034)]
> unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
> INIT: 7.17
> flags=3D0x0000047b
> max_readahead=3D0x00020000
> [New Thread 0x7ffff7f0c700 (LWP 21035)]
> [New Thread 0x7ffff7eee700 (LWP 21036)]
> [New Thread 0x7ffff7e4f700 (LWP 21037)]
> [New Thread 0x7ffff570d700 (LWP 21038)]
> [New Thread 0x7ffff56ef700 (LWP 21039)]
> [New Thread 0x7ffff5690700 (LWP 21040)]
> [New Thread 0x7ffff5631700 (LWP 21041)]
> [New Thread 0x7ffff55d2700 (LWP 21042)]
> [New Thread 0x7ffff5573700 (LWP 21043)]
> Starting AFS cache scan...found 0 non-empty cache files (0%).
> afsd: All AFS daemons started.
> [New Thread 0x7ffff54d3700 (LWP 21044)]
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7ffff697a700 (LWP 21033)]
> 0x0000000000000000 in ?? ()
> (gdb) bt
> #0 0x0000000000000000 in ?? ()
> #1 0x0000000000000000 in ?? ()
> (gdb) thread
> [Current thread is 2 (Thread 0x7ffff697a700 (LWP 21033))]
> (gdb) help thread
> Use this command to switch between threads.
> The new thread ID must be currently known.
>
> List of thread subcommands:
>
> thread apply -- Apply a command to a list of threads
> thread find -- Find threads that match a regular expression
> thread name -- Set the current thread's name
>
> Type "help thread" followed by thread subcommand name for full documentat=
ion.
> Type "apropos word" to search for commands related to "word".
> Command name abbreviations are allowed if unambiguous.
> (gdb) thread apply gdb
> (gdb) thread apply bt
> (gdb) thread apply bt all
> (gdb) help thread apply
> Apply a command to a list of threads.
>
> List of thread apply subcommands:
>
> thread apply all -- Apply a command to all threads
>
> Type "help thread apply" followed by thread apply subcommand name for ful=
l documentation.
> Type "apropos word" to search for commands related to "word".
> Command name abbreviations are allowed if unambiguous.
> (gdb) thread apply all bt
>
> Thread 13 (Thread 0x7ffff54d3700 (LWP 21044)):
> #0 __memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:385
> #1 0x000000000040402f in uafs_InitThread ()
> #2 0x0000000000404075 in get_user_struct ()
> #3 0x0000000000405c4d in call_syscall ()
> #4 0x000000000040e79d in afsd_call_syscall ()
> #5 0x000000000040ba6c in call_syscall_thread ()
> #6 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #7 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #8 0x0000000000000000 in ?? ()
>
> Thread 12 (Thread 0x7ffff5573700 (LWP 21043)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/=
x86_64/pthread_cond_wait.S:162
> #1 0x0000000000404e81 in afs_osi_Sleep ()
> #2 0x0000000000432766 in afs_BackgroundDaemon ()
> #3 0x000000000046659f in afs_syscall_call ()
> #4 0x0000000000449ef6 in Afs_syscall ()
> #5 0x0000000000405c65 in call_syscall ()
> #6 0x000000000040e79d in afsd_call_syscall ()
> #7 0x000000000040ba07 in afsd_syscall ()
> #8 0x000000000040ba27 in daemon_thread ()
> #9 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #10 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #11 0x0000000000000000 in ?? ()
>
> Thread 11 (Thread 0x7ffff55d2700 (LWP 21042)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/=
x86_64/pthread_cond_wait.S:162
> #1 0x0000000000404e81 in afs_osi_Sleep ()
> #2 0x0000000000432766 in afs_BackgroundDaemon ()
> #3 0x000000000046659f in afs_syscall_call ()
> #4 0x0000000000449ef6 in Afs_syscall ()
> #5 0x0000000000405c65 in call_syscall ()
> #6 0x000000000040e79d in afsd_call_syscall ()
> #7 0x000000000040ba07 in afsd_syscall ()
> #8 0x000000000040ba27 in daemon_thread ()
> #9 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #10 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #11 0x0000000000000000 in ?? ()
>
> Thread 10 (Thread 0x7ffff5631700 (LWP 21041)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/=
x86_64/pthread_cond_wait.S:162
> #1 0x0000000000404e81 in afs_osi_Sleep ()
> #2 0x0000000000432766 in afs_BackgroundDaemon ()
> ---Type <return> to continue, or q <return> to quit---
> #3 0x000000000046659f in afs_syscall_call ()
> #4 0x0000000000449ef6 in Afs_syscall ()
> #5 0x0000000000405c65 in call_syscall ()
> #6 0x000000000040e79d in afsd_call_syscall ()
> #7 0x000000000040ba07 in afsd_syscall ()
> #8 0x000000000040ba27 in daemon_thread ()
> #9 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #10 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #11 0x0000000000000000 in ?? ()
>
> Thread 9 (Thread 0x7ffff5690700 (LWP 21040)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/=
x86_64/pthread_cond_wait.S:162
> #1 0x0000000000404e81 in afs_osi_Sleep ()
> #2 0x0000000000432766 in afs_BackgroundDaemon ()
> #3 0x000000000046659f in afs_syscall_call ()
> #4 0x0000000000449ef6 in Afs_syscall ()
> #5 0x0000000000405c65 in call_syscall ()
> #6 0x000000000040e79d in afsd_call_syscall ()
> #7 0x000000000040ba07 in afsd_syscall ()
> #8 0x000000000040ba27 in daemon_thread ()
> #9 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #10 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #11 0x0000000000000000 in ?? ()
>
> Thread 8 (Thread 0x7ffff56ef700 (LWP 21039)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/=
x86_64/pthread_cond_wait.S:162
> #1 0x0000000000404bd6 in afs_osi_Wait ()
> #2 0x0000000000431548 in afs_CheckServerDaemon ()
> #3 0x0000000000466547 in afs_syscall_call ()
> #4 0x0000000000449ef6 in Afs_syscall ()
> #5 0x0000000000405c65 in call_syscall ()
> #6 0x000000000040e79d in afsd_call_syscall ()
> #7 0x000000000040ba6c in call_syscall_thread ()
> #8 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #9 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #10 0x0000000000000000 in ?? ()
>
> Thread 7 (Thread 0x7ffff570d700 (LWP 21038)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/=
x86_64/pthread_cond_wait.S:162
> #1 0x0000000000404bd6 in afs_osi_Wait ()
> #2 0x0000000000431da2 in afs_Daemon ()
> #3 0x0000000000466528 in afs_syscall_call ()
> #4 0x0000000000449ef6 in Afs_syscall ()
> #5 0x0000000000405c65 in call_syscall ()
> ---Type <return> to continue, or q <return> to quit---
> #6 0x000000000040e79d in afsd_call_syscall ()
> #7 0x000000000040ba6c in call_syscall_thread ()
> #8 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #9 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #10 0x0000000000000000 in ?? ()
>
> Thread 6 (Thread 0x7ffff7e4f700 (LWP 21037)):
> #0 pthread_cond_timedwait@@GLIBC_2.3.2 ()
> at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:21=
6
> #1 0x000000000041d5ab in afs_rxevent_daemon ()
> #2 0x000000000046661b in afs_syscall_call ()
> #3 0x0000000000449ef6 in Afs_syscall ()
> #4 0x0000000000405c65 in call_syscall ()
> #5 0x000000000040e79d in afsd_call_syscall ()
> #6 0x000000000040ba6c in call_syscall_thread ()
> #7 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #8 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #9 0x0000000000000000 in ?? ()
>
> Thread 5 (Thread 0x7ffff7eee700 (LWP 21036)):
> #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/=
x86_64/pthread_cond_wait.S:162
> #1 0x0000000000415cfd in rx_GetCall ()
> #2 0x0000000000415f81 in rxi_ServerProc ()
> #3 0x000000000041d7ad in rx_ServerProc ()
> #4 0x000000000042c033 in afs_RXCallBackServer ()
> #5 0x00000000004663a3 in afs_syscall_call ()
> #6 0x0000000000449ef6 in Afs_syscall ()
> #7 0x0000000000405c65 in call_syscall ()
> #8 0x000000000040e79d in afsd_call_syscall ()
> #9 0x000000000040ba6c in call_syscall_thread ()
> #10 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #11 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #12 0x0000000000000000 in ?? ()
>
> Thread 4 (Thread 0x7ffff7f0c700 (LWP 21035)):
> #0 0x00007ffff779f5ad in recvmsg () at ../sysdeps/unix/syscall-template.=
S:82
> #1 0x000000000041dcb3 in rxi_Recvmsg ()
> #2 0x00000000004211be in rxi_ReadPacket ()
> #3 0x000000000041d690 in rxi_ListenerProc ()
> #4 0x000000000041db13 in rxk_Listener ()
> #5 0x00000000004664bf in afs_syscall_call ()
> #6 0x0000000000449ef6 in Afs_syscall ()
> #7 0x0000000000405c65 in call_syscall ()
> #8 0x000000000040e79d in afsd_call_syscall ()
> ---Type <return> to continue, or q <return> to quit---
> #9 0x000000000040ba6c in call_syscall_thread ()
> #10 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #11 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #12 0x0000000000000000 in ?? ()
>
> Thread 3 (Thread 0x7ffff6179700 (LWP 21034)):
> #0 0x00007ffff779f18d in read () at ../sysdeps/unix/syscall-template.S:8=
2
> #1 0x00007ffff711fc5d in ?? () from /usr/lib/libfuse.so.2
> #2 0x00007ffff71201c6 in ?? () from /usr/lib/libfuse.so.2
> #3 0x00007ffff7797b50 in start_thread (arg=3D<optimized out>) at pthread=
_create.c:304
> #4 0x00007ffff6a5390d in clone () at ../sysdeps/unix/sysv/linux/x86_64/c=
lone.S:112
> #5 0x0000000000000000 in ?? ()
>
> Thread 2 (Thread 0x7ffff697a700 (LWP 21033)):
> #0 0x0000000000000000 in ?? ()
> #1 0x0000000000000000 in ?? ()
>
> Thread 1 (Thread 0x7ffff7fd4700 (LWP 21028)):
> #0 sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:86
> #1 0x00007ffff71203f8 in fuse_session_loop_mt () from /usr/lib/libfuse.s=
o.2
> #2 0x00007ffff712577d in ?? () from /usr/lib/libfuse.so.2
> #3 0x0000000000403cb3 in main (argc=3D<optimized out>, argv=3D0x7fffffff=
e0e8) at ./afsd_fuse.c:578
>
--=20
Derrick