[OpenAFS] OpenAFS 1.6.5/1.6.10 - server segfaults during migration to rxkad-k5

Volkmar Glauche volkmar.glauche@uniklinik-freiburg.de
Fri, 07 Nov 2014 17:46:19 +0100


Dear all,

here is an update to the issue described below. The segfault is
reproducible also when I run any of the command line tools with the
-localauth option, i.e. it is not specific to the server code.

I've configured OpenAFS 1.6.5.1 with

./configure --prefix=/usr --build=x86_64-pc-linux-gnu
--host=x86_64-pc-linux-gnu --mandir=/usr/share/man
--infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc
--localstatedir=/var/lib --libdir=/usr/lib64
--disable-dependency-tracking --enable-pam --enable-supergroups
--disable-kernel-module --disable-strip-binaries --enable-debug

and MIT Kerberos with

/var/tmp/portage/app-crypt/mit-krb5-1.12.2/work/krb5-1.12.2/src/configure --prefix=/usr
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
--mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share
--sysconfdir=/etc --localstatedir=/var/lib --libdir=/usr/lib64
--without-ldap --without-tcl --enable-pkinit --enable-thread-support
--without-hesiod --enable-shared --with-system-et --with-system-ss
--enable-dns-for-realm --enable-kdc-lookaside-cache --with-system-verto
--disable-rpath


When I then debug the command

/usr/bin/bos status localhost -localauth

I see:

# gdb /usr/bin/bos
GNU gdb (Gentoo 7.7.1 p1) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
...
Reading symbols from /usr/bin/bos...done.
(gdb) run status localhost -localauth
Starting program: /usr/bin/bos status localhost -localauth
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff793da8c in krb5_is_referral_realm (r=0x8) at
/var/tmp/portage/app-crypt/mit-krb5-1.12.2/work/krb5-1.12.2/src/lib/krb5/krb/princ_comp.c:147
147	    if (r->length==0)

and the backtrace looks like

(gdb) bt
#0  0x00007ffff793da8c in krb5_is_referral_realm (r=0x8) at
/var/tmp/portage/app-crypt/mit-krb5-1.12.2/work/krb5-1.12.2/src/lib/krb5/krb/princ_comp.c:147
#1  0x00007ffff790f003 in krb5_kt_get_entry (context=0x6db220,
keytab=0x6db670, principal=0x0, vno=0, enctype=0, entry=0x7fffffffdaa0)
    at
/var/tmp/portage/app-crypt/mit-krb5-1.12.2/work/krb5-1.12.2/src/lib/krb5/keytab/ktfns.c:62
#2  0x0000000000441f29 in pick_enctype_and_principal
(entry=0x7fffffffdaa0, service_principal=0x7fffffffda68,
enctype=<synthetic pointer>, allowed_enctypes=0x464834 <any_enctype>,
kt=0x6db670, context=0x6db220)
    at ./akimpersonate.c:477
#3  get_credv5_akimpersonate (context=0x6db220,
keytab=keytab@entry=0x6dacf0 "FILE:/etc/openafs/server/rxkad.keytab",
service_principal=service_principal@entry=0x0,
client_principal=0x6db4b0, starttime=starttime@entry=0,
    endtime=endtime@entry=2147483647, allowed_enctypes=0x464834
<any_enctype>, allowed_enctypes@entry=0x0,
out_creds=out_creds@entry=0x7fffffffdb90) at ./akimpersonate.c:728
#4  0x0000000000441574 in K5Auth (enclevel=0 '\000',
aindex=0x7fffffffe034, astr=0x7fffffffe038, adir=0x6d9f80) at
./authcon.c:135
#5  GenericAuth (adir=0x6d9f80, astr=astr@entry=0x7fffffffe038,
aindex=aindex@entry=0x7fffffffe034, enclevel=enclevel@entry=0 '\000',
noauth_fallback=1) at ./authcon.c:189
#6  0x0000000000441c3a in afsconf_PickClientSecObj (dir=<optimized out>,
flags=flags@entry=10, info=info@entry=0x0, cellName=<optimized out>,
sc=sc@entry=0x7fffffffe038, scIndex=scIndex@entry=0x7fffffffe034,
    expires=expires@entry=0x0) at ./authcon.c:461
#7  0x000000000040346d in GetConn (as=0x6bb500,
aencrypt=aencrypt@entry=0) at ./bos.c:214
#8  0x0000000000404df2 in StatServer (as=0x6bb500, arock=<optimized
out>) at ./bos.c:1029
#9  0x000000000041f7b0 in cmd_Dispatch (argc=argc@entry=4,
argv=argv@entry=0x7fffffffe348) at cmd.c:905
#10 0x0000000000407a6f in main (argc=4, argv=0x7fffffffe348) at ./bos.c:2204

Things seem to go wrong in akimpersonate.c/pick_principal(), which does
pick a principal but the picked principal is a NULL pointer.

Maybe there is something unexpected/wrong with my keytabs. Just in case,
here is an overview of what they look like in using both MIT and Heimdal
tools.

Using MIT ktutil, my keytab looks like this:

mit-krb5# ktutil
ktutil:  rkt /etc/openafs/server/rxkad.keytab
ktutil:  l
slot KVNO Principal
---- ----
---------------------------------------------------------------------
   1    0 afs/cell@REALM
   2    0 afs/cell@REALM
   3    0 afs/cell@REALM

The 3 slots correspond to aes256-cts-hmac-sha1-96, des3-cbc-sha1,
arcfour-hmac-md5 encryption of the keys, as heimdals ktutil shows:

heimdal# ktutil -k /tmp/rxkad.keytab list
/tmp/rxkad.keytab:

Vno  Type                     Principal
           Aliases
  0  aes256-cts-hmac-sha1-96  afs/cell@REALM
  0  des3-cbc-sha1            afs/cell@REALM
  0  arcfour-hmac-md5         afs/cell@REALM

As I already said below, things seem to work fine with this rxkad.keytab
and e.g. aes256-cts-hmac-sha1-96 tokens if the keytab is brought in
place after the servers have been started.

Any further help would be very much appreciated.

Best regards,

Volkmar

Am 06.11.2014 um 13:56 schrieb Volkmar Glauche:
> Dear all,
> 
> I have started migrating our AFS cell (OpenAFS 1.6.5) to use rxkad-k5
> following the instructions in
> http://www.openafs.org/pages/security/install-rxkad-k5-1.6.txt and
> http://www.openafs.org/pages/security/how-to-rekey.txt.
> After installing the rxkad.keytab everything seemed to work fine.
> However, when I began restarting the servers, I got reproducible
> segfaults in libkrb5.so.
> 
> Some more details:
> Linux distro - Gentoo, kernel 3.8.13
> OpenAFS - 1.6.5/1.6.10 on servers, 1.6.5 or newer on clients
> Kerberos KDC - Heimdal 1.3.3
> Kerberos on OpenAFS servers and clients - MIT Kerberos 1.12.2 or newer
> 
> I extracted the rxkad.keytab on the Heimdal KDC using Heimdal kadmin -l
> and distributed the file to the OpenAFS servers.
> 
> strace of a starting server process shows that the old KeyFile and the
> rxkad.keytab file are read. The segfault occurs right after closing the
> rxkad.keytab file. I'm not sure whether it is an issue with OpenAFS, MIT
> Kerberos or build options for either software, but maybe someone on this
> list has seen a similar issue?
> 
> Best,
> 
> Volkmar
>