[OpenAFS-devel] AIX build fails with missing symbol .krb5_c_make_random_key

[Benjamin Kaduk]

On Sun, Aug 14, 2022 at 07:41:43PM +0000, Ben Huntsman wrote:
> Hi Ben-
>    Thanks for the reply!
> 
>    My krb5-config and openafs's config.log are attached.
> 
>    When I ran configure, I used the argument --with-krb5=/opt/freeware

Thanks!

It looks like your /opt/freeware/bin/krb5-config is trying to use 64-bit
libraries, but at least configure is trying to build for 32-bit:

configure:40672: checking for krb5-config                                            
configure:40696: found /opt/freeware/bin/krb5-config                                 
configure:40708: result: /opt/freeware/bin/krb5-config                               
configure:41045: checking for krb5 support in krb5-config                            
configure:41058: result: yes                                                         
configure:41066: checking for --deps support in krb5-config                          
configure:41079: result: no                                                          
configure:41108: checking for krb5_init_context                                      
configure:41108: cc -o conftest   -I/opt/freeware/include    conftest.c              
-L/opt/freeware/lib64                                                                
-Wl,-blibpath:/opt/freeware/lib64:/opt/freeware/lib:/usr/lib:/lib                    
-L/opt/freeware/lib64 -L/opt/freeware/lib -Wl,-brtl -lpthreads -lkrb5
-lk5crypto     
-lcom_err  >&5                                                                       
ld: 0711-736 ERROR: Input file /opt/freeware/lib64/libkrb5.so:                       
        XCOFF64 object files are not allowed in 32-bit mode.                         

So if the krb5-config output gets rejected, configure will fall back to
some heuristics for how to link krb5, and those heuristics are wrong in
this case.

I'm not super familiar with the mechanisms to get 32- vs 64-bit output from
the compiler/linker on AIX, but random googling suggests that maybe passing
CFLAGS=-q64 to configure would affect that aspect of things?  (It might
break something else, of course.)

For what it's worth, most of the OS-specific build settings are in
src/cf/osconf.m4, and for rs_aix61 we currently show:

        rs_aix61)       
                CC="cc"
                DBG="-g"
                LIBSYS_AIX_EXP="afsl.exp"
                MT_CC="xlc_r"
                SHLIB_SUFFIX="o"
                XCFLAGS="-K -D_NONSTD_TYPES -D_MBI=void"
                XLIBS="${LIB_AFSDB} ${LIB_libintl} -ldl"
                SHLIB_LINKER="${MT_CC} -bM:SRE -berok"
                AIX32="no"
                AIX64="yes"
                TSM_IMPORTS="-bI:/lib/aio.exp -bI:/lib/netinet.exp -bI:/lib/sockets.exp -bI:/lib/statcmd.exp"
                TSM_LIBS="-lsys -lcsys -lc"
                ;;

The AIX32 and AIX64 variables are not widely used in the buildsystem, but
the one place they do show up suggest that this does indicate that we
"expect to" be building 64-bit libraries on this system.  Perhaps there's
some skew in compiler defaults between when that was written and what you
have.

-Ben