[OpenAFS-devel] Libtool on AIX
Ben Huntsman
ben@huntsmans.net
Mon, 11 Nov 2024 04:01:04 +0000
--_000_BYAPR07MB5879401CC61C77D7DF8CF5D1A7582BYAPR07MB5879namp_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Hi Ben!
Thank you for replying!
"make dest" seems to work on AIX when you use the --disable-shared optio=
n. I didn't think you could use "make install" with --enable-transarc-path=
s. Regardless, being a traditionalist I'd rather have the dest tree get bu=
ilt on AIX anyway, so I'm interested in fixing it if possible.
Anyway, I look forward to your additional findings when you get a chance=
!
Thank you so much!
-Ben
________________________________
From: Benjamin Kaduk <kaduk@mit.edu>
Sent: Sunday, November 10, 2024 5:32 PM
To: Ben Huntsman <ben@huntsmans.net>
Cc: openafs-devel@openafs.org <openafs-devel@openafs.org>
Subject: Re: [OpenAFS-devel] Libtool on AIX
tl'dr `make dest` is not going to work on AIX at all. You should be able
to get a decent approximation by setting a PREFIX and passing some specific
paths to configure for where various things go and using a `make install`
workflow.
I did not quickly find an explicit place where I documented this but the
commit message for 87ce2a6f05e313dad43311fba93224f33b86f54f is pretty clear
that `make dest` and libtool do not go well together. It kind of works by
accident on most platforms but is doomed on AIX because libtool on AIX
fundamentally requires re-linking at install time, but `make dest` assumes
that you can "install" to a path and then move the resulting binaries
around without breaking things.
I need to do something else right now but might be able to dig up some
references and write more at some point later.
-Ben
On Sun, Nov 10, 2024 at 11:34:49PM +0000, Ben Huntsman wrote:
> Hi there!
> I'm not super familiar with libtool's inner workings, so I was wonderi=
ng if someone could help point me in the right direction. On AIX, if we co=
nfigure with --enable-transarc-paths (though I think this happens even with=
out it), and after the make, run "make dest", the resulting binaries don't =
work.
>
> I'm currently trying this on AIX 7.1. The shared libraries should end=
up in rs_aix71/dest/lib. Let's say I try to run afsd:
>
> ./afsd --help
> exec(): 0509-036 Cannot load program ./afsd because of the following erro=
rs:
> 0509-150 Dependent module libafshcrypto.a(libafshcrypto.so.2) c=
ould not be loaded.
> 0509-022 Cannot load module libafshcrypto.a(libafshcrypto.so.2).
> 0509-026 System error: A file or directory in the path name does =
not exist.
>
> Ok, expected with shared libraries. On AIX I can set my LD_LIBRARY_PATH =
to /wherever/rs_aix71/dest/lib, and try again:
>
> $ ./afsd --help
> exec(): 0509-036 Cannot load program ./afsd because of the following erro=
rs:
> 0509-150 Dependent module /whereever/rs_aix71/dest/lib/libafshc=
rypto.a(libafshcrypto.so.2) could not be loaded.
> 0509-152 Member libafshcrypto.so.2 is not found in archive
>
> Let's see what is in that archive:
>
> $ ar t /wherever/rs_aix71/dest/lib/libafshcrypto.a
> aes.o
> camellia.o
> camellia-ntt.o
> des.o
> engine.o
> evp.o
> evp-hcrypto.o
> evp-cc.o
> hmac.o
> md2.o
> md4.o
> md5.o
> pkcs5.o
> rand-egd.o
> rand-timer.o
> rand-unix.o
> rand.o
> rc2.o
> rc4.o
> rijndael-alg-fst.o
> rnd_keys.o
> sha.o
> sha256.o
> sha512.o
> ui.o
> validate.o
> rand-fortuna.o
>
> Sure enough, that's the static archive. Let's see what exists in the bui=
ld directory:
>
> $ find /wherever -name libafshcrypto.a
> /wherever/lib/libafshcrypto.a
> /wherever/rs_aix71/dest/lib/libafshcrypto.a
> /wherever/src/crypto/hcrypto/.libs/libafshcrypto.a
> /wherever/src/crypto/hcrypto/libafshcrypto.a
>
> Ok, we're interested in the contents of src/crypto/hcrypto/libafshcrypto.=
a and src/crypto/hcrypto/.libs/libafshcrypto.a:
>
> $ ar t /whereever/src/crypto/hcrypto/libafshcrypto.a
> aes.o
> camellia.o
> camellia-ntt.o
> des.o
> engine.o
> evp.o
> evp-hcrypto.o
> evp-cc.o
> hmac.o
> md2.o
> md4.o
> md5.o
> pkcs5.o
> rand-egd.o
> rand-timer.o
> rand-unix.o
> rand.o
> rc2.o
> rc4.o
> rijndael-alg-fst.o
> rnd_keys.o
> sha.o
> sha256.o
> sha512.o
> ui.o
> validate.o
> rand-fortuna.o
>
> ... That's the static archive...
>
> $ ar t /whereever/src/crypto/hcrypto/.libs/libafshcrypto.a
> libafshcrypto.so.2
>
> Aha, that's the shared library archive!
>
> Now, on AIX, you can have an archive file with both static and shared obj=
ects in it. So what we really want here is for the installed libafshcrypto=
.a to have all the .o files, and also the libafshcrypto.so.2 in it.
>
> Let's look at the "dest" make rule in src/crypto/hcrypto/Makefile:
>
> dest: $(SHARED_LIBS) libafshcrypto.a
> ${LT_INSTALL_DATA} libafshcrypto.la ${DEST}/lib/libafshcrypto.la
> ${RM} ${DEST}/lib/libafshcrypto.la
> ${INSTALL_DATA} libafshcrypto.a ${DEST}/lib/libafshcrypto.a
>
> AHA, that looks like the problem! First we install the shared library, t=
hen we overwrite it with the static library! Of course this only breaks on=
AIX because on AIX the names of the shared and the static libraries are th=
e same. I thought that is what the --with-aix=3Dsoname=3Daix configure opt=
ion is supposed to take care of, which is the default. Ideally libtool cou=
ld either build the combined library in src/crypto/hcrypto so that this wor=
ks as-is, or else the Makefile needs to be heavily reworked with some AIX s=
pecifics. This library is just one example out of many, all of them have t=
he same issue on AIX with the "dest" target, as well as the "install" targe=
t.
>
> Another alternative that I might prefer, would be for the default on AIX =
to be to disable shared libraries altogether. This would be more similar t=
o the original IBM AFS. Would that be possible, something everyone could a=
gree on, and anyone know how to do that?
>
> Obviously this is a big problem, as the AIX build doesn't work unless you=
select only one or the other shared/static library options manually as a .=
/configure option.
>
> Thanks in advance for any suggestions!
>
> -Ben
>
--_000_BYAPR07MB5879401CC61C77D7DF8CF5D1A7582BYAPR07MB5879namp_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo=
ttom:0;} </style>
</head>
<body dir=3D"ltr">
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
Hi Ben!</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
Thank you for replying!</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
<br>
</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
"make dest" seems to work on AIX when you use the --=
disable-shared option. I didn't think you could use "make instal=
l" with --enable-transarc-paths. Regardless, being a traditional=
ist I'd rather have the dest tree get built on AIX anyway, so I'm intereste=
d
in fixing it if possible.</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
<br>
</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
Anyway, I look forward to your additional findings when you ge=
t a chance!</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
<br>
</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
Thank you so much!</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
<br>
</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
-Ben</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
<br>
</div>
<div id=3D"appendonsend"></div>
<hr style=3D"display:inline-block;width:98%" tabindex=3D"-1">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st=
yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> Benjamin Kaduk <ka=
duk@mit.edu><br>
<b>Sent:</b> Sunday, November 10, 2024 5:32 PM<br>
<b>To:</b> Ben Huntsman <ben@huntsmans.net><br>
<b>Cc:</b> openafs-devel@openafs.org <openafs-devel@openafs.org><br>
<b>Subject:</b> Re: [OpenAFS-devel] Libtool on AIX</font>
<div> </div>
</div>
<div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;=
">
<div class=3D"PlainText">tl'dr `make dest` is not going to work on AIX at a=
ll. You should be able<br>
to get a decent approximation by setting a PREFIX and passing some specific=
<br>
paths to configure for where various things go and using a `make install`<b=
r>
workflow.<br>
<br>
I did not quickly find an explicit place where I documented this but the<br=
>
commit message for 87ce2a6f05e313dad43311fba93224f33b86f54f is pretty clear=
<br>
that `make dest` and libtool do not go well together. It kind of work=
s by<br>
accident on most platforms but is doomed on AIX because libtool on AIX<br>
fundamentally requires re-linking at install time, but `make dest` assumes<=
br>
that you can "install" to a path and then move the resulting bina=
ries<br>
around without breaking things.<br>
<br>
I need to do something else right now but might be able to dig up some<br>
references and write more at some point later.<br>
<br>
-Ben<br>
<br>
On Sun, Nov 10, 2024 at 11:34:49PM +0000, Ben Huntsman wrote:<br>
> Hi there!<br>
> I'm not super familiar with libtool's inner workings=
, so I was wondering if someone could help point me in the right direction.=
On AIX, if we configure with --enable-transarc-paths (though I think=
this happens even without it), and after the make, run
"make dest", the resulting binaries don't work.<br>
> <br>
> I'm currently trying this on AIX 7.1. The shar=
ed libraries should end up in rs_aix71/dest/lib. Let's say I try to r=
un afsd:<br>
> <br>
> ./afsd --help<br>
> exec(): 0509-036 Cannot load program ./afsd because of the following e=
rrors:<br>
> 0509-150 D=
ependent module libafshcrypto.a(libafshcrypto.so.2) could not be loaded.<br=
>
> 0509-022 Cannot load m=
odule libafshcrypto.a(libafshcrypto.so.2).<br>
> 0509-026 System error:=
A file or directory in the path name does not exist.<br>
> <br>
> Ok, expected with shared libraries. On AIX I can set my LD_LIBRA=
RY_PATH to /wherever/rs_aix71/dest/lib, and try again:<br>
> <br>
> $ ./afsd --help<br>
> exec(): 0509-036 Cannot load program ./afsd because of the following e=
rrors:<br>
> 0509-150 D=
ependent module /whereever/rs_aix71/dest/lib/libafshcrypto.a(libafshcrypto.=
so.2) could not be loaded.<br>
> 0509-152 M=
ember libafshcrypto.so.2 is not found in archive<br>
> <br>
> Let's see what is in that archive:<br>
> <br>
> $ ar t /wherever/rs_aix71/dest/lib/libafshcrypto.a<br>
> aes.o<br>
> camellia.o<br>
> camellia-ntt.o<br>
> des.o<br>
> engine.o<br>
> evp.o<br>
> evp-hcrypto.o<br>
> evp-cc.o<br>
> hmac.o<br>
> md2.o<br>
> md4.o<br>
> md5.o<br>
> pkcs5.o<br>
> rand-egd.o<br>
> rand-timer.o<br>
> rand-unix.o<br>
> rand.o<br>
> rc2.o<br>
> rc4.o<br>
> rijndael-alg-fst.o<br>
> rnd_keys.o<br>
> sha.o<br>
> sha256.o<br>
> sha512.o<br>
> ui.o<br>
> validate.o<br>
> rand-fortuna.o<br>
> <br>
> Sure enough, that's the static archive. Let's see what exists in=
the build directory:<br>
> <br>
> $ find /wherever -name libafshcrypto.a<br>
> /wherever/lib/libafshcrypto.a<br>
> /wherever/rs_aix71/dest/lib/libafshcrypto.a<br>
> /wherever/src/crypto/hcrypto/.libs/libafshcrypto.a<br>
> /wherever/src/crypto/hcrypto/libafshcrypto.a<br>
> <br>
> Ok, we're interested in the contents of src/crypto/hcrypto/libafshcryp=
to.a and src/crypto/hcrypto/.libs/libafshcrypto.a:<br>
> <br>
> $ ar t /whereever/src/crypto/hcrypto/libafshcrypto.a<br>
> aes.o<br>
> camellia.o<br>
> camellia-ntt.o<br>
> des.o<br>
> engine.o<br>
> evp.o<br>
> evp-hcrypto.o<br>
> evp-cc.o<br>
> hmac.o<br>
> md2.o<br>
> md4.o<br>
> md5.o<br>
> pkcs5.o<br>
> rand-egd.o<br>
> rand-timer.o<br>
> rand-unix.o<br>
> rand.o<br>
> rc2.o<br>
> rc4.o<br>
> rijndael-alg-fst.o<br>
> rnd_keys.o<br>
> sha.o<br>
> sha256.o<br>
> sha512.o<br>
> ui.o<br>
> validate.o<br>
> rand-fortuna.o<br>
> <br>
> ... That's the static archive...<br>
> <br>
> $ ar t /whereever/src/crypto/hcrypto/.libs/libafshcrypto.a<br>
> libafshcrypto.so.2<br>
> <br>
> Aha, that's the shared library archive!<br>
> <br>
> Now, on AIX, you can have an archive file with both static and shared =
objects in it. So what we really want here is for the installed libaf=
shcrypto.a to have all the .o files, and also the libafshcrypto.so.2 in it.=
<br>
> <br>
> Let's look at the "dest" make rule in src/crypto/hcrypto/Mak=
efile:<br>
> <br>
> dest: $(SHARED_LIBS) libafshcrypto.a<br>
> ${LT_INSTALL_DATA} lib=
afshcrypto.la ${DEST}/lib/libafshcrypto.la<br>
> ${RM} ${DEST}/lib/liba=
fshcrypto.la<br>
> ${INSTALL_DATA} libafs=
hcrypto.a ${DEST}/lib/libafshcrypto.a<br>
> <br>
> AHA, that looks like the problem! First we install the shared li=
brary, then we overwrite it with the static library! Of course this o=
nly breaks on AIX because on AIX the names of the shared and the static lib=
raries are the same. I thought that is what
the --with-aix=3Dsoname=3Daix configure option is supposed to take care of=
, which is the default. Ideally libtool could either build the combin=
ed library in src/crypto/hcrypto so that this works as-is, or else the Make=
file needs to be heavily reworked with some
AIX specifics. This library is just one example out of many, all of =
them have the same issue on AIX with the "dest" target, as well a=
s the "install" target.<br>
> <br>
> Another alternative that I might prefer, would be for the default on A=
IX to be to disable shared libraries altogether. This would be more s=
imilar to the original IBM AFS. Would that be possible, something eve=
ryone could agree on, and anyone know how to
do that?<br>
> <br>
> Obviously this is a big problem, as the AIX build doesn't work unless =
you select only one or the other shared/static library options manually as =
a ./configure option.<br>
> <br>
> Thanks in advance for any suggestions!<br>
> <br>
> -Ben<br>
> <br>
</div>
</span></font></div>
</body>
</html>
--_000_BYAPR07MB5879401CC61C77D7DF8CF5D1A7582BYAPR07MB5879namp_--