Fixes #462 - Static linking issue with multiple libraries by timlegge · Pull Request #463 · Perl-Toolchain-Gang/ExtUtils-MakeMaker

timlegge · 2024-11-30T02:42:16Z

This may not be the best fix happy to make changes

…braries

Leont · 2024-11-30T03:06:13Z

I'm very confused why you would want to install multiple static libraries? And I'm a bit worried it's completely wrong (but I'm not sure it is).

timlegge · 2024-11-30T03:21:23Z

@Leont that is completely possible. And it is possible that the static build of IO::Compress::Brotli is doing something wrong:

make veryclean; perl Makefile.PL LINKTYPE=static
rm -f \
  blib/script/bro-perl 
rm -f \
  Brotli.bso Brotli.def \
  Brotli.exp Brotli.x \
  Brotli.bs blib/arch/auto/IO/Compress/Brotli/extralibs.all \
  blib/arch/auto/IO/Compress/Brotli/extralibs.ld Makefile.aperl \
  *.a *.o \
  *perl.core Brotli.base \
  Brotli.bs Brotli.bso \
  Brotli.c Brotli.def \
  Brotli.exp Brotli.o \
  Brotli_def.old MYMETA.json \
  MYMETA.yml blibdirs.ts \
  core core.*perl.*.? \
  core.[0-9] core.[0-9][0-9] \
  core.[0-9][0-9][0-9] core.[0-9][0-9][0-9][0-9] \
  core.[0-9][0-9][0-9][0-9][0-9] libBrotli.def \
  mon.out perl \
  perl perl.exe \
  perlmain.c pm_to_blib \
  pm_to_blib.ts so_locations \
  tmon.out 
rm -rf \
  blib brotli/CMakeCache.txt \
  brotli/CMakeFiles/* brotli/CTestTestfile.cmake \
  brotli/DartConfiguration.tcl brotli/Makefile \
  brotli/brotli* brotli/cmake_install.cmake \
  brotli/libbrotlicommon.a brotli/libbrotlicommon.pc \
  brotli/libbrotlidec.a brotli/libbrotlidec.pc \
  brotli/libbrotlienc.a brotli/libbrotlienc.pc 
mv Makefile Makefile.old > /dev/null 2>&1
rm -f \
  Makefile Makefile.old \
  Brotli.o 
rm -rf \
  IO-Compress-Brotli-0.018 
rm -f *~ */*~ *.orig */*.orig *.bak */*.bak *.old */*.old
WARNING: EXTRALIBS is not a known parameter.
Checking if your kit is complete...
Looks good
'EXTRALIBS' is not a known MakeMaker parameter name.
Generating a Unix-style Makefile
Writing Makefile for IO::Compress::Brotli
Writing MYMETA.yml and MYMETA.json
tim@vivobook:~/code/perl-IO-Compress-Brotli$ make test_static
"/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/bin/perl" "/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/ExtUtils/xsubpp"  -typemap '/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/ExtUtils/typemap' -typemap '/home/tim/code/perl-IO-Compress-Brotli/typemap'  Brotli.xs > Brotli.xsc
mv Brotli.xsc Brotli.c
cc -c  -Ibrotli/c/include -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2   -DVERSION=\"0.018\" -DXS_VERSION=\"0.018\" -fPIC "-I/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux/CORE"   Brotli.c
In file included from Brotli.xs:6:
ppport.h:4385: warning: "WIDEST_UTYPE" redefined
 4385 | # define WIDEST_UTYPE U64TYPE
      | 
In file included from /home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux/CORE/perl.h:3327,
                 from Brotli.xs:3:
/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux/CORE/handy.h:1422: note: this is the location of the previous definition
 1422 | #define WIDEST_UTYPE PERL_UINTMAX_T
      | 
cd brotli && "/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/site_perl/5.38.2/x86_64-linux/auto/share/dist/Alien-cmake3/bin/cmake" -DCMAKE_MAKE_PROGRAM=make -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./installed . && make
-- The C compiler identification is GNU 13.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Build type is 'Release'
-- Performing Test BROTLI_EMSCRIPTEN
-- Performing Test BROTLI_EMSCRIPTEN - Failed
-- Compiler is not EMSCRIPTEN
-- Looking for log2
-- Looking for log2 - not found
-- Looking for log2
-- Looking for log2 - found
-- Configuring done (0.3s)
-- Generating done (0.0s)
-- Build files have been written to: /home/tim/code/perl-IO-Compress-Brotli/brotli
make[1]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[2]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
[  2%] Building C object CMakeFiles/brotlicommon.dir/c/common/constants.c.o
[  5%] Building C object CMakeFiles/brotlicommon.dir/c/common/context.c.o
[  8%] Building C object CMakeFiles/brotlicommon.dir/c/common/dictionary.c.o
[ 11%] Building C object CMakeFiles/brotlicommon.dir/c/common/platform.c.o
[ 13%] Building C object CMakeFiles/brotlicommon.dir/c/common/shared_dictionary.c.o
[ 16%] Building C object CMakeFiles/brotlicommon.dir/c/common/transform.c.o
[ 19%] Linking C static library libbrotlicommon.a
make[3]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
[ 19%] Built target brotlicommon
make[3]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
[ 22%] Building C object CMakeFiles/brotlidec.dir/c/dec/bit_reader.c.o
[ 25%] Building C object CMakeFiles/brotlidec.dir/c/dec/decode.c.o
[ 27%] Building C object CMakeFiles/brotlidec.dir/c/dec/huffman.c.o
[ 30%] Building C object CMakeFiles/brotlidec.dir/c/dec/state.c.o
[ 33%] Linking C static library libbrotlidec.a
make[3]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
[ 33%] Built target brotlidec
make[3]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
[ 36%] Building C object CMakeFiles/brotlienc.dir/c/enc/backward_references.c.o
[ 38%] Building C object CMakeFiles/brotlienc.dir/c/enc/backward_references_hq.c.o
[ 41%] Building C object CMakeFiles/brotlienc.dir/c/enc/bit_cost.c.o
[ 44%] Building C object CMakeFiles/brotlienc.dir/c/enc/block_splitter.c.o
[ 47%] Building C object CMakeFiles/brotlienc.dir/c/enc/brotli_bit_stream.c.o
[ 50%] Building C object CMakeFiles/brotlienc.dir/c/enc/cluster.c.o
[ 52%] Building C object CMakeFiles/brotlienc.dir/c/enc/command.c.o
[ 55%] Building C object CMakeFiles/brotlienc.dir/c/enc/compound_dictionary.c.o
[ 58%] Building C object CMakeFiles/brotlienc.dir/c/enc/compress_fragment.c.o
[ 61%] Building C object CMakeFiles/brotlienc.dir/c/enc/compress_fragment_two_pass.c.o
[ 63%] Building C object CMakeFiles/brotlienc.dir/c/enc/dictionary_hash.c.o
[ 66%] Building C object CMakeFiles/brotlienc.dir/c/enc/encode.c.o
[ 69%] Building C object CMakeFiles/brotlienc.dir/c/enc/encoder_dict.c.o
[ 72%] Building C object CMakeFiles/brotlienc.dir/c/enc/entropy_encode.c.o
[ 75%] Building C object CMakeFiles/brotlienc.dir/c/enc/fast_log.c.o
[ 77%] Building C object CMakeFiles/brotlienc.dir/c/enc/histogram.c.o
[ 80%] Building C object CMakeFiles/brotlienc.dir/c/enc/literal_cost.c.o
[ 83%] Building C object CMakeFiles/brotlienc.dir/c/enc/memory.c.o
[ 86%] Building C object CMakeFiles/brotlienc.dir/c/enc/metablock.c.o
[ 88%] Building C object CMakeFiles/brotlienc.dir/c/enc/static_dict.c.o
[ 91%] Building C object CMakeFiles/brotlienc.dir/c/enc/utf8_util.c.o
[ 94%] Linking C static library libbrotlienc.a
make[3]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
[ 94%] Built target brotlienc
make[3]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[3]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
[ 97%] Building C object CMakeFiles/brotli.dir/c/tools/brotli.c.o
[100%] Linking C executable brotli
make[3]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
[100%] Built target brotli
make[2]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
make[1]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli/brotli'
rm -f "blib/arch/auto/IO/Compress/Brotli/Brotli.a"
cp brotli/libbrotlienc.a brotli/libbrotlidec.a brotli/libbrotlicommon.a blib/arch/auto/IO/Compress/Brotli
/usr/bin/ar cr "blib/arch/auto/IO/Compress/Brotli/Brotli.a" Brotli.o
: "blib/arch/auto/IO/Compress/Brotli/Brotli.a"
chmod 755 blib/arch/auto/IO/Compress/Brotli/Brotli.a
cp lib/IO/Compress/Brotli.pm blib/lib/IO/Compress/Brotli.pm
cp lib/IO/Uncompress/Brotli.pm blib/lib/IO/Uncompress/Brotli.pm
cp bin/bro-perl blib/script/bro-perl
"/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/bro-perl
Writing "Makefile.aperl" for this perl
WARNING: EXTRALIBS is not a known parameter.
'EXTRALIBS' is not a known MakeMaker parameter name.
Generating a Unix-style Makefile.aperl
Writing Makefile.aperl for IO::Compress::Brotli
Writing MYMETA.yml and MYMETA.json
make -f Makefile.aperl perl
make[1]: Entering directory '/home/tim/code/perl-IO-Compress-Brotli'
Writing perlmain.c
mv perlmain.ct perlmain.c
cc -c  -Ibrotli/c/include -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2   -DVERSION=\"0.018\" -DXS_VERSION=\"0.018\" -fPIC "-I/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux/CORE"   perlmain.c
cc -c  -Ibrotli/c/include -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2   -DVERSION=\"0.018\" -DXS_VERSION=\"0.018\" -fPIC "-I/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux/CORE"   Brotli.c
In file included from Brotli.xs:6:
ppport.h:4385: warning: "WIDEST_UTYPE" redefined
 4385 | # define WIDEST_UTYPE U64TYPE
      | 
In file included from /home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux/CORE/perl.h:3327,
                 from Brotli.xs:3:
/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux/CORE/handy.h:1422: note: this is the location of the previous definition
 1422 | #define WIDEST_UTYPE PERL_UINTMAX_T
      | 
rm -f "blib/arch/auto/IO/Compress/Brotli/Brotli.a"
cp brotli/libbrotlienc.a brotli/libbrotlidec.a brotli/libbrotlicommon.a blib/arch/auto/IO/Compress/Brotli
/usr/bin/ar cr "blib/arch/auto/IO/Compress/Brotli/Brotli.a" Brotli.o
: "blib/arch/auto/IO/Compress/Brotli/Brotli.a"
chmod 755 blib/arch/auto/IO/Compress/Brotli/Brotli.a
cat blib/arch/auto/IO/Compress/Brotli/extralibs.ld >> blib/arch/auto/IO/Compress/Brotli/extralibs.all
cc -fstack-protector-strong -L/usr/local/lib -Wl,-E Brotli.o -O2 ./perlmain.o -o perl "blib/arch/auto/IO/Compress/Brotli/Brotli.a" "/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux/CORE/libperl.a" `cat blib/arch/auto/IO/Compress/Brotli/extralibs.all` -lpthread -ldl -lm -lcrypt -lutil -lc 
To install the new 'perl' binary, call
    make -f Makefile.aperl inst_perl MAP_TARGET=perl
    make -f Makefile.aperl map_clean
make[1]: Leaving directory '/home/tim/code/perl-IO-Compress-Brotli'
PERL_DL_NONLAZY=1 "/home/tim/code/perl-IO-Compress-Brotli/perl" "-Iblib/arch" "-Iblib/lib" "-I/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2/x86_64-linux" "-I/home/tim/perl5/perlbrew/perls/perl-5.38.2-nothread/lib/5.38.2" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00-load.t ........ ok   
t/01-uncompress.t .. ok       
t/02-roundtrip.t ... ok       
All tests successful.
Files=3, Tests=260,  4 wallclock secs ( 0.03 usr  0.00 sys +  3.83 cusr  0.19 csys =  4.05 CPU)
Result: PASS

Leont · 2024-11-30T18:46:33Z

MakeMaker is assuming here that MYEXTLIB is exactly one library. What it does for static linking is copy that library to the target location, and then add the XS object file to it. This way you have one new static library that includes the XS code. For dynamic linking it just happens to work by accident.

So I think that what you're doing is wrong within the currently existing paradigm. But what you want to do is not reasonable, just complicated.

For dynamic links you want it to link these static libraries into the loadable object (which either MYEXTLIB or LIBS or should do), but static links you want to copy them to the archlib (so that they are found when gathering a new static perl that included this code) AND mark them as "these should be linked in". Otherwise they will be lost again the next time someone links a perl with a new library. This will need the use of EXTRALIBS (a subset of LIBS) instead of MYEXTLIB.

Possibly an Alien:: type approach makes more sense here (but inconveniently the Alien::Brotli namespace seems to be taken already). And also Alien::Base isn't very static perl friendly.

There's a reason few people do this, it's really hard to get it right.

Leont · 2024-11-30T19:34:35Z

Using LIBS instead of MYEXTLIB it almost works right, it builds fine but it stores the wrong path (build dir instead of install dir) so on the next rebuild it won't be able to find the brotli libs. Something Alien-like would definitely be recommended here (that way you'd have a stable library path).

LIBS             => '-Lbrotli -lbrotlienc -lbrotlidec -lbrotlicommon',

timlegge · 2024-11-30T19:39:22Z

@Leont

https://github.com/timlegge/perl-IO-Compress-Brotli/blob/8b44c83b23bb4658179e1494af4b725a1bc476bc/Makefile.PL#L44

I use EXTRALIBS here which I did need in order to get it to work. However, if I remove MYEXTLIB the build fails as you note above I think

timlegge · 2024-11-30T19:43:13Z

However, that does allow me to link to the system install library

Fixes Perl-Toolchain-Gang#462 - Static linking issue with multiple li…

44bb911

…braries

timlegge mentioned this pull request Nov 30, 2024

Static build failed + workaround timlegge/perl-IO-Compress-Brotli#4

Closed

vadimkantorov mentioned this pull request Dec 30, 2024

IO::Compress::Brotli makefile error with static build timlegge/perl-IO-Compress-Brotli#6

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #462 - Static linking issue with multiple libraries#463

Fixes #462 - Static linking issue with multiple libraries#463
timlegge wants to merge 1 commit into
Perl-Toolchain-Gang:masterfrom
timlegge:fix

timlegge commented Nov 30, 2024

Uh oh!

Leont commented Nov 30, 2024

Uh oh!

timlegge commented Nov 30, 2024

Uh oh!

Leont commented Nov 30, 2024

Uh oh!

Leont commented Nov 30, 2024 •

edited

Loading

Uh oh!

timlegge commented Nov 30, 2024

Uh oh!

timlegge commented Nov 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

timlegge commented Nov 30, 2024

Uh oh!

Leont commented Nov 30, 2024

Uh oh!

timlegge commented Nov 30, 2024

Uh oh!

Leont commented Nov 30, 2024

Uh oh!

Leont commented Nov 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

timlegge commented Nov 30, 2024

Uh oh!

timlegge commented Nov 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Leont commented Nov 30, 2024 •

edited

Loading