[libjulia] Add v1.6.0

[fingolfin]

No description provided.

[fingolfin]

Regarding the version, perhaps we should imitate what Julia itself does, and use something like 1.6.0-DEV.1390 as version. (I'd not bother to replicate the code for computing the integer '1390' but rather copy it manually from a build of the given git commit).

Sounds OK?

[vchuravy]

You can also use contrib/commit-name.sh to map from a git sha to a julia build name.

[fingolfin]

Sure, but I don't see how that helps, as the build script which has access to this is only run inside the build_tarballs function, which already requires the version as an argument?

[barche]

Trying to build libjulia 1.6 to see if it will fix the macOS problems for libcxxwrap-julia, but it errors differently here than on my own machine. Here the result is:

ERROR: LoadError: Unsatisfiable requirements detected for package SuiteSparse_jll [bea87d4a]:
 SuiteSparse_jll [bea87d4a] log:
 ├─possible versions are: 5.4.0-5.4.1 or uninstalled
 ├─restricted to versions 5.4.0 by an explicit requirement, leaving only versions 5.4.0
 └─restricted by julia compatibility requirements to versions: 5.4.1 or uninstalled — no versions left

On my local attempt I get a 404 error from curl when downloading CompilerSupportLibraries.v0.3.6+1.x86_64-apple-darwin14-libgfortran4.tar.gz.

[fingolfin]

Which macOS problem do you mean? For JuliaLang/julia#38925 there is a pending PR.

This JLL here currently does not really work due to the fake JLL work; I'll update it once there is a chance, but this isn't a priority as the binary for Julia 1.5 ought to work (the issues preventing it from working right now would affect the results of this PR from working, too)

[barche]

Yes, that is the one, I hadn't seen that issue, but concluded from JuliaLang/julia#38829 that a 1.6 JLL was needed. If that won't fix it, this isn't urgent.

[giordano]

It'd be nice to resume this now that v1.6 isn't that far away 🙂

[giordano]

Those are quite some compilation errors....

[vchuravy]

[17:47:03] /opt/i686-linux-gnu/i686-linux-gnu/include/c++/7.1.0/bits/c++14_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2014 standard. This support must be enabled with the -std=c++14 or -std=gnu++14 compiler options.
[17:47:03]  #error This file requires compiler and library support \

[vchuravy]

We need to update:

LVM_CXXFLAGS="-I${prefix}/include -std=c++11 -fno-exceptions -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS"

to be std=c++14 or... try if we can use llvm-config through a HostDependency

[giordano]

Ok, manually setting --std=c++14 makes all Linux pass (but we should try with the host dependency!). Windows fails only on copying openlibm.a. macOS fails to download some dependencies

[vchuravy]

Yeah it's trying to download:

[01:49:19] /workspace/srcdir/julia/deps/tools/jldownload /workspace/srcdir/julia/deps/srccache/CompilerSupportLibraries.v0.3.6+1.x86_64-apple-darwin14-libgfortran4.tar.gz https://github.com/JuliaBinaryWrappers/CompilerSupportLibraries_jll.jl/releases/download/CompilerSupportLibraries-v0.3.6+1/CompilerSupportLibraries.v0.3.6.x86_64-apple-darwin14-libgfortran4.tar.gz

But that has the darwin14 mangling, the uploaded one does not

CompilerSupportLibraries.v0.3.6.x86_64-apple-darwin-libgfortran3.tar.gz

https://github.com/JuliaBinaryWrappers/CompilerSupportLibraries_jll.jl/releases/download/CompilerSupportLibraries-v0.3.6%2B1/CompilerSupportLibraries.v0.3.6.x86_64-apple-darwin-libgfortran3.tar.gz

cc: @staticfloat

[vchuravy]

    # We don't trust the system libm in places
    # So we include a private copy of libopenlibm
    mkdir -p usr/lib
    cp ${prefix}/lib/libopenlibm.a usr/lib/

in recent OpenLIBM_jll this is in bin/ on Windows

[giordano]

Ok, worked around the issue for Windows (but the recipe is getting more and more messy 😁). Any clue where the platform used to download the binaries comes from before I dive into Makefiles? 😛

[giordano]

Ok, I believe I found it, it should be BB_TRIPLET_LIBGFORTRAN_CXXABI at

Yggdrasil/L/libjulia/common.jl

Line 157 in b4a92b2

override BB_TRIPLET_LIBGFORTRAN_CXXABI=${bb_full_target/-julia_version+([^-])}

but after manually removing the 14 from darwin14 (it remains to be seen how to do this in a sane way in the builder....), I get stuck at

make[3]: Entering directory '/workspace/srcdir/julia-1.6.0/src/flisp/host'
Warning: git information unavailable; versioning information limited
_sysctl: applet not found
ld: warning: directory not found for option '-F/System/Library/Frameworks/'
make[3]: *** No rule to make target 'builtins.o', needed by 'flisp'.  Stop.
make[3]: Leaving directory '/workspace/srcdir/julia-1.6.0/src/flisp/host'
make[2]: *** [Makefile:119: host/flisp] Error 2
make[2]: Leaving directory '/workspace/srcdir/julia-1.6.0/src/flisp'
make[1]: *** [Makefile:267: flisp/host/flisp] Error 2
make[1]: Leaving directory '/workspace/srcdir/julia-1.6.0/src'
make: *** [Makefile:43: julia_flisp.boot.inc.phony] Error 2

I think someone more familiar with Julia build system than me should take it from here 🙂 Nevermind, hacking BB_TRIPLET_LIBGFORTRAN_CXXABI was sufficient

[giordano]

All platforms but macOS now work. For reference, the error is

[01:58:42]  x86_64-apple-darwin14-clang++ -mmacosx-version-min=10.9 -m64 -shared  -pipe -fPIC -fno-rtti -pedantic -D_FILE_OFFSET_BITS=64 -I/workspace/destdir/include -DNDEBUG  -O3 -g -D_GNU_SOURCE -I. -I/workspace/srcdir/julia-1.6.0/src -I/workspace/srcdir/julia-1.6.0/src/flisp -I/workspace/srcdir/julia-1.6.0/src/support -I/workspace/destdir/include -I/workspace/srcdir/julia-1.6.0/usr/include -DLIBRARY_EXPORTS -I/workspace/srcdir/julia-1.6.0/deps/valgrind -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fvisibility=hidden -fno-common -Wno-comment -Wpointer-arith -Wundef -DJL_BUILD_ARCH='"x86_64"' -DJL_BUILD_UNAME='"Darwin"' -I -DLLVM_SHLIB "-DJL_SYSTEM_IMAGE_PATH=\"../lib/julia/sys.dylib\"" "-DJL_LIBJULIA_SONAME=\"libjulia.1.dylib\""       "-DJL_LIBJULIA_INTERNAL_SONAME=\"libjulia-internal.1.dylib\"" ./jloptions.o ./runtime_ccall.o ./rtutils.o ./codegen.o ./llvm-ptls.o ./jltypes.o ./gf.o ./typemap.o ./smallintset.o ./ast.o ./builtins.o ./module.o ./interpreter.o ./symbol.o ./dlload.o ./sys.o ./init.o ./task.o ./array.o ./dump.o ./staticdata.o ./toplevel.o ./jl_uv.o ./datatype.o ./simplevector.o ./runtime_intrinsics.o ./precompile.o ./threading.o ./partr.o ./stackwalk.o ./gc.o ./gc-debug.o ./gc-pages.o ./gc-stacks.o ./method.o ./jlapi.o ./signal-handling.o ./safepoint.o ./timing.o ./subtype.o ./crc32c.o ./APInt-C.o ./processor.o ./ircode.o ./jitlayers.o ./aotcompile.o ./debuginfo.o ./disasm.o ./llvm-simdloop.o ./llvm-muladd.o ./llvm-final-gc-lowering.o ./llvm-pass-helpers.o ./llvm-late-gc-lowering.o ./llvm-lower-handlers.o ./llvm-gc-invariant-verifier.o ./llvm-propagate-addrspaces.o ./llvm-multiversioning.o ./llvm-alloc-opt.o ./cgmemmgr.o ./llvm-api.o ./llvm-remove-addrspaces.o ./llvm-remove-ni.o ./llvm-julia-licm.o ./llvm-demote-float16.o -Wl,-rpath,'@loader_path/julia/' -Wl,-rpath,'@loader_path/' -o /workspace/srcdir/julia-1.6.0/usr/lib/libjulia-internal.1.6.dylib -L/workspace/destdir/lib -Wl,-compatibility_version,1 -Wl,-current_version,1.6.0 -L/workspace/srcdir/julia-1.6.0/usr/lib -L/workspace/srcdir/julia-1.6.0/usr/lib -Xlinker -all_load ./flisp/libflisp.a -Xlinker -all_load ./support/libsupport.a -ljulia /workspace/destdir/lib/libuv.a -lutf8proc  -losxunwind -L/workspace/destdir/lib -lLLVM -framework CoreFoundation 
[01:58:42] install_name_tool -id @rpath/libjulia-internal.dylib /workspace/srcdir/julia-1.6.0/usr/lib/libjulia-internal.1.6.dylib
[01:58:42] dsymutil /workspace/srcdir/julia-1.6.0/usr/lib/libjulia-internal.1.6.dylib
[01:58:43] /opt/bin/x86_64-apple-darwin14-libgfortran4-cxx11-julia_version+1.6.0/dsymutil: line 5:  3572 Segmentation fault      "$@"
[01:58:43] make[1]: *** [Makefile:317: /workspace/srcdir/julia-1.6.0/usr/lib/libjulia-internal.1.6.dylib] Error 139
[01:58:43] make[1]: Leaving directory '/workspace/srcdir/julia-1.6.0/src'
[01:58:43] make: *** [Makefile:76: julia-src-release] Error 2

[giordano]

Ok, we can't easily debug the dsymutil segmentation fault, as a workaround.... we can skip this step 😬

@vchuravy @staticfloat could you please review this PR, including my last 3-4 commits? They're quite messy...

[fingolfin]

Is this patch already in Julia master, or at least in a PR there? Perhaps it could alao be backported to the release-1.6 branch?

[giordano]

I'm confused as well: the patch applies cleanly, but the corresponding PR was merged and appears to have been backported to v1.6

[giordano]

Uh, it looks like this commit was skipped from the backport

[barche]

Seems OK to me and I can build libcxxwrap-julia using this, but in the JLL it doesn't add the julia_version tag to the Artifacts.toml anymore, see the last entry here in my test build:

https://github.com/barche/libjulia_jll.jl/blob/5f4ebd90f6137fb2f0527040f053a32ae64365bb/Artifacts.toml

When I tried using this test version as a build dependency for libcxxwrap-julia, it would pick up the Julia 1.5 libjulia instead of the newest one, even though I was requesting Julia 1.6. Not sure if this is a problem that only appears when building locally, or also when building here on Yggdrasil.

[giordano]

When I tried using this test version as a build dependency for libcxxwrap-julia, it would pick up the Julia 1.5 libjulia instead of the newest one, even though I was requesting Julia 1.6. Not sure if this is a problem that only appears when building locally, or also when building here on Yggdrasil.

How did you build? julia build_tarballs.jl <platform>? What was <platform>?

[rgcv]

How did you build? julia build_tarballs.jl ? What was ?

Looking at the latest revision of Artifacts.toml, there is only one target present, that being GNU/Linux x64 (C++11 string abi) x86_64-linux-gnu-cxx11. The generated README file only points out this target as well.

[giordano]

Good point looking at the README, thanks. Yes, you should build for x86_64-linux-gnu-cxx11-julia_version+1.6.0 to specify the Julia version, it's part of the "platform"

[barche]

Correct, I built with only x86_64-linux-gnu-cxx11, so it's not a problem with this PR. Which means that I see no objections to merging this.

[giordano]

Ok, well, let's merge this and see how it goes! Thanks everybody!