Skip to content

Commit 28dd020

Browse files
committed
feat(embed): support baking PythonCall into a juliacall system image
When PythonCall is compiled into a juliacall system image, its __init__ runs during jl_init_with_image, before juliacall's bootstrap defines Main.__PythonCall_libptr. Embedding was therefore mis-detected as non-embedded and failed with "'juliacall' module already exists". Add an opt-in embedded preference / JULIA_PYTHONCALL_EMBEDDED (via the same getpref mechanism as exe/lib) that forces the embedded path and obtains libpython from the lib preference / JULIA_PYTHONCALL_LIB (already loaded in the host process). Unset, behaviour is unchanged. Docs and CHANGELOG updated.
1 parent beec1ec commit 28dd020

4 files changed

Lines changed: 54 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Changelog
22

3+
## Unreleased
4+
* Support baking `PythonCall` into a juliacall system image via the new opt-in
5+
`embedded` preference / `JULIA_PYTHONCALL_EMBEDDED` option, removing the
6+
`using PythonCall` cost from cold start. No behaviour change unless opted in.
7+
38
## 0.9.34 (2026-05-18)
49
* Bug fixes.
510

docs/src/juliacall.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,26 @@ systems that may be readonly. Note that the project set in
115115
`PYTHON_JULIACALL_PROJECT` *must* already have PythonCall.jl installed and it
116116
*must* match the JuliaCall version, otherwise loading Julia will fail.
117117

118+
### Baking PythonCall into a system image
119+
120+
For the fastest possible startup you can compile `PythonCall` itself (alongside
121+
your own packages) into a system image with
122+
[PackageCompiler.jl](https://github.com/JuliaLang/PackageCompiler.jl), so that
123+
the `using PythonCall` performed at startup is a memory-map rather than a load.
124+
125+
When `PythonCall` is baked into the system image its `__init__` runs *during*
126+
`jl_init_with_image`, before juliacall's bootstrap has defined the
127+
`Main.__PythonCall_libptr` global it normally uses to detect that it is
128+
embedded. To support this, set the `embedded` preference (or the
129+
`JULIA_PYTHONCALL_EMBEDDED=yes` environment variable) together with the `lib`
130+
preference / `JULIA_PYTHONCALL_LIB` pointing at the running interpreter's
131+
libpython. With `embedded` set, PythonCall takes the embedded path even without
132+
the global and opens libpython by path (it is already loaded in the process, so
133+
this is just a handle). The default is `no`, leaving normal behaviour
134+
unchanged. Use this together with `PYTHON_JULIACALL_SYSIMAGE` (below), and
135+
`PYTHON_JULIACALL_EXE` / `PYTHON_JULIACALL_PROJECT` so juliacall resolves the
136+
baked environment directly.
137+
118138
## [Configuration](@id julia-config)
119139

120140
Some features of the Julia process, such as the optimization level or number of threads, may

src/C/context.jl

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -105,11 +105,36 @@ on_main_thread
105105

106106
function init_context()
107107

108-
CTX.is_embedded = hasproperty(Base.Main, :__PythonCall_libptr)
108+
# Normally PythonCall is embedded when Python (via juliacall) defines the
109+
# global `Main.__PythonCall_libptr`, set by juliacall's bootstrap *after*
110+
# `jl_init_with_image`. If PythonCall is baked into a juliacall system
111+
# image, its `__init__` runs *during* `jl_init_with_image` — before that
112+
# global exists — yet we are still embedded (Python is the running host).
113+
# The opt-in `embedded` preference / `JULIA_PYTHONCALL_EMBEDDED` forces the
114+
# embedded path in that case; libpython is obtained by path since it is
115+
# already loaded in this process. Unset, behaviour is unchanged.
116+
has_libptr = hasproperty(Base.Main, :__PythonCall_libptr)
117+
CTX.is_embedded = has_libptr || Utils.getpref_embedded()
109118

110119
if CTX.is_embedded
111-
# In this case, getting a handle to libpython is easy
112-
CTX.lib_ptr = Base.Main.__PythonCall_libptr::Ptr{Cvoid}
120+
if has_libptr
121+
# In this case, getting a handle to libpython is easy
122+
CTX.lib_ptr = Base.Main.__PythonCall_libptr::Ptr{Cvoid}
123+
else
124+
# Baked into a sysimage: open libpython by path (the `lib`
125+
# preference / JULIA_PYTHONCALL_LIB). dlopen of an
126+
# already-loaded library just returns a handle to it.
127+
lib_path = something(Utils.getpref_lib(), Some(nothing))
128+
lib_path === nothing && error(
129+
"JULIA_PYTHONCALL_EMBEDDED is set but libpython is unknown; " *
130+
"set the `lib` preference or JULIA_PYTHONCALL_LIB to its path.",
131+
)
132+
lib_ptr = dlopen_e(lib_path, CTX.dlopen_flags)
133+
lib_ptr == C_NULL &&
134+
error("Python library $(repr(lib_path)) could not be opened.")
135+
CTX.lib_path = lib_path
136+
CTX.lib_ptr = lib_ptr
137+
end
113138
init_pointers()
114139
# Check Python is initialized
115140
Py_IsInitialized() == 0 && error("Python is not already initialized.")

src/Utils/Utils.jl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ checkpref(::Type{String}, x::AbstractString) = convert(String, x)
1717
getpref_exe() = getpref(String, "exe", "JULIA_PYTHONCALL_EXE", "")
1818
getpref_lib() = getpref(String, "lib", "JULIA_PYTHONCALL_LIB", nothing)
1919
getpref_pickle() = getpref(String, "pickle", "JULIA_PYTHONCALL_PICKLE", "pickle")
20+
getpref_embedded() = getpref(String, "embedded", "JULIA_PYTHONCALL_EMBEDDED", "no") == "yes"
2021

2122
function explode_union(T)
2223
@nospecialize T

0 commit comments

Comments
 (0)