InitError: Python

[henry2004y]

Hi,

When I tried to install the package in a fresh Julia 1.11.5 environment,

Installation

(@v1.11) pkg> activate .
(spedas) pkg> add https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git
     Cloning git-repo `https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git`
    Updating git-repo `https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git`
   Resolving package versions...
   Installed micromamba_jll ─ v1.5.8+0
   Installed UnsafePointers ─ v1.0.0
   Installed MicroMamba ───── v0.1.14
   Installed CondaPkg ─────── v0.2.29
   Installed SpaceDataModel ─ v0.1.9
   Installed Pidfile ──────── v1.3.0
   Installed Scratch ──────── v1.3.0
   Installed PythonCall ───── v0.9.25
   Installed pixi_jll ─────── v0.41.3+0
    Updating `D:\Computer\spedas\Project.toml`
  [668d2cef] + PySPEDAS v0.1.0 `https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git#main`
    Updating `D:\Computer\spedas\Manifest.toml`
  [7d9f7c33] + Accessors v0.1.42
  [79e6a3ab] + Adapt v4.3.0
  [4fba245c] + ArrayInterface v7.19.0
  [a33af91c] + CompositionsBase v0.1.2
  [992eb4ea] + CondaPkg v0.2.29
  [187b0558] + ConstructionBase v1.6.0
  [9a962f9c] + DataAPI v1.16.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [0703355e] + DimensionalData v0.29.17
  [411431e0] + Extents v0.1.6
  [85a1e053] + Interfaces v0.3.2
  [8197267c] + IntervalSets v0.7.11
  [3587e190] + InverseFunctions v0.1.17
  [41ab1584] + InvertedIndices v1.3.1
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.7.0
  [0f8b85d8] + JSON3 v1.14.3
  [1914dd2f] + MacroTools v0.5.16
  [0b3b1443] + MicroMamba v0.1.14
  [bac558e1] + OrderedCollections v1.8.1
  [69de0a69] + Parsers v2.8.3
  [fa939f87] + Pidfile v1.3.0
⌅ [aea7be01] + PrecompileTools v1.2.1
  [21216c6a] + Preferences v1.4.3
  [668d2cef] + PySPEDAS v0.1.0 `https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git#main`
  [6099a3de] + PythonCall v0.9.25
  [3cdcf5f2] + RecipesBase v1.3.4
  [ae029012] + Requires v1.3.1
  [6c6a2e73] + Scratch v1.3.0
  [0b37b92c] + SpaceDataModel v0.1.9
  [1e83bf80] + StaticArraysCore v1.4.3
  [10745b16] + Statistics v1.11.1
  [856f2bd8] + StructTypes v1.11.0
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.12.1
  [e17b2a0c] + UnsafePointers v1.0.0
  [f8abcde7] + micromamba_jll v1.5.8+0
  [4d7b5844] + pixi_jll v0.41.3+0
  [0dad84c5] + ArgTools v1.1.2
  [56f22d72] + Artifacts v1.11.0
  [2a0f44e3] + Base64 v1.11.0
  [ade2ca70] + Dates v1.11.0
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching v1.11.0
  [b77e0a4c] + InteractiveUtils v1.11.0
  [4af54fe1] + LazyArtifacts v1.11.0
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2 v1.11.0
  [8f399da3] + Libdl v1.11.0
  [37e2e46d] + LinearAlgebra v1.11.0
  [56ddb016] + Logging v1.11.0
  [d6f4376e] + Markdown v1.11.0
  [a63ad114] + Mmap v1.11.0
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.11.0
  [de0858da] + Printf v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization v1.11.0
  [2f01184e] + SparseArrays v1.11.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test v1.11.0
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] + LibCURL_jll v8.6.0+0
  [e37daf67] + LibGit2_jll v1.7.2+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.6+0
  [14a3606d] + MozillaCACerts_jll v2023.12.12
  [4536629a] + OpenBLAS_jll v0.3.27+1
  [bea87d4a] + SuiteSparse_jll v7.7.0+0
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850b90] + libblastrampoline_jll v5.11.0+0
  [8e850ede] + nghttp2_jll v1.59.0+0
  [3f19e933] + p7zip_jll v17.4.0+2
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling project...
  11 dependencies successfully precompiled in 160 seconds. 74 already precompiled.
  3 dependencies had output during precompilation:
┌ MicroMamba
│   Downloading artifact: micromamba
└
┌ DimensionalData → DimensionalDataPythonCall
│      CondaPkg Found dependencies: C:\Users\hyzho\.julia\packages\PythonCall\L4cjh\CondaPkg.toml
│      CondaPkg Found dependencies: C:\Users\hyzho\.julia\packages\DimensionalData\KlTtO\CondaPkg.toml
│      CondaPkg Found dependencies: C:\Users\hyzho\.julia\packages\PySPEDAS\5fBAi\CondaPkg.toml
│      CondaPkg Resolving changes
│               + netcdf4
│               + numpy
│               + pyspedas (pip)
│               + python
│               + uv
│               + xarray
│      CondaPkg Initialising pixi
│               │ C:\Users\hyzho\.julia\artifacts\aa881b8313f97d7b8797934574a1cfe551f85222\bin\pixi.exe
│               │ init
│               │ --format pixi
│               └ D:\Computer\spedas\.CondaPkg
│  Created D:\Computer\spedas\.CondaPkg\pixi.toml
│      CondaPkg Wrote D:\Computer\spedas\.CondaPkg\pixi.toml
│               │ [dependencies]
│               │ netcdf4 = "*"
│               │ uv = ">=0.4"
│               │ xarray = "*"
│               │ numpy = "*"
│               │
│               │     [dependencies.python]
│               │     channel = "conda-forge"
│               │     build = "*cpython*"
│               │     version = ">=3.8,<4"
│               │
│               │ [project]
│               │ name = ".CondaPkg"
│               │ platforms = ["win-64"]
│               │ channels = ["conda-forge"]
│               │ channel-priority = "strict"
│               │ description = "automatically generated by CondaPkg.jl"
│               │
│               │ [pypi-dependencies.pyspedas]
│               └ git = "https://github.com/spedas/pyspedas"
│      CondaPkg Installing packages
│               │ C:\Users\hyzho\.julia\artifacts\aa881b8313f97d7b8797934574a1cfe551f85222\bin\pixi.exe
│               │ install
│               └ --manifest-path D:\Computer\spedas\.CondaPkg\pixi.toml
│  The default environment has been installed.
└
┌ CondaPkg
│   Downloading artifact: pixi
└

It seems find for add, but not for using:

Import (netCDF4 DLL not found)

julia> using PySPEDAS
ERROR: InitError: Python: ImportError: DLL load failed while importing _netCDF4: The specified procedure could not be found.
Python stacktrace:
 [1] <module>
   @ D:\Computer\spedas\.CondaPkg\.pixi\envs\default\Lib\site-packages\netCDF4\__init__.py:3
 [2] <module>
   @ D:\Computer\spedas\.CondaPkg\.pixi\envs\default\Lib\site-packages\pytplot\importers\netcdf_to_tplot.py:9
 [3] <module>
   @ D:\Computer\spedas\.CondaPkg\.pixi\envs\default\Lib\site-packages\pytplot\__init__.py:157
 [4] <module>
   @ D:\Computer\spedas\.CondaPkg\.pixi\envs\default\Lib\site-packages\pyspedas\analysis\avg_data.py:11
 [5] <module>
   @ D:\Computer\spedas\.CondaPkg\.pixi\envs\default\Lib\site-packages\pyspedas\__init__.py:5
Stacktrace:
  [1] pythrow()
    @ PythonCall.Core C:\Users\hyzho\.julia\packages\PythonCall\L4cjh\src\Core\err.jl:92
  [2] errcheck
    @ C:\Users\hyzho\.julia\packages\PythonCall\L4cjh\src\Core\err.jl:10 [inlined]
  [3] pyimport(m::String)
    @ PythonCall.Core C:\Users\hyzho\.julia\packages\PythonCall\L4cjh\src\Core\builtins.jl:1561
  [4] __init__()
    @ PySPEDAS C:\Users\hyzho\.julia\packages\PySPEDAS\5fBAi\src\PySPEDAS.jl:27
  [5] run_module_init(mod::Module, i::Int64)
    @ Base .\loading.jl:1378
  [6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base .\loading.jl:1366
  [7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{…}, ignore_native::Nothing; register::Bool)
    @ Base .\loading.jl:1254
  [8] _include_from_serialized (repeats 2 times)
    @ .\loading.jl:1210 [inlined]
  [9] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128, stalecheck::Bool; reasons::Dict{…}, DEPOT_PATH::Vector{…})
    @ Base .\loading.jl:2057
 [10] _require(pkg::Base.PkgId, env::String)
    @ Base .\loading.jl:2527
 [11] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base .\loading.jl:2388
 [12] #invoke_in_world#3
    @ .\essentials.jl:1089 [inlined]
 [13] invoke_in_world
    @ .\essentials.jl:1086 [inlined]
 [14] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base .\loading.jl:2375
 [15] macro expansion
    @ .\loading.jl:2314 [inlined]
 [16] macro expansion
    @ .\lock.jl:273 [inlined]
 [17] __require(into::Module, mod::Symbol)
    @ Base .\loading.jl:2271
 [18] #invoke_in_world#3
    @ .\essentials.jl:1089 [inlined]
 [19] invoke_in_world
    @ .\essentials.jl:1086 [inlined]
 [20] require(into::Module, mod::Symbol)
    @ Base .\loading.jl:2260
during initialization of module PySPEDAS
Some type information was truncated. Use `show(err)` to see complete types.

[Beforerr]

Interesting, I could not reproduce this on Mac or Linux. Maybe a very specific Windows problem of conda netCDF4.

Do you have netCDF4 installed somewhere globally?

Right now we use Conda.Pkg to handle netcdf4, something you can try is to let Julia handle netcdf installation by adding NetCDF_jll.jl or NetCDF.jl to Project.yml but I do not have a device to test it.

[Beforerr]

Can you test this one, seems to work on CI windows #4

[henry2004y]

Hmm, #4 still doesn't work on my Win11 machine:

(spedas) pkg> st
Status `D:\Computer\spedas\Project.toml`
  [668d2cef] PySPEDAS v0.1.0 `https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git#push-zltppnxqxtrn`
  [3930f61c] SPEDAS v0.1.9
  [09b745ad] Speasy v0.3.2

julia> using PySPEDAS
ERROR: InitError: Python: ImportError: DLL load failed while importing _netCDF4: The specified procedure could not be found.
Python stacktrace:
 [1] <module>
   @ D:\Computer\spedas\.CondaPkg\.pixi\envs\default\Lib\site-packages\netCDF4\__init__.py:3
 [2] <module>
   @ D:\Computer\spedas\.CondaPkg\.pixi\envs\default\Lib\site-packages\pytplot\importers\netcdf_to_tplot.py:9

This is a bit strange, given that CI has passed. I do see netCDF downloaded in the branch:

(spedas) pkg> add https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git#push-zltppnxqxtrn
    Updating git-repo `https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git`
   Resolving package versions...
   Installed Blosc_jll ── v1.21.7+0
   Installed Lz4_jll ──── v1.10.1+0
   Installed NetCDF_jll ─ v401.900.300+0
   Installed libzip_jll ─ v1.11.3+0
  Downloaded artifact: Lz4
  Downloaded artifact: NetCDF
  Downloaded artifact: libzip
  Downloaded artifact: Blosc
    Updating `D:\Computer\spedas\Project.toml`
  [668d2cef] + PySPEDAS v0.1.0 `https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git#push-zltppnxqxtrn`
    Updating `D:\Computer\spedas\Manifest.toml`
  [3da0fdf6] + MPIPreferences v0.1.11
  [668d2cef] + PySPEDAS v0.1.0 `https://github.com/JuliaSpacePhysics/PySPEDAS.jl.git#push-zltppnxqxtrn`
  [0b7ba130] + Blosc_jll v1.21.7+0
  [0234f1f7] + HDF5_jll v1.14.6+0
  [e33a78d0] + Hwloc_jll v2.12.1+0
  [5ced341a] + Lz4_jll v1.10.1+0
  [7cb0a576] + MPICH_jll v4.3.0+1
  [f1f71cc9] + MPItrampoline_jll v5.5.3+0
  [9237b28f] + MicrosoftMPI_jll v10.1.4+3
  [7243133f] + NetCDF_jll v401.900.300+0

Manually adding NetCDF_jll.jl does not help though. I feel like the error message is complaining about missing netCDF4 from the Python side?

[Beforerr]

Oh maybe try a clean installation? (Delete the .Condapkg folder) Adding netcdf_jll sets some env for the python side.

[henry2004y]

Just confirmed that a fresh installation with the new branch #4 works.