ewoks install: fresh env by default with the pip-venv package manager by woutdenolf · Pull Request #332 · ewoks-kit/ewoks

woutdenolf · 2026-08-10T11:26:43Z

In #65 we changed the graph attributes

"requirements": ["ewokscore==5.1.0", "networkx==3.4.2"]

to

"requirements": {
    "python": {"version": "3.12.11", "implementation": "CPython", "...": "..."},
    "system": {"system": "Linux", "machine": "x86_64", "...": "..."},
    "distributions": [
        {"name": "ewokscore", "version": "5.1.0", "installer": "pip"},
        {"name": "networkx", "version": "3.4.2", "installer": "pip"}
    ],
    "manager": {
        "name": "pip",   # Renamed in this PR to "pip-venv"
        "version": "25.0.1",
        "files": {"requirements.txt": "ewokscore==5.1.0\nnetworkx==3.4.2\n"}
    }
}

The final goal is to support more package managers than pip (uv, poetry, pixi and conda).

This PR is another step:

Refactor the Pip manager to support environment creation and use it by default (rename it to pip-venv).
In-place installation is opt-in.
Remove the non-Pip managers to be handled in follow-up PRs.

See docs on how this PR affects ewoks install and ewoks convert

doc/howtoguides/requirements.rst
doc/tutorials/install.rst
doc/tutorials/install/pip_venv.rst

This was cherry-picked from #330 to make the review easier. Still a large PR, sorry. The other PRs already adds uv, pixi, conda and poetry. This PR already lays the groundwork for those in terms of core, test and doc infrastructure.

FYI python -m ewoks._requirements.__init__ does a pip gathering in your current environment. Might be useful for the reviewing.

codecov · 2026-08-10T11:30:55Z

Codecov Report

❌ Patch coverage is 81.20805% with 112 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/ewoks/_requirements/utils/detect.py	51.11%	22 Missing ⚠️
src/ewoks/_requirements/__init__.py	75.90%	20 Missing ⚠️
src/ewoks/__main__.py	62.79%	16 Missing ⚠️
src/ewoks/_requirements/utils/base_manager.py	84.52%	13 Missing ⚠️
src/ewoks/bindings.py	43.75%	9 Missing ⚠️
src/ewoks/_requirements/utils/files.py	72.00%	7 Missing ⚠️
src/ewoks/_requirements/utils/requirements_txt.py	93.02%	6 Missing ⚠️
src/ewoks/_requirements/utils/process.py	79.16%	5 Missing ⚠️
...quirements/utils/metadata/from_requirements_txt.py	60.00%	4 Missing ⚠️
.../ewoks/_requirements/utils/metadata/last_resort.py	20.00%	4 Missing ⚠️
... and 3 more

📢 Thoughts on this report? Let us know!

payno

Thanks for the detailed documentation.

To be honest, it's hard for to be very critical about the PR. It would require a deeper knowledge of this system and/or package managers. And this would require quiet some time for me. I think Loïc will give a better review, but feel free to merge if needed.

My main concern is trying to reduce long-term maintenance. I'm very worried about the effort needed to handle all the use cases. IMO, forcing "reproducers" to use the same package manager isn't a big requirement, compared to the time/energy we'd need to spend making sure we don't end up with corner cases — and I'd advocate for this, at least.

payno · 2026-08-13T14:16:50Z

+        """Location of the environment with this name, inside a root directory of
+        the user or inside the root directory of this package manager.
+        """
+        return Path(root or self.environments_root()).expanduser() / name


Hard to be very accurate at the moment because I don't know this part of the code yet.

But having this or in Path(root or self.environments_root()).expanduser() / name sounds a bit weird.

Makes me think that the environment location might not be static. From what I get from claude he/she says this will happen when user override the default location. But in this case I would expect environments_root to also be redefined.

This function takes the root (optional coming from the user) or the environment root, expands ~ and return the child name. Not sure what the problem is.

Not really a problem but more a concern. I'm afraid that having root as a parameter allowed from several functions can make things a bit complex.

Can we imagine having root as a parameter of the constructor and letting the 'overwrite' happen here? This would simplify the class IMO.

self.environments_root() might be actually executing commands so not sure we want to do this in the constructor. But passing the root to the constructor should be ok.

Ok even that would make the code very awkward

def _managers() -> Dict[str, BaseManager]: return { name: manager_cls() for name, manager_cls in get_supported_managers().items() }

would become

def _managers(env_root=None) -> Dict[str, BaseManager]: return { name: manager_cls(env_root=env_root) for name, manager_cls in get_supported_managers().items() }

And we would provide an environment root suitable for only one manager to all managers.

payno · 2026-08-13T14:24:25Z

+    managers = _managers()
+    scores = {name: (manager.PRIORITY,) for name, manager in managers.items()}
+    manager = _first_available(managers, scores)
+    if manager is None:
+        raise RuntimeError("No known package manager installed or available")
+
+    return manager


Not simple but should we consider this case ? the original manager not being here ?
To be honest I am a bit afraid that trying to bridge the gap between the different package manager is a pain we might want to avoid (at least to start. Because once we engage ourself this way we must continue).

Requirements are always stored in two ways:

python's distribution info extracted from importlib.metadata

package manager specific (requirements.txt for pip for example)

We already need to fallback to importlib.metadata, especially for pip, since the result of pip freeze is not always installable (e.g. points to local paths).

Since we already need this fallback mechanism even when the producer and installer both use the same package manager, cross-manager support comes for free (using the importlib.metadata fallback).

payno · 2026-08-13T14:34:03Z

+Install any package that provides :term:`tasks <Task>` instead of
+``ewoks`` for a real :term:`workflow`.
+
+Re-producer side


I think there's a corner case worth highlighting.

Right now, if I'm not mistaken, nothing prevents a user from converting a workflow and ending up with a Python environment that fully supports running it — which is very convenient.

However, if that user then shares it with a third party, the installation becomes useless.

However, if that user then shares it with a third party, the installation becomes useless.

Don't get it. Shares what with a third party?

user1 generates a JSON file (ewoks convert) and share it with user2.

user2 receives an JSON file and installs (ewoks install) and executes (ewoks execute).

We are not sharing the environment, we are sharing the workflow (JSON or whatever).

What I meant is that what is shared through the JSON contains a list of requirements.
It comes directly from the user1 python environment.
But not makes sure this environment allows executing the workflow on itself.
User1 could have several environment and not activate the correct one when doing the ewoks convert.

Usually the JSON file comes from execute_graph(..., convert_destination=...) which calls ewoks_convert but also executes the workflow.

Of course I could imaging there are all kinds of reasons the reproducer can still not execution the workflow.

A task input point to a file that does not exist.

A task is implemented to us an environment variable

...

I'm sure there are many. But it is worth mentioning in the docs as you said.

I added a section Limitations and caveats to the "Install and execute a workflow" tutorials.

Co-authored-by: payno <payno@users.noreply.github.com>

woutdenolf linked an issue Aug 10, 2026 that may be closed by this pull request

Ewoks install: support fresh environments #331

Open

woutdenolf marked this pull request as ready for review August 10, 2026 11:42

woutdenolf requested a review from a team August 10, 2026 11:48

payno reviewed Aug 13, 2026

View reviewed changes

woutdenolf requested a review from a team August 16, 2026 07:06

woutdenolf and others added 2 commits August 18, 2026 10:49

ewoks install: fresh env by default with the pip-venv package manager

2cc763e

Apply suggestions from code review

1c35860

Co-authored-by: payno <payno@users.noreply.github.com>

woutdenolf force-pushed the 331-ewoks-install-support-fresh-environments branch from 0b666ee to 1c35860 Compare August 18, 2026 08:49

limitations on ewoks install+execute

192dd89

Uh oh!

Conversation

woutdenolf commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Aug 10, 2026

Codecov Report

Uh oh!

payno left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

woutdenolf Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

woutdenolf Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

woutdenolf Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

woutdenolf commented Aug 10, 2026 •

edited

Loading

woutdenolf Aug 18, 2026 •

edited

Loading

woutdenolf Aug 16, 2026 •

edited

Loading

woutdenolf Aug 16, 2026 •

edited

Loading