From 5878911f275ca577d8bbbd94b10cadf4ebfbe654 Mon Sep 17 00:00:00 2001 From: Julien Jerphanion Date: Tue, 28 Apr 2026 09:52:50 +0200 Subject: [PATCH 1/3] docs: Add detailed "What is conda-forge?" page Signed-off-by: Julien Jerphanion --- docs/_sidebar.json | 1 + docs/index.md | 1 + docs/user/introduction.md | 1 + docs/user/what-is-conda-forge.md | 106 +++++++++++++++++++++++++++++++ 4 files changed, 109 insertions(+) create mode 100644 docs/user/what-is-conda-forge.md diff --git a/docs/_sidebar.json b/docs/_sidebar.json index d1ea19a62b4..26b131a6c67 100644 --- a/docs/_sidebar.json +++ b/docs/_sidebar.json @@ -9,6 +9,7 @@ }, "items": [ "user/introduction", + "user/what-is-conda-forge", "user/tipsandtricks", "user/ci-skeleton", "user/transitioning_from_defaults", diff --git a/docs/index.md b/docs/index.md index c02f5b26241..ba7f2c9bb52 100644 --- a/docs/index.md +++ b/docs/index.md @@ -12,6 +12,7 @@ title: 'conda-forge documentation' conda-forge is a community effort and a GitHub organization which contains repositories of conda recipes and thus provides conda packages for a wide range of software. The built distributions are uploaded to [anaconda.org/conda-forge](https://anaconda.org/conda-forge) and can be installed with [conda](https://conda.pydata.org/docs/intro.html). +For a deeper explanation of conda-forge's mission, model, and benefits, see [What is conda-forge?](user/what-is-conda-forge.md). **Missing a package that you would love to install with conda?** diff --git a/docs/user/introduction.md b/docs/user/introduction.md index 06c997502a7..9a1b53de03e 100644 --- a/docs/user/introduction.md +++ b/docs/user/introduction.md @@ -17,6 +17,7 @@ conda-forge is a community effort that provides conda packages for a wide range You can [search](https://anaconda.org/) for packages online. Look out for packages provided by our conda-forge organization. **Cannot find a package or only outdated versions of a package?** - Everybody is welcome to contribute to our package stack! Please refer to [Becoming involved](contributing.mdx), for an overview on how to start contributing. +For a more detailed explanation of conda-forge's mission, model, and ecosystem benefits, see [What is conda-forge?](what-is-conda-forge.md). diff --git a/docs/user/what-is-conda-forge.md b/docs/user/what-is-conda-forge.md new file mode 100644 index 00000000000..e5e00eeb9c4 --- /dev/null +++ b/docs/user/what-is-conda-forge.md @@ -0,0 +1,106 @@ +--- +title: "What is conda-forge?" +--- + +# Why conda-forge as a software distribution? + +Conda-forge's mission is to make software easily distributable and usable by the world. + +conda-forge solves problem of distributing scientific software properly and consistently for many applications and platforms. + +conda-forge is much more than `python` and `conda`, and it covers much more than scientific software applications: conda-forge distributes all the dependencies chain for anyone to be able to run their projects. + +## Encompassing ecosystems' across languages + +Projects are using differents languages, and languages' ecosystem do not exist in isolation: Python, C++, R, Fortran, and potentially assets (like model weights) live all together in end-users applications. + +conda-forge distributes all of them so that they can be used consistently altogether in end-users projects. + +:::info Benefits +conda-forge provides plethora of packages from different ecosystems, making sure they can be used all together. +::: + +## Build on an open package format, with no Terms of single entity attached + +conda packages originated from maintainers of Scientific Python projects at Anaconda who wanted to properly solve distributing their projects given their complexity (mixture of Python, C, Fortran, Cython, C++ code; interface with external C and Fortran libraries) compared to normal python-only projects. + +Anaconda came up with a package format which could meet the needs of distributing Python projects with native extensions: the conda package. + +While Anaconda's distribution is linked to specific legal Terms of Services, conda-forge reuse this package format but without any specific legal terms, but the original projects' license. + +:::info Benefits +conda-forge's users aren't technically constrained by the operations of a single entity, and aren't legally constrained more than by the license of projects they use. +::: + +## Open, automated, distributed and transparent maintenance of projects' distribution + +Anaconda distribution was really popular, but also was faced with a tremondous number of requests to add or modify packages' that people decided to create a distributed alternative that could scale: conda-forge. + +Contrarily to other distributions which are only managed by a few number of people without any potential understanding of the process of publishing packages for outsiders, conda-forge adopted a model where each project distribution is managed by a repository on GitHub called feedstock. + +Feedstocks make the recipes of the distribution of package inspectable by anyone. Moreover, people can contribute to the distribution as they would contribute to a repository and also can maintain the distribution, integrating changes from contributors. + +Contributors and maintainers of feedstocks aren't necessarily the ones of the original projects: this empowers anyone to adapt the distribution for new needs. + +Furthermore, a significant part of the maintenance burden (such as version updates, and testing packages) are handled by bots and the CI, easing publishing new builds of packages. + +:::info Benefits +conda-forge makes new versions of packages available promptly, automate the maintenance of the distribution of packages which is shared among people so that bottlenecks are minimized. Anyone can contribute to project's distributions and is welcome to. +::: + +## Managing the entire dependencies tree of packages + +Projects on high-level languages do not depend solely on other high-level languages projects, they also depend (for instance) on C/C++ projects at compile time and at runtime. + +Most notably popular projects in the Scientific Python ecosystem like NumPy, SciPy, scikit-learn and pyarrow depends on C/C++ code (e.g. implementation of BLAS, implementations of OpenMP, `libarrow`, ...), yet some popular distributions and packages do not keep track of those dependencies properly: this is for instance the case of python wheels, causing many problems at runtime (such as duplication of shared objects like NumPy's and SciPy's distributions of an implementation of BLAS, which causes undefined behaviors such as dispatch failure or race conditions in shared thread pools). + +Tracking all the dependencies has the benefits of having the entire chain be managed for the users, and to avoid them trying to fidget with their system's distribution. + +More information about it is provided on the [`pypackaging-native` website](https://pypackaging-native.github.io/). + +:::info Benefits +conda-forge users do not need to perfom ad-hoc potentially hacky modifications of their environment at all to obtain something consistent and usable. +::: + +## Moving the entire ecosystem forward + +Some core dependencies are being used by a large number of packages at compile time. + +When a new version of them is being released, all the packages depending on this core dependencies have to be rebuilt for it to be usable with them in end-users environment. + +For the entire process to be correct, they have to be rebuilt in a particular order whilst the previous version still must be the reference one for the entire ecosystem. This is done by a global pinning system and migration system. + +The global pinning system specifies which version of the main dependencies feedstocks have to use. This guarantees that all packages built at a given time are compatible with one another. + +Migrations rebuild, via bots, packages for new version of project in the direct acyclic order of the updated project dependencies by opening Pull Request on their feedstock, overriding the version in the global pinning for this new one. + +:::info Benefits +conda-forge users do not need to care about managing the release of their projects' new dependencies and the entire ecosystem move forward together. +::: + +## Supporting numerous machine configurations + +Many platforms exist out there in the world, most of them being combinations of: + +- an operating system, notably Linux, macOS, and Windows +- an architecture, most notably `x86_64`, `arm64`, `ppc64` + +conda-forge supports all those combinations, and even microarchitectures instructions for `x86_64` (v1, v2, v3, v4). + +CUDA is also supported, and builds of packages for CUDA are pulled when possible. + +:::info Benefits +conda-forge distributes packages which makes best use of the users' machines' hardware. +::: + +## Unified toolchain across the distribution + +Toolchains (consisting of the compilers, linkers, and implementations of standard libraries) for each targeted platform exist. + +Each of those toolchains and standard library implementations are specific to the targeted platforms. + +conda-forge makes educated and conservative choices for each platform and for the versions of the C standard library used, to make sure that packages can run on a wide breadth of systems, in particular LTS Linux distributions (such as CentOS). + +:::info Benefits +conda-forge use of an uniform toolchan guarantees that almost surely no problem exist at compile time when projects are built, that problems at runtime are minimized, and that a wide range of an operating system versions are covered. +::: From 35ea285ed10d6b6ffb8fb1a6e647faf0e960b2b1 Mon Sep 17 00:00:00 2001 From: Julien Jerphanion Date: Thu, 30 Apr 2026 11:06:22 +0200 Subject: [PATCH 2/3] Mention BSD-3-Clause used on all the artifacts Signed-off-by: Julien Jerphanion Co-authored-by: Isuru Fernando --- docs/user/what-is-conda-forge.md | 36 ++++++++++++++++---------------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/user/what-is-conda-forge.md b/docs/user/what-is-conda-forge.md index e5e00eeb9c4..c52b9e97547 100644 --- a/docs/user/what-is-conda-forge.md +++ b/docs/user/what-is-conda-forge.md @@ -6,73 +6,73 @@ title: "What is conda-forge?" Conda-forge's mission is to make software easily distributable and usable by the world. -conda-forge solves problem of distributing scientific software properly and consistently for many applications and platforms. +conda-forge solves the problem of distributing scientific software properly and consistently for many applications and platforms. -conda-forge is much more than `python` and `conda`, and it covers much more than scientific software applications: conda-forge distributes all the dependencies chain for anyone to be able to run their projects. +conda-forge is much more than `python` and `conda`, and it covers much more than scientific software applications: conda-forge distributes the entire dependency chain for anyone to be able to run their projects. -## Encompassing ecosystems' across languages +## Encompassing ecosystems across languages -Projects are using differents languages, and languages' ecosystem do not exist in isolation: Python, C++, R, Fortran, and potentially assets (like model weights) live all together in end-users applications. +Projects use different languages, and language ecosystems do not exist in isolation: Python, C++, R, Fortran, and potentially assets (like model weights) live all together in end-users applications. conda-forge distributes all of them so that they can be used consistently altogether in end-users projects. :::info Benefits -conda-forge provides plethora of packages from different ecosystems, making sure they can be used all together. +conda-forge provides a plethora of packages from different ecosystems, making sure they can be used all together. ::: -## Build on an open package format, with no Terms of single entity attached +## Built on an open package format, with community-managed artifacts conda packages originated from maintainers of Scientific Python projects at Anaconda who wanted to properly solve distributing their projects given their complexity (mixture of Python, C, Fortran, Cython, C++ code; interface with external C and Fortran libraries) compared to normal python-only projects. Anaconda came up with a package format which could meet the needs of distributing Python projects with native extensions: the conda package. -While Anaconda's distribution is linked to specific legal Terms of Services, conda-forge reuse this package format but without any specific legal terms, but the original projects' license. +While Anaconda's distribution is linked to specific legal Terms of Service, conda-forge reuses this package format in a community-run distribution where artifacts are published under BSD-3-Clause, alongside the original projects' licenses. :::info Benefits -conda-forge's users aren't technically constrained by the operations of a single entity, and aren't legally constrained more than by the license of projects they use. +conda-forge's users aren't technically constrained by the operations of a single entity, and package usage is governed by artifact and upstream project licenses. ::: ## Open, automated, distributed and transparent maintenance of projects' distribution -Anaconda distribution was really popular, but also was faced with a tremondous number of requests to add or modify packages' that people decided to create a distributed alternative that could scale: conda-forge. +Anaconda distribution was really popular, but it was also faced with a tremendous number of requests to add or modify packages, so people decided to create a distributed alternative that could scale: conda-forge. -Contrarily to other distributions which are only managed by a few number of people without any potential understanding of the process of publishing packages for outsiders, conda-forge adopted a model where each project distribution is managed by a repository on GitHub called feedstock. +Contrary to other distributions that are only managed by a small number of people without any potential understanding of the process of publishing packages for outsiders, conda-forge adopted a model where each project distribution is managed by a repository on GitHub called a feedstock. -Feedstocks make the recipes of the distribution of package inspectable by anyone. Moreover, people can contribute to the distribution as they would contribute to a repository and also can maintain the distribution, integrating changes from contributors. +Feedstocks make package distribution recipes inspectable by anyone. Moreover, people can contribute to the distribution as they would contribute to a repository and also can maintain the distribution, integrating changes from contributors. Contributors and maintainers of feedstocks aren't necessarily the ones of the original projects: this empowers anyone to adapt the distribution for new needs. Furthermore, a significant part of the maintenance burden (such as version updates, and testing packages) are handled by bots and the CI, easing publishing new builds of packages. :::info Benefits -conda-forge makes new versions of packages available promptly, automate the maintenance of the distribution of packages which is shared among people so that bottlenecks are minimized. Anyone can contribute to project's distributions and is welcome to. +conda-forge makes new versions of packages available promptly, automates package distribution maintenance shared among people so that bottlenecks are minimized. Anyone can contribute to projects' distributions and is welcome to. ::: ## Managing the entire dependencies tree of packages Projects on high-level languages do not depend solely on other high-level languages projects, they also depend (for instance) on C/C++ projects at compile time and at runtime. -Most notably popular projects in the Scientific Python ecosystem like NumPy, SciPy, scikit-learn and pyarrow depends on C/C++ code (e.g. implementation of BLAS, implementations of OpenMP, `libarrow`, ...), yet some popular distributions and packages do not keep track of those dependencies properly: this is for instance the case of python wheels, causing many problems at runtime (such as duplication of shared objects like NumPy's and SciPy's distributions of an implementation of BLAS, which causes undefined behaviors such as dispatch failure or race conditions in shared thread pools). +Most notably, popular projects in the Scientific Python ecosystem like NumPy, SciPy, scikit-learn, and pyarrow depend on C/C++ code (e.g. implementation of BLAS, implementations of OpenMP, `libarrow`, ...), yet some popular distributions and packages do not keep track of those dependencies properly: this is for instance the case of Python wheels, causing many problems at runtime (such as duplication of shared objects like NumPy's and SciPy's distributions of an implementation of BLAS, which causes undefined behaviors such as dispatch failure or race conditions in shared thread pools). Tracking all the dependencies has the benefits of having the entire chain be managed for the users, and to avoid them trying to fidget with their system's distribution. More information about it is provided on the [`pypackaging-native` website](https://pypackaging-native.github.io/). :::info Benefits -conda-forge users do not need to perfom ad-hoc potentially hacky modifications of their environment at all to obtain something consistent and usable. +conda-forge users do not need to perform ad-hoc potentially hacky modifications of their environment at all to obtain something consistent and usable. ::: ## Moving the entire ecosystem forward Some core dependencies are being used by a large number of packages at compile time. -When a new version of them is being released, all the packages depending on this core dependencies have to be rebuilt for it to be usable with them in end-users environment. +When a new version of them is released, all the packages depending on these core dependencies have to be rebuilt for it to be usable with them in end-users environments. For the entire process to be correct, they have to be rebuilt in a particular order whilst the previous version still must be the reference one for the entire ecosystem. This is done by a global pinning system and migration system. The global pinning system specifies which version of the main dependencies feedstocks have to use. This guarantees that all packages built at a given time are compatible with one another. -Migrations rebuild, via bots, packages for new version of project in the direct acyclic order of the updated project dependencies by opening Pull Request on their feedstock, overriding the version in the global pinning for this new one. +Migrations rebuild, via bots, packages for a new version of a project in the directed acyclic order of the updated project dependencies by opening Pull Requests on their feedstocks, overriding the version in the global pinning for the new one. :::info Benefits conda-forge users do not need to care about managing the release of their projects' new dependencies and the entire ecosystem move forward together. @@ -90,7 +90,7 @@ conda-forge supports all those combinations, and even microarchitectures instruc CUDA is also supported, and builds of packages for CUDA are pulled when possible. :::info Benefits -conda-forge distributes packages which makes best use of the users' machines' hardware. +conda-forge distributes packages that make best use of users' machine hardware. ::: ## Unified toolchain across the distribution @@ -102,5 +102,5 @@ Each of those toolchains and standard library implementations are specific to th conda-forge makes educated and conservative choices for each platform and for the versions of the C standard library used, to make sure that packages can run on a wide breadth of systems, in particular LTS Linux distributions (such as CentOS). :::info Benefits -conda-forge use of an uniform toolchan guarantees that almost surely no problem exist at compile time when projects are built, that problems at runtime are minimized, and that a wide range of an operating system versions are covered. +conda-forge's use of a uniform toolchain guarantees that almost surely no problems exist at compile time when projects are built, that problems at runtime are minimized, and that a wide range of operating system versions are covered. ::: From 844e2d70221e5aae3bfd1fbc3ce8f54e3245c9c3 Mon Sep 17 00:00:00 2001 From: Julien Jerphanion Date: Wed, 3 Jun 2026 12:36:46 +0200 Subject: [PATCH 3/3] Address review comments Signed-off-by: Julien Jerphanion Co-authored-by: H. Vetinari --- docs/user/introduction.md | 2 +- docs/user/what-is-conda-forge.md | 14 ++++++-------- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/docs/user/introduction.md b/docs/user/introduction.md index 9a1b53de03e..05f858bc68b 100644 --- a/docs/user/introduction.md +++ b/docs/user/introduction.md @@ -17,7 +17,7 @@ conda-forge is a community effort that provides conda packages for a wide range You can [search](https://anaconda.org/) for packages online. Look out for packages provided by our conda-forge organization. **Cannot find a package or only outdated versions of a package?** - Everybody is welcome to contribute to our package stack! Please refer to [Becoming involved](contributing.mdx), for an overview on how to start contributing. -For a more detailed explanation of conda-forge's mission, model, and ecosystem benefits, see [What is conda-forge?](what-is-conda-forge.md). +For a more detailed explanation of conda-forge's goals, model, and ecosystem benefits, see [What is conda-forge?](what-is-conda-forge.md). diff --git a/docs/user/what-is-conda-forge.md b/docs/user/what-is-conda-forge.md index c52b9e97547..a5095552671 100644 --- a/docs/user/what-is-conda-forge.md +++ b/docs/user/what-is-conda-forge.md @@ -4,17 +4,15 @@ title: "What is conda-forge?" # Why conda-forge as a software distribution? -Conda-forge's mission is to make software easily distributable and usable by the world. +Conda-forge's goal is to make software easily installable and usable by the world. -conda-forge solves the problem of distributing scientific software properly and consistently for many applications and platforms. +Conda-forge grew out of the intersection between Python and the scientific software ecosystem, not least because it solves many problems in this space that had not been solved elsewhere. Scientific computing is still a core constituency of conda-forge, but over the last 10+ years, the ecosystem has grown to encompass packages from essentially all programming languages, across all major platforms and architectures. See also [key issues in native Python packaging](https://pypackaging-native.github.io/#key-issues). -conda-forge is much more than `python` and `conda`, and it covers much more than scientific software applications: conda-forge distributes the entire dependency chain for anyone to be able to run their projects. +In other words, conda-forge is much more than `python` and `conda`: it distributes the entire dependency chain (including the compilers!) for pretty much anyone to be able to run their projects. ## Encompassing ecosystems across languages -Projects use different languages, and language ecosystems do not exist in isolation: Python, C++, R, Fortran, and potentially assets (like model weights) live all together in end-users applications. - -conda-forge distributes all of them so that they can be used consistently altogether in end-users projects. +Projects use different languages, and language ecosystems do not exist in isolation: Python, C++, R, Fortran, and potentially assets (like model weights) live all together in end-users applications, and conda-forge distributes all of them so that they can be used consistently altogether in end-users projects. :::info Benefits conda-forge provides a plethora of packages from different ecosystems, making sure they can be used all together. @@ -22,9 +20,9 @@ conda-forge provides a plethora of packages from different ecosystems, making su ## Built on an open package format, with community-managed artifacts -conda packages originated from maintainers of Scientific Python projects at Anaconda who wanted to properly solve distributing their projects given their complexity (mixture of Python, C, Fortran, Cython, C++ code; interface with external C and Fortran libraries) compared to normal python-only projects. +The conda package manager and format were created to address packaging challenges for the NumPy stack and other software with native extensions—projects that mix Python with C, Fortran, Cython, C++, and interfaces to external libraries, and that are harder to ship with tools aimed at pure-Python packages alone. -Anaconda came up with a package format which could meet the needs of distributing Python projects with native extensions: the conda package. +At the first PyData meetup, several developers asked Guido van Rossum how to fix Python packaging for the NumPy stack; his answer was to "solve the problem ourselves." Engineers at Continuum (now Anaconda, Inc.) took that seriously: they studied dpkg, rpm, pip/virtualenv, Homebrew, Nix, and other systems, drew on experience with Enthought Python Distribution (EPD), and created the conda package manager and conda environments. A fuller account of this history is in progress in the [conda-forge history article](https://github.com/conda-forge/conda-forge.github.io/pull/2298) (see also [this talk](https://www.youtube.com/watch?v=U2oa_RLbTVA) and [this talk](https://www.youtube.com/watch?v=gFEE3w7F0ww)). While Anaconda's distribution is linked to specific legal Terms of Service, conda-forge reuses this package format in a community-run distribution where artifacts are published under BSD-3-Clause, alongside the original projects' licenses.