Skip to content

Commit c6b6671

Browse files
aignasrickeylev
aignas
and
rickeylev
authored
refactor(pypi): make whl_library a macro and split impl into whl_archive and pip_archive (bazel-contrib#3948)
Before this PR the `whl_library` would be a do-all repository rule. Whilst it is convenient to reuse the code, it is actually really difficult to maintain and make it more performant. Side effect here is that the python dependencies (like `setuptools`, etc) will no longer be downloaded for whl-only extracts, it makes it a tiny bit faster. With this split we can drop certain dependencies from the whl extraction and optimize the common path - whl extraction where the URL for downloading the wheel is known. This also allows us to start handling the sdists in an entirely different way. In a followup PR I plan to split the part which just extracts the wheel to lay a more surgical foundation to bazel-contrib#3856. Foundation work for bazel-contrib#2410. Split out of bazel-contrib#3856. Work towards bazel-contrib#2948. --------- Co-authored-by: Richard Levasseur <richardlev@gmail.com>
1 parent 4f231ac commit c6b6671

3 files changed

Lines changed: 284 additions & 186 deletions

File tree

‎python/private/pypi/attrs.bzl‎

Lines changed: 0 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -64,79 +64,6 @@ here do not cause packages to be re-fetched. Don't fetch different things based
6464
on the value of these variables.
6565
""",
6666
),
67-
"experimental_requirement_cycles": attr.string_list_dict(
68-
default = {},
69-
doc = """\
70-
A mapping of dependency cycle names to a list of requirements which form that cycle.
71-
72-
Requirements which form cycles will be installed together and taken as
73-
dependencies together in order to ensure that the cycle is always satisified.
74-
75-
Example:
76-
`sphinx` depends on `sphinxcontrib-serializinghtml`
77-
When listing both as requirements, ala
78-
79-
```
80-
py_binary(
81-
name = "doctool",
82-
...
83-
deps = [
84-
"@pypi//sphinx:pkg",
85-
"@pypi//sphinxcontrib_serializinghtml",
86-
]
87-
)
88-
```
89-
90-
Will produce a Bazel error such as
91-
92-
```
93-
ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph:
94-
//:doctool (...)
95-
@pypi//sphinxcontrib_serializinghtml:pkg (...)
96-
.-> @pypi_sphinxcontrib_serializinghtml//:pkg (...)
97-
| @pypi_sphinxcontrib_serializinghtml//:_pkg (...)
98-
| @pypi_sphinx//:pkg (...)
99-
| @pypi_sphinx//:_pkg (...)
100-
`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...)
101-
```
102-
103-
Which we can resolve by configuring these two requirements to be installed together as a cycle
104-
105-
```
106-
pip_parse(
107-
...
108-
experimental_requirement_cycles = {
109-
"sphinx": [
110-
"sphinx",
111-
"sphinxcontrib-serializinghtml",
112-
]
113-
},
114-
)
115-
```
116-
117-
Warning:
118-
If a dependency participates in multiple cycles, all of those cycles must be
119-
collapsed down to one. For instance `a <-> b` and `a <-> c` cannot be listed
120-
as two separate cycles.
121-
""",
122-
),
123-
"extra_hub_aliases": attr.string_list_dict(
124-
doc = """\
125-
Extra aliases to make for specific wheels in the hub repo. This is useful when
126-
paired with the {attr}`whl_modifications`.
127-
128-
:::{versionadded} 0.38.0
129-
130-
For `pip.parse` with bzlmod
131-
:::
132-
133-
:::{versionadded} 1.0.0
134-
135-
For `pip_parse` with workspace.
136-
:::
137-
""",
138-
mandatory = False,
139-
),
14067
"extra_pip_args": attr.string_list(
14168
doc = """Extra arguments to pass on to pip. Must not contain spaces.
14269

‎python/private/pypi/pip_repository_attrs.bzl‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,79 @@ repositories."""
2121
load(":attrs.bzl", COMMON_ATTRS = "ATTRS")
2222

2323
ATTRS = {
24+
"experimental_requirement_cycles": attr.string_list_dict(
25+
default = {},
26+
doc = """\
27+
A mapping of dependency cycle names to a list of requirements which form that cycle.
28+
29+
Requirements which form cycles will be installed together and taken as
30+
dependencies together in order to ensure that the cycle is always satisified.
31+
32+
Example:
33+
`sphinx` depends on `sphinxcontrib-serializinghtml`
34+
When listing both as requirements, ala
35+
36+
```
37+
py_binary(
38+
name = "doctool",
39+
...
40+
deps = [
41+
"@pypi//sphinx:pkg",
42+
"@pypi//sphinxcontrib_serializinghtml",
43+
]
44+
)
45+
```
46+
47+
Will produce a Bazel error such as
48+
49+
```
50+
ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph:
51+
//:doctool (...)
52+
@pypi//sphinxcontrib_serializinghtml:pkg (...)
53+
.-> @pypi_sphinxcontrib_serializinghtml//:pkg (...)
54+
| @pypi_sphinxcontrib_serializinghtml//:_pkg (...)
55+
| @pypi_sphinx//:pkg (...)
56+
| @pypi_sphinx//:_pkg (...)
57+
`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...)
58+
```
59+
60+
Which we can resolve by configuring these two requirements to be installed together as a cycle
61+
62+
```
63+
pip_parse(
64+
...
65+
experimental_requirement_cycles = {
66+
"sphinx": [
67+
"sphinx",
68+
"sphinxcontrib-serializinghtml",
69+
]
70+
},
71+
)
72+
```
73+
74+
Warning:
75+
If a dependency participates in multiple cycles, all of those cycles must be
76+
collapsed down to one. For instance `a <-> b` and `a <-> c` cannot be listed
77+
as two separate cycles.
78+
""",
79+
),
80+
"extra_hub_aliases": attr.string_list_dict(
81+
doc = """\
82+
Extra aliases to make for specific wheels in the hub repo. This is useful when
83+
paired with the {attr}`whl_modifications`.
84+
85+
:::{versionadded} 0.38.0
86+
87+
For `pip.parse` with bzlmod
88+
:::
89+
90+
:::{versionadded} 1.0.0
91+
92+
For `pip_parse` with workspace.
93+
:::
94+
""",
95+
mandatory = False,
96+
),
2497
"requirements_by_platform": attr.label_keyed_string_dict(
2598
doc = """\
2699
The requirements files and the comma delimited list of target platforms as values.

0 commit comments

Comments
 (0)