Python build system by roddyrap · Pull Request #88 · guyush1/gdb-static

roddyrap · 2025-12-27T11:44:13Z

Closes #62

roddyrap · 2025-12-27T13:01:57Z

@guyush1 Do you prefere ini or toml for the like 2 configurations we currently expose for the full build?

guyush1 · 2025-12-27T13:07:07Z

@guyush1 Do you prefere ini or toml for the like 2 configurations we currently expose for the full build?

Toml, i liked it in rust :)

guyush1 · 2025-12-28T12:32:51Z

rebase above develop please

guyush1 · 2025-12-28T12:50:18Z

src/compilation/build.py

+            self.CFLAGS.append(flag)
+            self.CXXFLAGS.append(flag)
+
+    def add_flags(self, library: "LibraryMetadata"):


This is a bad name for the function
Maybe something like: add_lib_search_path or something similar.

guyush1 · 2025-12-28T12:52:40Z

src/compilation/build.py

+    source_path: Path
+    commands: List[str]
+
+    # If relative then it's relative to the install path.


at least from the code, i can see the path here is ALWAYS relative to the build dir.

Suggested change

# If relative then it's relative to the install path.

# Relative to the build path

But the documentation should be comprehensive. Currently we only use it relative to the install dir, but someone could put an absolute path there. Things will work.

Changing the comment will make it less precise.

guyush1 · 2025-12-28T12:54:28Z

src/compilation/build.py

+
+    env_setup: Callable[[CompilationEnv, Dict[str, str]], None] = lambda _, __: None
+
+    # If relative then it's relative to the install path.


Same here, code implies its always to the install path

Suggested change

# If relative then it's relative to the install path.

# Relative to install path.

#88 (comment)

guyush1 · 2025-12-28T12:55:58Z

src/compilation/build.py

+    # If relative then it's relative to the install path.
+    result_files: List[Path] = field(default_factory=list)
+
+    build_suffix: str = ""


i don't see why it takes a default "" construction. shouldn't that be an error we would want to discover?

No? Most libraries don't use it. It's fine.
We only use for gdb, to differentiate full and slim builds.

guyush1 · 2025-12-28T12:57:31Z

src/compilation/build.py

+    def get_build_path(self, arch: CompilationArch) -> Path:
+        return self.source_path / f"build-{arch.value}{self.build_suffix}"
+
+    def get_install_path(self, arch: CompilationArch) -> Path:
+        return Path(self.get_build_path(arch) / self.install_path)
+
+    def get_missing_files(self, arch: CompilationArch) -> List[Path]:
+        if len(self.result_files) == 0:
+            return []
+
+        return list(filter(lambda file: not (self.get_install_path(arch) / file).exists(), self.result_files))


would be nice if it was a property instead like you did in the previous class

For build path and install path, yes. For the missing files? No.

They can change, property isn't suitable here.

guyush1 · 2025-12-28T13:34:36Z

src/compilation/build.py

+    libraries = process_libraries(comp_env)
+
+    # Add libraries with dependencies
+    libraries["mpfr"] = LibraryMetadata(
+        comp_env.packages_dir / "mpfr",
+        commands=[
+            f"../configure --enable-static --prefix={{install}} --host={{HOST}} --with-gmp={libraries["gmp"].get_install_path(comp_env.arch)}",
+            "make -j$(nproc)",
+            "make install"
+        ],
+        result_files=[Path("lib/libmpfr.a")]
+    )
+
+    if args.build_variant == BuildVariant.FULL:
+        # An easy way to configure the full build without needing to modify docker arguments or
+        # parts of this file to ahieve the desired result.
+        with open(comp_env.script_dir / "full_build_conf.toml", "rb") as full_build_config_file:
+            build_config_dict = tomllib.load(full_build_config_file)
+
+        is_cross_arch_debugging = build_config_dict["full_build_cross_arch_debugging"]
+        is_building_python = build_config_dict["full_build_python_support"]
+
+        gdb_flags = []
+        if is_cross_arch_debugging:
+            gdb_flags.extend([
+                f"--enable-targets='{args.bfd_arches}'",
+                "--enable-64-bit-bfd",
+                "--disable-sim",
+            ])
+
+        if is_building_python:
+            libraries["libffi"] = LibraryMetadata(
+                comp_env.submodules_dir / "libffi",
+                commands=[
+                    "cd .. && ./autogen.sh",
+                    "CFLAGS='{CFLAGS} -DNO_JAVA_RAW_API' ../configure --enable-silent-rules --enable-static --disable-shared --disable-docs --host={HOST} --prefix={install}",
+                    "make -j$(nproc)",
+                    "make install"
+                ],
+                result_files=[Path("lib/libffi.a")]
+            )
+            libraries["python"] = LibraryMetadata(
+                comp_env.submodules_dir / "cpython-static",
+                env_setup=python_env_setup,
+                # We install Python to the build path because otherwise Python breaks and sets incorrect
+                # paths for libhacl in python-configure.
+                install_path=Path("."),
+                commands=[
+                    """
+                    ../configure \
+                        --prefix={install} \
+                        --disable-test-modules \
+                        --with-ensurepip=no \
+                        --without-decimal-contextvar \
+                        --build=$(gcc -dumpmachine) \
+                        --host={HOST} \
+                        --with-build-python="/usr/bin/{python}" \
+                        --disable-ipv6 \
+                        --disable-shared
+                    """,
+                    "{python} ../Tools/build/freeze_modules.py",
+                    "make regen-frozen",
+                    "make -j$(nproc)",
+                    "make install"
+                ],
+                result_files=[Path("bin/python3-config"), Path(f"lib/lib{comp_env.python}.a")]
+            )
+            gdb_flags.append(
+                f"--with-python={libraries["python"].get_install_path(comp_env.arch) / "bin/python3-config"}",
+            )
+    else:
+        gdb_flags = ["--without-python"]


this whole process of configuring library compilation commands does not belong in the main function.

guyush1 · 2025-12-28T13:34:42Z

src/compilation/build.py

+    if len(GDB_PROGRAM_SUFFIX) > 0:
+        gdb_flags.append("--program-suffix=" + GDB_PROGRAM_SUFFIX)
+
+    libraries["gdb"] = LibraryMetadata(


src/compilation/build.py

guyush1 · 2025-12-28T13:38:05Z

src/compilation/full_build_conf.toml

+full_build_cross_arch_debugging = true
+full_build_python_support = true


Add headers, so instead of full_build_cross_arch_debugging we could have full_build.cross_arch_debugging and full_build.python_support.

guyush1 · 2025-12-28T13:39:10Z

Makefile

 _build-%: symlink-git-packages download-packages build-docker-image
 	mkdir -p build
-	docker run $(TTY_ARG) --user $(shell id -u):$(shell id -g) \
+	docker run -e GDB_PROGRAM_SUFFIX="${GDB_PROGRAM_SUFFIX}" $(TTY_ARG) --user $(shell id -u):$(shell id -g) \


did this variable exist before in some way? what does it do?

Yes - I want to package it for the AUR, but the problem is that if we just build it regularly then static gdb and regular gdb will conflict because the files will be the same.

I want to use this variable to add a suffix to the resulting binaries, removing the conflict.

If you prefer I can add it to the build config file as an optional variable.

If you prefer I can add it to the build config file as an optional variable.

yes please

guyush1 · 2025-12-28T13:40:05Z

fix automation as well please :)

guyush1 · 2026-01-29T10:07:35Z

This MR will go through 2 review cycles:

General code-review.
Extensibility focused code-review, where we focus on making sure that it is still easy to do things like adding new libraries, changing compilers / architectures.

The few differneces are that we directly install more packages (The only package that doesn't have a unique install dir is Python, which installs directly to its build-dir because otherwise module link-flags fail) and that we allow to add a suffix to the gdb files.

roddyrap force-pushed the python-build-system branch 5 times, most recently from 6f310f2 to e34ab0a Compare December 27, 2025 12:14

roddyrap requested a review from guyush1 December 27, 2025 13:05

roddyrap force-pushed the python-build-system branch 2 times, most recently from 31129d0 to 7e5d9a9 Compare December 27, 2025 22:15

guyush1 requested changes Dec 28, 2025

View reviewed changes

guyush1 assigned roddyrap Jan 29, 2026

roddyrap force-pushed the python-build-system branch from 7e5d9a9 to 8431a5c Compare February 27, 2026 14:13

roddyrap force-pushed the python-build-system branch from 8431a5c to 819cb68 Compare February 27, 2026 14:46

	# If relative then it's relative to the install path.
	# Relative to the build path


		env_setup: Callable[[CompilationEnv, Dict[str, str]], None] = lambda _, __: None

		# If relative then it's relative to the install path.

	# If relative then it's relative to the install path.
	# Relative to install path.

		full_build_cross_arch_debugging = true
		full_build_python_support = true

Conversation

roddyrap commented Dec 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

roddyrap commented Dec 27, 2025

Uh oh!

guyush1 commented Dec 27, 2025

Uh oh!

guyush1 commented Dec 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guyush1 commented Dec 28, 2025

Uh oh!

guyush1 commented Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

roddyrap commented Dec 27, 2025 •

edited

Loading