Update Zig toolchain by mcadariu · Pull Request #105 · xataio/pgzx

mcadariu · 2025-12-11T11:53:20Z

PR to close #104

urso · 2025-12-16T11:41:04Z

Hi, thank you for giving this a try.

The removal of usingnamespace is quite unfortunate and we essentially end up with pg.c.c here. As @cImport will also go away I was thinking to move the header translation to build.zig and finally expose that module via pg.c.

The project also uses nix to provide a stable development environment. See flake.nix. We need to update zig-stable to 0.15.2 and use nix flake update to update zig-overlay and zls dependencies. Otherwise this change will break downstream projects.

mcadariu · 2025-12-16T12:57:17Z

@urso thanks for your feedback, I will try it out.

urso

Nice. The move to TranslateC is a great improvement. There are still a few things we can improve on, though. I still wonder if we can reduce the amount of breaking code.

urso · 2025-12-17T12:14:07Z

 // Export common set of postgres headers.
-pub const c = @import("pgzx_pgsys"); // keep for backwards compatibility
-pub const pg = c;
+pub const pg = @import("pgzx_pgsys");


I was wondering if it is possible to use pg = @import("c_translated") here. the pgzx_pgsys module basically was c.zig that is now replaced by headers.h. The goal of pgzx_pgsys was to have a base-module just doing the C header translation.

urso · 2025-12-18T11:58:45Z

The Zig changes look good, but something seems to be wrong with the nix setup :(

Will take me some time to figure this out, unless you want to give that a go as well.

urso · 2025-12-19T13:10:50Z

    };
    zls = {
-      url = "github:zigtools/zls";
+      url = "github:zigtools/zls/0.15.0";


Oh, I see. ZLS normally follow the development version. This bit me as well a few times in the past. Normally I'm updating the dependency like so:

nix flake update zls --override-input github:zigtools/zls/0.15.0

With ZLS I kinda agree updating the URL here. It is quite clear what to update when upgrading to new zig version. Let's leave it here as is.

I wonder about the pre-commit-hooks-nix dependency though. Why did we need to pin it? A breaking change? For that dependency I would prefer to pin it in the flake.lock file only via --override-input. If there is a breaking change we eventually have to address that in the future.

Edit: good news, CI now succeeds

Without the pin, it throws this error:

error: evaluation aborted with the following error message: 'lib.customisation.callPackageWith: Function called without required argument "zizmor" at /nix/store/r1wrhwj8m29ni4j36dxha1pkzpjdg3jy-source/nix/tools.nix:105'

Should we try to upgrade nixpkgs? (to try to avoid the pinning, but it might be a good idea even for its own sake)

oh, good point. We still use 2405. Current is already 2511. Yeah, I think we should try to update this instead.

urso · 2025-12-22T10:22:13Z

Updating nixpkgs also updates the linters who seem to be more strict 🤦

The linters are correct. In .envrc we should add #!/usr/bin/env bash. This is not really used by direnv, but explicitely document that direnv actually uses a bash sub-shell to load the .envrc file.

The deadnix linter is also correct. in nixpkgs.nix the config at line 2 and the pkgs at line 75 are unused.

You can run the linters locally via: nix develop --command -- pre-commit run --all-files --color always

mcadariu · 2025-12-22T11:13:27Z

Probably was already considered on your side. I was thinking if the following would be a good idea. Set up a pipeline that runs let's say weekly. Downloads the latest Zig and tries to build the project and the example extensions and other such checks. Fails if there's any issues with that.

urso · 2025-12-22T12:16:29Z

Probably was already considered on your side. I was thinking if the following would be a good idea. Set up a pipeline that runs let's say weekly. Downloads the latest Zig and tries to build the project and the example extensions and other such checks. Fails if there's any issues with that.

Yeah, we initially considered that. On the other hand Zig language and standard library is still changing quite a bit between releases. Plus some breaking changes only become apparent when functions are actually used to to Zig 'lazy' nature of compiling code. For this reason we decided to pin to a stable release and even keep to a stable release a little longer in case of critical bugs being discovered. It is still a balance.

The ZLS dependency is also a pain to track as this project normally builds against main. A few times we got caught up in incompatibilities because Zig and ZLS overlays/releases were not well in sync.

I think this level of automation still make sense for patch releases.

urso · 2025-12-22T12:21:35Z

Oh no, looks like postgres headers can't be found after updating nixpkgs:

install
+- install generated to pghostname_zig.so
   +- compile lib pghostname_zig Debug native
      +- translate-c 1 errors
/home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h:1:10: error: 'postgres.h' file not found
#include "postgres.h"
         ^
error: the following command failed with 1 compilation errors:
/nix/store/dbkgn0cfg79318znr71kc4lxjxlz14zr-zig-0.15.2/bin/zig translate-c -lc --cache-dir .zig-cache --global-cache-dir /home/runner/.cache/zig --listen=- -I /home/runner/work/pgzx/pgzx/examples/pghostname_zig/../.././src/pgzx/c/include/ -I /usr/include/postgresql/14/server -I /usr/include/postgresql /home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h
install
+- install generated to spi_sql.so
   +- compile lib spi_sql Debug native
      +- translate-c 1 errors
/home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h:1:10: error: 'postgres.h' file not found
#include "postgres.h"
         ^
error: the following command failed with 1 compilation errors:
/nix/store/dbkgn0cfg79318znr71kc4lxjxlz14zr-zig-0.15.2/bin/zig translate-c -lc --cache-dir .zig-cache --global-cache-dir /home/runner/.cache/zig --listen=- -I /home/runner/work/pgzx/pgzx/examples/spi_sql/../.././src/pgzx/c/include/ -I /usr/include/postgresql/14/server -I /usr/include/postgresql /home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h
install
+- install generated to sqlfns.so
   +- compile lib sqlfns Debug native
      +- translate-c 1 errors
/home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h:1:10: error: 'postgres.h' file not found
#include "postgres.h"
         ^

mcadariu · 2025-12-22T16:22:11Z

I think it's due to this change:

postgresql and libpq don’t provide pg_config by default anymore. Instead, pg_config is available via postgresql.pg_config or libpq.pg_config. This allowed implementing it as a shell script, which can be built for both the build and host systems when cross-compiling. If your build fails to find pg_config, add postgresql.pg_config or libpq.pg_config to nativeBuildInputs.

Source: https://nixos.org/manual/nixpkgs/stable/release-notes?#sec-nixpkgs-release-25.05-incompatibilities

mcadariu · 2025-12-23T16:12:19Z

pglocal now fails with

+ /home/runner/work/pgzx/pgzx/out/default/bin/pg_ctl initdb -D /home/runner/work/pgzx/pgzx/out/default/var/postgres/data -o --encoding=UTF8
initdb: error: file "/home/runner/work/pgzx/pgzx/out/default/share/postgresql/postgres.bki" does not exist

I think it is now confused about where to find files due to the presence of both pkgs.postgresql_16_jit and pkgs.postgresql_16_jit.pg_config

urso · 2025-12-29T10:15:47Z

Good news: tests compile now.
Bad news: Tests seem to crash postgres.

See:

INFO:  Running test: testMemoryContext_reset

server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

…hat it's the culprit

…mand array

urso · 2026-01-05T10:14:15Z

Happy new year.

This is a lot of back and forth creating a commit for every change. While I appreciate the effort I can imagine this being quite frustrating on your side.

If you check the check.yaml github workflow you will find that it is quite easy to run the CI scripts. Enter a development shell via nix develop and then run ./ci/setup.sh once to create a local postgres instance. The postgres setup will be copied into your local development folders and does not mess with your systems postgres installation (well, besides trying to use the same socket). Once CI is setup you run ./ci/run.sh. We wanted to make it easy to run everything local.

In case you don't have nix installed on your dev machine you can do everything via docker as well. User ./dev/docker/build.sh to build the docker image with nix. Then start a the docker based development shell via ./dev/docker/run.sh (the container will be deleted when you quit). Now you can run ./ci/setup.sh or ./ci/run.sh as usual.

I hope this helps a little.

mcadariu · 2026-01-05T11:26:12Z

Happy new year!
I think indeed Docker saves the day.

With Docker I managed to create a local setup and with the latest commit the ./ci/run.sh passes here.

...
Run example CI script ./examples/sqlfns
Build extension sqlfns
Create extension sqlfns
CREATE EXTENSION
Run regression tests: sqlfns
/nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/bin/zig translate-c -lc --cache-dir .zig-cache --global-cache-dir /home/dev/.cache/zig --listen=- -I /home/dev/workdir/examples/sqlfns/../.././src/pgzx/c/include/ -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include /home/dev/workdir/src/pgzx/c/include/headers.h
/nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/bin/zig build-exe -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include -Mroot=/home/dev/workdir/tools/gennodetags/main.zig -lc --cache-dir .zig-cache --global-cache-dir /home/dev/.cache/zig --name gennodetags --zig-lib-dir /nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/lib/ --listen=-
install -C /home/dev/workdir/examples/sqlfns/extension/sqlfns.control /home/dev/workdir/examples/sqlfns/zig-out/share/postgresql/extension/sqlfns.control
install -C /home/dev/workdir/examples/sqlfns/extension/sqlfns--0.1.sql /home/dev/workdir/examples/sqlfns/zig-out/share/postgresql/extension/sqlfns--0.1.sql
CLICOLOR_FORCE=1 pkg-config --list-all
/nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/bin/zig build-lib -ODebug -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server --dep pgzx -Mroot=/home/dev/workdir/examples/sqlfns/src/main.zig -ODebug --dep c_translated --dep gen_node_tags -Mpgzx=/home/dev/workdir/src/pgzx.zig -cflags -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server -- /home/dev/workdir/./src/pgzx/c/libpqsrv.c -lpq -ODebug -I /home/dev/workdir/examples/sqlfns/../.././src/pgzx/c/include/ -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include -L /nix/store/9rynhw9pyiz2jpmhvyrgkh67q7xx49yx-postgresql-16.11-lib/lib -Mc_translated=.zig-cache/o/30fcf3312dbbc667efbd952ab3f6ee08/headers.zig --dep c_translated -Mgen_node_tags=.zig-cache/o/64e397b0ab8da351e868c2cd824e57c5/nodetags.zig -lc -fallow-shlib-undefined --cache-dir .zig-cache --global-cache-dir /home/dev/.cache/zig --name sqlfns -dynamic --version 0.1.0 --zig-lib-dir /nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/lib/ --listen=-
install -C .zig-cache/o/a6c15ec3d6d9e764aef383a120736354/libsqlfns.so.0.1.0 /home/dev/workdir/examples/sqlfns/zig-out/lib/sqlfns.so
# using postmaster on Unix socket, port 5432
ok 1         - sqlfns_test                                16 ms
1..1g_regress
# All 1 tests passed.
Drop extension sqlfns
DROP EXTENSION
Success!
Stopping PostgreSQL cluster ''
waiting for server to shut down.... done
server stopped

urso · 2026-01-09T11:08:13Z

With Docker I managed to create a local setup and with the latest commit the ./ci/run.sh passes here.

That's great. But we still have some tests commented out. Can you reenable and see if they run through?

mcadariu · 2026-01-09T12:31:45Z

Good catch, if I uncomment the pgzx.mem.TestSuite_Mem it fails with the following output. I can investigate and work on it locally. At least we have narrowed it down.

mcadariu · 2026-01-10T14:43:53Z

Still passes here.. one option would be to revert to commit 8972e51 which was passing the CI but had something pinned.

mcadariu · 2026-01-22T20:09:39Z

@urso shall I revert this PR to commit 8972e51 (which was passing the CI pipeline)? I can not work actively anymore however was wondering if that would be an acceptable way to wrap up this PR.

urso · 2026-01-23T09:00:23Z

Hi,
Sorry for the late reply, I've been a little busy the last weeks.
I would say it is better to keep in all the commits. I will pick up this PR when I have time and see if I can reproduce and fix the last issue. Having all history available to do so will be helpful I think.

Thank you for all the effort you've put into it. I understand it was quite time consuming and maybe a little frustrating at times. The more I appreciate the dedication and time you spend on this PR. I'm sure we're this close to get it merged.

mcadariu · 2026-01-23T15:18:00Z

Hi,
Great, thanks! No rush, it was just a curiosity of mine to try Zig a bit with PG extension.
Just wanted to "informally" handover so that I can "close the loop" in my head and not leave the PR hanging after I created it.

jhf · 2026-02-01T17:23:52Z

HTab Test Crash - Root Cause Found

Hi @mcadariu, I've been investigating the CI failure in this PR and found the root cause of the server crash during unit tests.

The Problem

The crash occurs because Zig's MachO linker uses two-level namespace by default, which binds hash_create, hash_destroy, and hash_search to the wrong library.

On macOS (and apparently Linux CI too), the system's libc exports BSD hsearch functions with the same names but different signatures than PostgreSQL's internal functions:

BSD: HTAB *hash_create(int nel, HASHCTL *hctl, int flags)
PostgreSQL: HTAB *hash_create(const char *tabname, long nelem, const HASHCTL *info, int flags)

When the extension calls hash_create(), it actually calls the BSD version, which returns garbage, causing the crash.

Evidence

$ nm -m extension.dylib | grep hash_create
                 (undefined) external _hash_create (from libSystem)

It should show (from executable) or just (undefined) with flat namespace.

No Zig Workaround

I tested extensively and Zig's linker doesn't support -flat_namespace or -bundle linking:

zig cc -Xlinker -flat_namespace ...  # Error: unsupported linker arg

The only fix is building with clang: clang -bundle -bundle_loader $(pg_config --bindir)/postgres

Solution

Disable HTab tests until Zig adds support for flat namespace linking. I've documented this and created a draft Zig issue.

I'm working on a separate branch with:

Your Zig 0.15.2 API fixes (will adapt from your PR)
Nix fixes (nixpkgs 24.11, PG18 support)
HTab tests disabled with proper documentation
Matrix testing infrastructure

Happy to collaborate on getting this merged. Would you like to coordinate?

@mcadariu

Replace the deprecated @cImport approach with build-time translate-c for C header translation. This aligns with Zig's direction of removing @cImport in future versions. Changes: - Add src/pgzx/c/include/headers.h with all PostgreSQL includes - Update build.zig to use addTranslateC() instead of @cImport - Rename module from 'pgzx_pgsys' to 'c_translated' - Update all internal imports to use the new module name - Keep test_filter support for selective test running This change is based on PR xataio#105 by @mcadariu, adapted to work with our existing infrastructure and test framework. Co-authored-by: Marius Cadariu <mcadariu@users.noreply.github.com>

urso · 2026-02-02T09:30:46Z

Disable HTab tests until Zig adds support for flat namespace linking. I've documented this and created a draft Zig issue.

I have to disagree. We should not disable the tests. Due to the zig linker issue it is not safe to use pgzx on macOS in general. Postgres fortunately builds with --bundle to overcome the issue.

The issue is known and has reported more than once to the Zig team. For example:

This is why for the local tests we use Docker. We want to run the tests in a linux environment. The tests did fail on our linux runners, not macOS (although it will fail on macOS for sure).

If you want to work on macOS, a good compromise would be to use a comp-time conditional to not include the test for darwin targets.

jhf · 2026-02-03T13:05:30Z

Disable HTab tests until Zig adds support for flat namespace linking. I've documented this and created a draft Zig issue.

I have to disagree. We should not disable the tests. Due to the zig linker issue it is not safe to use pgzx on macOS in general. Postgres fortunately builds with --bundle to overcome the issue.

Yes, disabling the test was to verify that everything else works.
I currently have a workaround that uses clang in build.zig - that makes all the tests pass on macOs,
as I have an interest in making that work since I use it for development and quick iteration.
I hope to open a separate PR with my work soon, for review and feedback.

jhf · 2026-02-03T23:18:04Z

I did push my work in #106 trying to combine your work mcadariu and my ideas.
I did also use AI tools to create https://codeberg.org/veridit/zig/src/branch/feature/macho-bundle-loader that adds the requires support to Zig.
However, Zig has a strict no LLM - human only policy - that I failed to research before doing that work, so it will never be merged.
As such, I will close my #106 - as I can not hope for a Zig release with -bundle_loader support, but leave it online in case it serves a purpose for you.

mcadariu added 2 commits December 11, 2025 12:55

fix breaking changes

8e19d18

cleanup

8843195

apply review comments

64bcf11

urso reviewed Dec 17, 2025

View reviewed changes

remove c.zig

8656618

mcadariu added 2 commits December 18, 2025 16:24

pin pre-commit-hooks.nix

4063394

apply Zig formatter and pin zls

8972e51

urso reviewed Dec 19, 2025

View reviewed changes

update nixpkgs to 2511 and remove pin on pre-commit-hooks.nix

904debe

fix lint errors

588e1d3

mcadariu added 2 commits December 22, 2025 13:43

fix postgres packages

a13b114

cleanup path export

61beaaa

add pkgs.postgresql_16_jit.pg_config

29593ec

mcadariu added 3 commits December 23, 2025 16:45

create unified postgres package

797f945

bring back pglocal changes

b175ef2

fix bitcode directory copying issue

933a06b

mcadariu added 4 commits December 29, 2025 17:49

temporarily disable node tests in order to determine with certainty t…

c6ad224

…hat it's the culprit

New experiment: disable memory tests temporarily

77f961b

fix lint issue

3251787

more logging around process spawning

8cd6106

use {any} which works on any type instead of {s} for printing the com…

1921d8f

…mand array

fix pg_regress path

1dc82c4

ensure fresh data directory

6be6f20

jhf mentioned this pull request Feb 1, 2026

Zig 0.15.2 upgrade with translate-c and nix fixes #106

Closed

Conversation

mcadariu commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

urso commented Dec 16, 2025

Uh oh!

mcadariu commented Dec 16, 2025

Uh oh!

urso left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

urso Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

mcadariu Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

urso commented Dec 18, 2025

Uh oh!

urso Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcadariu Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

mcadariu Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

urso Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

urso commented Dec 22, 2025

Uh oh!

mcadariu commented Dec 22, 2025

Uh oh!

urso commented Dec 22, 2025

Uh oh!

urso commented Dec 22, 2025

Uh oh!

mcadariu commented Dec 22, 2025

Uh oh!

mcadariu commented Dec 23, 2025

Uh oh!

urso commented Dec 29, 2025

Uh oh!

urso commented Jan 5, 2026

Uh oh!

mcadariu commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

urso commented Jan 9, 2026

Uh oh!

mcadariu commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcadariu commented Jan 10, 2026

Uh oh!

mcadariu commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

urso commented Jan 23, 2026

Uh oh!

mcadariu commented Jan 23, 2026

Uh oh!

jhf commented Feb 1, 2026

HTab Test Crash - Root Cause Found

The Problem

Evidence

No Zig Workaround

Solution

Uh oh!

urso commented Feb 2, 2026

Uh oh!

jhf commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mcadariu commented Dec 11, 2025 •

edited

Loading

mcadariu Dec 18, 2025 •

edited

Loading

urso Dec 19, 2025 •

edited

Loading

mcadariu Dec 19, 2025 •

edited

Loading

mcadariu commented Jan 5, 2026 •

edited

Loading

mcadariu commented Jan 9, 2026 •

edited

Loading

mcadariu commented Jan 22, 2026 •

edited

Loading

jhf commented Feb 3, 2026 •

edited

Loading