Skip to content

download dynamic lib#4

Merged
adsharma merged 10 commits into
LadybugDB:masterfrom
jcalifornia:download_lib
Dec 25, 2025
Merged

download dynamic lib#4
adsharma merged 10 commits into
LadybugDB:masterfrom
jcalifornia:download_lib

Conversation

@jcalifornia
Copy link
Copy Markdown
Contributor

Download the dynamic lib into the expected place as well, to fix #2

@adsharma
Copy link
Copy Markdown
Contributor

Perhaps the CI can be updated to use this flow as well, so we don't have similar logic in two places.

@adsharma
Copy link
Copy Markdown
Contributor

  • some trailing whitespace in the diff
  • mac build needs fix (darwin vs darwin-arm64?)

jcalifornia added a commit to jcalifornia/go-ladybug that referenced this pull request Dec 18, 2025
@jcalifornia
Copy link
Copy Markdown
Contributor Author

Previously I was trying not to touch cgo_shared.go

@adsharma
Copy link
Copy Markdown
Contributor

Revert the changes to windows? It was working as of your earlier revision.

Otherwise looks good. We can deal with CI in a separate PR

@adsharma
Copy link
Copy Markdown
Contributor

I would revert to 03985f5 and change the line in cgo_shared.go back to:

#cgo windows LDFLAGS: -L${SRCDIR}/lib/dynamic/windows -llbug_shared

@jcalifornia
Copy link
Copy Markdown
Contributor Author

sorry for complicating what should have been a simple fix... i reverted back to the previous and inadvertently ended up committing my local mod tidy. I'll look at whether this is all working tomorrow

@jcalifornia
Copy link
Copy Markdown
Contributor Author

I'm thinking this should do it now. I reverted back to 1391260 where it looks like the issue was that the GitHub runner was using lib/dynamic/osx for OSX rather than lib/dynamic/darwin. Made relevant changes in the download script and cgo_shared.go

@jcalifornia
Copy link
Copy Markdown
Contributor Author

For what it's worth, I ran the runners within my fork and they passed: jcalifornia#3 @adsharma

@jcalifornia
Copy link
Copy Markdown
Contributor Author

So I wasn't aware of how go:generate operates until just now. Looks like it is not triggered by building downstream packages automatically so it appears that we need to include the generated code directly in the repo (including the downloaded libs)? Is that your understanding?

@jcalifornia
Copy link
Copy Markdown
Contributor Author

When running the download script with a flag like so: DOWNLOAD_ALL_LIBS=1 ./download_lbug.sh it now will download the libs for all the platforms.

Since libs are in the repo, as needed for compiling downstream packages that include go-ladybug, removed the download stuff from the github action.

@adsharma
Copy link
Copy Markdown
Contributor

CI seems to be passing, which is good news.

Could we avoid checking in the binaries into the repo? Downloading each time in CI seems to be a better choice because of the impact to the size of the repo and the potential for new future platforms.

@jcalifornia
Copy link
Copy Markdown
Contributor Author

jcalifornia commented Dec 24, 2025

The issue is that when someone includes go-ladybug on their downstream package, and wants to build that package, I don't know if there is a way to invoke go-generate in go-ladybug. The old kuzu go package seemed to have the binary libs included in the repo. If there isn't another workaround we could try compressing the libraries so they take less space and then decompressing at build time? That's the approach I used in another package: soundprediction/go-embedeverything#7

@jcalifornia
Copy link
Copy Markdown
Contributor Author

Here is what claude says about possible remedies @adsharma let me know which one you prefer:

Query: I have a golang project A that builds c bindings, and it requires running a script defined in go:generate to build

I have a golang project B that depends on A. How do I set up both B and A so that when building B using go build that A will pull in what it needs to build

Reply:

To make project B automatically trigger the build requirements of project A (including go:generate directives), you need to structure things carefully. Here's how to set this up:

The Challenge

By default, go build on project B won't automatically run go:generate in project A's dependencies. You have a few approaches:

Option 1: Pre-generate and Commit (Recommended)

The most common approach is to run go generate in project A and commit the generated files to the repository. This way:

In Project A:

//go:generate ./build-c-bindings.sh

Run once and commit:

cd project-A
go generate ./...
git add <generated-files>
git commit -m "Add generated C bindings"

In Project B:

go get github.com/yourorg/project-A
go build  # Just works, no special setup needed

Pros: Simple, works with all Go tooling, faster builds
Cons: Generated code in version control (though this is standard practice)

Option 2: Build Script in Project B

If you can't commit generated files, wrap the build in a script or Makefile:

In Project B's Makefile:

.PHONY: build
build:
	cd $(go list -f '{{.Dir}}' -m github.com/yourorg/project-A) && go generate ./...
	go build ./...

Or a build script:

#!/bin/bash
# build.sh in project B
A_DIR=$(go list -f '{{.Dir}}' -m github.com/yourorg/project-A)
(cd "$A_DIR" && go generate ./...)
go build ./...

Option 3: Use go generate in Project B

Add a generate directive in project B that triggers A's generation:

In Project B (e.g., in a tools.go or main file):

//go:generate sh -c "cd $(go list -f '{{.Dir}}' -m github.com/yourorg/project-A) && go generate ./..."

package main

Then users run:

cd project-B
go generate ./...
go build

Option 4: init() Function (Not Recommended)

You could theoretically use an init() function in project A to check if generated files exist and error if they don't, but this doesn't solve the automation problem.

Best Practice Recommendation

Use Option 1 (pre-generate and commit). This is the Go community standard because:

  • Generated code is treated as source code
  • Works seamlessly with all tooling
  • Faster CI/CD builds
  • No surprise build-time code generation
  • Clear what code is actually being compiled

If the generated C bindings are platform-specific, you might generate for multiple platforms and commit all variants with build tags.

@adsharma
Copy link
Copy Markdown
Contributor

option 1 for header files (lbug.h)
option 2 or 3 for project B

Also it's not clear if Claude understood the distinction between generated source code vs build artifacts (shared libs) from your prompt.

@jcalifornia
Copy link
Copy Markdown
Contributor Author

Ok that would be fine by me though it would be a deviation from go-kuzu. We need to document in the README that one would need to properly set up go:generate in the downstream project which would be fine. I'll test out in my downstream projects first before committing something

@jcalifornia
Copy link
Copy Markdown
Contributor Author

Currently we have lbug.h in the repo and have excluded the compiled libs. Added a note to the README.md on how to use go:generate in downstream projects in order to download the required binary for go-ladybug. Tests passed in my fork.

@adsharma adsharma merged commit 4b462d5 into LadybugDB:master Dec 25, 2025
4 checks passed
@adsharma
Copy link
Copy Markdown
Contributor

Looks great! Thank you for the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compilation issue

2 participants