Go generate cannot modify pkg cache directory.

[hungrymonkey]

Steps to reproduce the bug

1. Setup a project that requires cgo shared libraries
2. go generate

Description:

go-ladybug download lbug.h as a readonly file. download_lbug.sh cannot copy lbug.h into the cache directory.

Expected results:

The script succeed.

Actual results.

download_lbug.sh script fails

go generate ./...
🔍 Detected OS: osx, Architecture: aarch64
📦 Downloading asset: liblbug-osx-universal.tar.gz
   Downloading from: https://github.com/LadybugDB/ladybug/releases/latest/download/liblbug-osx-universal.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 9244k  100 9244k    0     0  21.1M      0 --:--:-- --:--:-- --:--:-- 21.1M
cp: /Users/ted/go/pkg/mod/github.com/!ladybug!d!b/go-ladybug@v0.13.1/lbug.h: Permission denied
cgo_shared.go:3: running "sh": exit status 1
main.go:3: running "sh": exit status 1

 go generate
++ uname -s
+ os=Darwin
+ case $os in
+ os=osx
++ uname -m
+ arch=arm64
+ case $arch in
+ arch=aarch64
+ '[' osx = osx ']'
+ asset=liblbug-osx-universal.tar.gz
+ ext=tar.gz
+ echo '🔍 Detected OS: osx, Architecture: aarch64'
🔍 Detected OS: osx, Architecture: aarch64
+ echo '📦 Downloading asset: liblbug-osx-universal.tar.gz'
📦 Downloading asset: liblbug-osx-universal.tar.gz
++ mktemp -d
+ temp_dir=/var/folders/v1/6wdw86q9739_sw6ymfkp0lpm0000gn/T/tmp.sRkl1C0Gd8
+ cd /var/folders/v1/6wdw86q9739_sw6ymfkp0lpm0000gn/T/tmp.sRkl1C0Gd8
+ download_url=https://github.com/LadybugDB/ladybug/releases/latest/download/liblbug-osx-universal.tar.gz
+ echo '   Downloading from: https://github.com/LadybugDB/ladybug/releases/latest/download/liblbug-osx-universal.tar.gz'
   Downloading from: https://github.com/LadybugDB/ladybug/releases/latest/download/liblbug-osx-universal.tar.gz
+ command -v curl
+ curl -L -o liblbug-osx-universal.tar.gz https://github.com/LadybugDB/ladybug/releases/latest/download/liblbug-osx-universal.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 9244k  100 9244k    0     0  24.9M      0 --:--:-- --:--:-- --:--:-- 24.9M
+ '[' tar.gz = tar.gz ']'
+ tar -xzf liblbug-osx-universal.tar.gz
++ find . -name lbug.h
++ head -1
+ lbug_file=./lbug.h
+ '[' -n ./lbug.h ']'
+ cp ./lbug.h '/Users/ted/go/pkg/mod/github.com/!ladybug!d!b/go-ladybug@v0.13.1'

 ls -l /Users/ted/go/pkg/mod/github.com/\!ladybug\!d\!b/go-ladybug@v0.13.1 
total 448
-r--r--r--  1 ted  staff    485 Dec 23 13:15 cgo_shared.go
-r--r--r--  1 ted  staff    708 Dec 23 13:15 CODE_OF_CONDUCT.md
-r--r--r--  1 ted  staff   4044 Dec 23 13:15 connection_test.go
-r--r--r--  1 ted  staff   5514 Dec 23 13:15 connection.go
-r--r--r--  1 ted  staff   2058 Dec 23 13:15 CONTRIBUTING.md
-r--r--r--  1 ted  staff   2508 Dec 23 13:15 database_test.go
-r--r--r--  1 ted  staff   3170 Dec 23 13:15 database.go
dr-xr-xr-x  4 ted  staff    128 Dec 23 13:15 dataset
-r--r--r--  1 ted  staff   1705 Dec 23 13:15 download_lbug.sh
-r--r--r--  1 ted  staff   2385 Dec 23 13:15 driver_test.go
-r--r--r--  1 ted  staff   7306 Dec 23 13:15 driver.go
-r--r--r--  1 ted  staff   2458 Dec 23 13:15 flat_tuple_test.go
-r--r--r--  1 ted  staff   2315 Dec 23 13:15 flat_tuple.go
-r--r--r--  1 ted  staff    323 Dec 23 13:15 go.mod
-r--r--r--  1 ted  staff   1221 Dec 23 13:15 go.sum
-r--r--r--  1 ted  staff   2670 Dec 23 13:15 helper_test.go
-r--r--r--  1 ted  staff  71582 Dec 23 13:15 lbug.h
-r--r--r--  1 ted  staff   1072 Dec 23 13:15 LICENSE
-r--r--r--  1 ted  staff   8332 Dec 23 13:15 parameter_test.go
-r--r--r--  1 ted  staff    713 Dec 23 13:15 prepared_statement.go
-r--r--r--  1 ted  staff   4775 Dec 23 13:15 query_result_test.go
-r--r--r--  1 ted  staff   4819 Dec 23 13:15 query_result.go
-r--r--r--  1 ted  staff   2707 Dec 23 13:15 README.md
-r--r--r--  1 ted  staff   2247 Dec 23 13:15 time_helper.go
-r--r--r--  1 ted  staff  22306 Dec 23 13:15 value_helper.go
-r--r--r--  1 ted  staff  14547 Dec 23 13:15 value_test.go

[jcalifornia]

it's related to our solution to #4 ... the header file is already in the repo/is skippable so we can have the shell script gracefully handle not being able to write for it. Concern is for the libs where it is looking in these directories:

/*
#cgo darwin LDFLAGS: -lc++ -L${SRCDIR}/lib/dynamic/osx -llbug -Wl,-rpath,${SRCDIR}/lib/dynamic/osx
#cgo linux,amd64 LDFLAGS: -L${SRCDIR}/lib/dynamic/linux-amd64 -llbug -Wl,-rpath,${SRCDIR}/lib/dynamic/linux-amd64
#cgo linux,arm64 LDFLAGS: -L${SRCDIR}/lib/dynamic/linux-arm64 -llbug -Wl,-rpath,${SRCDIR}/lib/dynamic/linux-arm64
#cgo windows LDFLAGS: -L${SRCDIR}/lib/dynamic/windows -llbug_shared
#include "lbug.h"
*/

[hungrymonkey]

Tested go master. It didn't solve the issue

go generate
Target Directory: lib/dynamic/osx
Detected OS: osx, Architecture: aarch64
   Downloading from: https://github.com/LadybugDB/ladybug/releases/latest/download/liblbug-osx-universal.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 9244k  100 9244k    0     0  25.0M      0 --:--:-- --:--:-- --:--:-- 69.4M
cp: /Users/ted/go/pkg/mod/github.com/!ladybug!d!b/go-ladybug@v0.13.2-0.20251225172650-4b462d5b189a/lbug.h: Permission denied
cgo_shared.go:3: running "sh": exit status 1
main.go:3: running "sh": exit status 1
ted@Teds-MacBook-Pro microservice % git diff
diff --git a/microservice/go.mod b/microservice/go.mod
index 1860877..423f030 100644
--- a/microservice/go.mod
+++ b/microservice/go.mod
@@ -37,7 +37,7 @@ require (
 
 require (
        cloud.google.com/go/compute/metadata v0.9.0 // indirect
-       github.com/LadybugDB/go-ladybug v0.12.2-patch2 // indirect
+       github.com/LadybugDB/go-ladybug v0.13.2-0.20251225172650-4b462d5b189a // indirect
        github.com/PuerkitoBio/goquery v1.10.2 // indirect
        github.com/ajg/form v1.5.1 // indirect
        github.com/andybalholm/brotli v1.2.0 // indirect
diff --git a/microservice/go.sum b/microservice/go.sum
index 8c1af5f..be2063c 100644
--- a/microservice/go.sum
+++ b/microservice/go.sum
@@ -4,6 +4,10 @@ filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
 filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
 github.com/LadybugDB/go-ladybug v0.12.2-patch2 h1:IcqpgYbODFzgqjyNoaz1c6OPg3ckLiSi1qcW2Qgm2WY=
 github.com/LadybugDB/go-ladybug v0.12.2-patch2/go.mod h1:f5RET9iUFgH+gLI6l/uJxAE4tXdYRdsDP9dN0Gr3M1M=
+github.com/LadybugDB/go-ladybug v0.13.1 h1:X11ch5sIsHHY2wqKx5phmvXi5aES9zMjRj3qkpUWTgU=
+github.com/LadybugDB/go-ladybug v0.13.1/go.mod h1:f5RET9iUFgH+gLI6l/uJxAE4tXdYRdsDP9dN0Gr3M1M=
+github.com/LadybugDB/go-ladybug v0.13.2-0.20251225172650-4b462d5b189a h1:v4Ma6qkT0WDhzzQEJ/fQQNZI0pjrYBKLGn5dzUJP9gs=
+github.com/LadybugDB/go-ladybug v0.13.2-0.20251225172650-4b462d5b189a/go.mod h1:ddOn3V5BN15P7nQT7ndCTNj06lIUcgbPIm8Wenznv+s=
 github.com/PuerkitoBio/goquery v1.10.2 h1:7fh2BdHcG6VFZsK7toXBT/Bh1z5Wmy8Q9MV9HqT2AM8=
 github.com/PuerkitoBio/goquery v1.10.2/go.mod h1:0guWGjcLu9AYC7C1GHnpysHy056u9aEkUHwhdnePMCU=
 github.com/ajg/form v1.5.1 h1:t9c7v8JUKu/XxOGBU0yjNpaMloxGEJhUkqFRq0ibGeU=
@@ -130,6 +134,8 @@ github.com/jackc/pgx/v5 v5.7.6 h1:rWQc5FwZSPX58r1OQmkuaNicxdmExaEz5A2DO2hUuTk=
 github.com/jackc/pgx/v5 v5.7.6/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
 github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
 github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
+github.com/jcalifornia/go-ladybug v0.0.0-20251223183102-c1ad75dc2fee h1:1Ksoak9Bu/q4dIn7F7J3H2pajavdB4yLEwZvbd2mnpM=
+github.com/jcalifornia/go-ladybug v0.0.0-20251223183102-c1ad75dc2fee/go.mod h1:ddOn3V5BN15P7nQT7ndCTNj06lIUcgbPIm8Wenznv+s=
 github.com/jmoiron/sqlx v1.4.0 h1:1PLqN7S1UYp5t4SrVVnt4nUVNemrDAtxlulVe+Qgm3o=
 github.com/jmoiron/sqlx v1.4.0/go.mod h1:ZrZ7UsYB/weZdl2Bxg6jCRO9c3YHl8r3ahlKmRT4JLY=
 github.com/kaptinlin/jsonrepair v0.2.4 h1:PmPBdbT7N8We8RseBuhCB2oW8s5pikMeLOWF04qUUQs=

[hungrymonkey]

https://go.dev/ref/mod?utm_source=chatgpt.com#build-commands

Build commands[¶](https://go.dev/ref/mod?utm_source=chatgpt.com#build-commands)

All commands that load information about packages are module-aware. This includes:

    go build
    go fix
    go generate
    go install
    go list
    go run
    go test
    go vet

When run in module-aware mode, these commands use go.mod files to interpret import paths listed on the command line or written in Go source files. These commands accept the following flags, common to all module commands.

    The -mod flag controls whether go.mod may be automatically updated and whether the vendor directory is used.
        -mod=mod tells the go command to ignore the vendor directory and to [automatically update](https://go.dev/ref/mod?utm_source=chatgpt.com#go-mod-file-updates) go.mod, for example, when an imported package is not provided by any known module.
        -mod=readonly tells the go command to ignore the vendor directory and to report an error if go.mod needs to be updated.
        -mod=vendor tells the go command to use the vendor directory. In this mode, the go command will not use the network or the module cache.
        By default, if the [go version](https://go.dev/ref/mod?utm_source=chatgpt.com#go-mod-file-go) in go.mod is 1.14 or higher and a vendor directory is present, the go command acts as if -mod=vendor were used. Otherwise, the go command acts as if -mod=readonly were used.
        go get rejects this flag as the purpose of the command is to modify dependencies, which is only allowed by -mod=mod.
    The -modcacherw flag instructs the go command to create new directories in the module cache with read-write permissions instead of making them read-only. When this flag is used consistently (typically by setting GOFLAGS=-modcacherw in the environment or by running go env -w GOFLAGS=-modcacherw), the module cache may be deleted with commands like rm -r without changing permissions first. The [go clean -modcache](https://go.dev/ref/mod?utm_source=chatgpt.com#go-clean-modcache) command may be used to delete the module cache, whether or not -modcacherw was used.
    The -modfile=file.mod flag instructs the go command to read (and possibly write) an alternate file instead of go.mod in the module root directory. The file’s name must end with .mod. A file named go.mod must still be present in order to determine the module root directory, but it is not accessed. When -modfile is specified, an alternate go.sum file is also used: its path is derived from the -modfile flag by trimming the .mod extension and appending .sum.

[adsharma]

Can download_lbug.sh be modified to skip lbug.h if it already exists?

We'll need a special mode for running go generate on major release upgrades when the API could've changed.

Also who or what is setting up DOWNLOAD_ALL_LIBS? I don't see any references to it outside that script.

[jcalifornia]

I added DOWNLOAD_ALL_LIBS back when we were including the libs in the repo and kept it for the purpose of doing cross-builds. We should just skip lbug.h if it exists but the problem is still what to do about lib/dynamic

[jcalifornia]

note that the fix won't work until after changes as merged because it is looking to run the script https://raw.githubusercontent.com/LadybugDB/go-ladybug/refs/heads/master/download_lbug.sh
which wont have the necessary changes.

for now, using
https://raw.githubusercontent.com/jcalifornia/go-ladybug/refs/heads/readonly_modcache_fix/download_lbug.sh
in the go:generate directive works @hungrymonkey

[jcalifornia]

refactored the download script. It also now doesn't unnecessarily place the header file into external source anymore.