Skip to content

android build: cache p4a recipe downloads even for FRESH_CLONE builds #10744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
SomberNight wants to merge 2 commits into
base: master
Choose a base branch
from
+21 −7
Open

android build: cache p4a recipe downloads even for FRESH_CLONE builds #10744

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dc1072c
android build: cache p4a recipe downloads even for FRESH_CLONE builds
SomberNight Jul 8, 2026
7785865
android build: fix prev: chown fresh_clone base dir before git clone
SomberNight Jul 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 21 additions & 7 deletions contrib/android/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,10 @@ mkdir -p "${PROJECT_ROOT}/.buildozer_$1"
ln -s ".buildozer_$1" ${PROJECT_ROOT}/.buildozer

DOCKER_BUILD_FLAGS=""
DOCKER_RUN_FLAGS=""
if [ ! -z "$ELECBUILD_NOCACHE" ] ; then
info "ELECBUILD_NOCACHE is set. forcing rebuild of docker image."
DOCKER_BUILD_FLAGS="--pull --no-cache"
DOCKER_BUILD_FLAGS="$DOCKER_BUILD_FLAGS --pull --no-cache"
fi

if [ -z "$ELECBUILD_COMMIT" ] ; then # local dev build
Expand All @@ -51,21 +52,34 @@ docker build \
# maybe do fresh clone
if [ ! -z "$ELECBUILD_COMMIT" ] ; then
info "ELECBUILD_COMMIT=$ELECBUILD_COMMIT. doing fresh clone and git checkout."
FRESH_CLONE=${FRESH_CLONE:-"/var/tmp/electrum_build/android/fresh_clone/electrum"}
rm -rf "$FRESH_CLONE" 2>/dev/null || ( info "we need sudo to rm prev FRESH_CLONE." && sudo rm -rf "$FRESH_CLONE" )
FRESH_CLONE_BASE=${FRESH_CLONE_BASE:-"/var/tmp/electrum_build/android"}
FRESH_CLONE="$FRESH_CLONE_BASE/electrum"
rm -rf "$FRESH_CLONE" 2>/dev/null || (
info "we need sudo to rm prev FRESH_CLONE." &&
sudo chown "$(id -u)" "$FRESH_CLONE_BASE" &&
sudo rm -rf "$FRESH_CLONE" )
umask 0022
git clone "$PROJECT_ROOT" "$FRESH_CLONE"
cd "$FRESH_CLONE"
git checkout "$ELECBUILD_COMMIT"
PROJECT_ROOT_OR_FRESHCLONE_ROOT="$FRESH_CLONE"
if [ -z "$ELECBUILD_NOCACHE" ] ; then
info "ELECBUILD_NOCACHE is not set. mounting p4a download cache."
GIT_TAG="$(git describe --abbrev=0)"
mkdir -p "$FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages"
mkdir -p "$FRESH_CLONE"/.buildozer/android/platform/build-{armeabi-v7a,arm64-v8a,x86,x86_64}/packages
DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages:/home/user/wspace/electrum/.buildozer/android/platform/build-armeabi-v7a/packages"
DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages:/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/packages"
DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages:/home/user/wspace/electrum/.buildozer/android/platform/build-x86/packages"
DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages:/home/user/wspace/electrum/.buildozer/android/platform/build-x86_64/packages"

@SomberNight SomberNight Jul 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: .buildozer/android/platform/build-$ARCH/packages is the folder p4a internally uses for caching downloads. This PR would just persist the cache through build attempts.

note2: p4a already puts a marker next to each download to detect staleness, with the version in the name. Though I remark it does not work for git clones (see plyer):

dir structure of packages folder 
$ tree -L2 -a /var/tmp/electrum_build/android/downloads_cache/4.8.0/p4a_packages/
/var/tmp/electrum_build/android/downloads_cache/4.8.0/p4a_packages/
├── cffi
│   ├── .mark-v2.0.0.tar.gz
│   └── v2.0.0.tar.gz
├── hostpython3
│   ├── .mark-Python-3.11.15.tgz
│   └── Python-3.11.15.tgz
├── hostqt6
│   ├── .mark-qt-everywhere-src-6.10.2.tar.xz
│   └── qt-everywhere-src-6.10.2.tar.xz
├── libffi
│   ├── .mark-v3.4.8.tar.gz
│   └── v3.4.8.tar.gz
├── libiconv
│   ├── .mark-libiconv-1.18.tar.gz
│   └── libiconv-1.18.tar.gz
├── libsecp256k1
│   ├── .mark-1a53f4961f337b4d166c25fce72ef0dc88806618.zip
│   └── 1a53f4961f337b4d166c25fce72ef0dc88806618.zip
├── libzbar
│   ├── .mark-bb05ec54eec57f8397cb13fb9161372a281a1219.zip
│   └── bb05ec54eec57f8397cb13fb9161372a281a1219.zip
├── openssl
│   ├── .mark-openssl-3.0.21.tar.gz
│   └── openssl-3.0.21.tar.gz
├── packaging
│   ├── .mark-packaging-26.0.tar.gz
│   └── packaging-26.0.tar.gz
├── plyer
│   ├── .mark-plyer
│   └── plyer
├── pycparser
│   ├── .mark-pycparser-2.22.tar.gz
│   └── pycparser-2.22.tar.gz
├── pycryptodomex
│   ├── .mark-v3.23.0x.tar.gz
│   └── v3.23.0x.tar.gz
├── pyjnius
│   ├── .mark-1.7.0.zip
│   └── 1.7.0.zip
├── pyparsing
│   ├── .mark-pyparsing-3.0.7.tar.gz
│   └── pyparsing-3.0.7.tar.gz
├── pyqt6
│   ├── .mark-pyqt6-6.10.2.tar.gz
│   └── pyqt6-6.10.2.tar.gz
├── pyqt6sip
│   ├── .mark-pyqt6_sip-13.10.3.tar.gz
│   └── pyqt6_sip-13.10.3.tar.gz
├── pyqt_builder
│   ├── .mark-pyqt_builder-1.19.1.tar.gz
│   └── pyqt_builder-1.19.1.tar.gz
├── python3
│   ├── .mark-Python-3.11.15.tgz
│   └── Python-3.11.15.tgz
├── qt6
│   ├── .mark-qt-everywhere-src-6.10.2.tar.xz
│   └── qt-everywhere-src-6.10.2.tar.xz
├── sip
│   ├── .mark-sip-6.15.1.tar.gz
│   └── sip-6.15.1.tar.gz
├── six
│   ├── .mark-six-1.17.0.tar.gz
│   └── six-1.17.0.tar.gz
└── sqlite3
    ├── .mark-sqlite-amalgamation-3500000.zip
    └── sqlite-amalgamation-3500000.zip

24 directories, 43 files

fi
else
info "not doing fresh clone."
fi

DOCKER_RUN_FLAGS=""
if [[ "$3" == "release" ]] ; then
info "'release' mode selected. mounting ~/.keystore inside container."
DOCKER_RUN_FLAGS="-v $HOME/.keystore:/home/user/.keystore"
DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $HOME/.keystore:/home/user/.keystore"
fi
if sh -c ": >/dev/tty" >/dev/null 2>/dev/null; then
info "/dev/tty is available and usable"
Expand All @@ -77,8 +91,8 @@ mkdir --parents "$PROJECT_ROOT_OR_FRESHCLONE_ROOT"/.buildozer/.gradle
# check uid and maybe chown. see #8261
if [ ! -z "$ELECBUILD_COMMIT" ] ; then # fresh clone (reproducible build)
if [ $(id -u) != "1000" ] || [ $(id -g) != "1000" ] ; then
info "need to chown -R FRESH_CLONE dir. prompting for sudo."
sudo chown -R 1000:1000 "$FRESH_CLONE"
info "need to chown -R FRESH_CLONE_BASE dir. prompting for sudo."
sudo chown -R 1000:1000 "$FRESH_CLONE_BASE"
Comment thread
f321x marked this conversation as resolved.
fi
fi
docker run --rm \
Expand Down