Skip to content

Infra Update v8: spack v1 Support#32

Open
CodeGat wants to merge 18 commits intomainfrom
infra-update-v8
Open

Infra Update v8: spack v1 Support#32
CodeGat wants to merge 18 commits intomainfrom
infra-update-v8

Conversation

@CodeGat
Copy link
Member

@CodeGat CodeGat commented Nov 18, 2025

References issue ACCESS-NRI/build-cd#313 and PR ACCESS-NRI/build-cd#326
References rollout issue ACCESS-NRI/build-cd#328
References project Spack `0.22` to `1.0` Migration

Important

This PR is a major update to the deployment infrastructure. See below for the prerequisites for this repository to be able to merge this PR.

Important

This major version change marks the end of major infrastructure updates for deployments to spack < 1.0.
If you want to deploy to instances of spack < 1.0, you must use build-cd < v8.
If you want to deploy to instances of spack >= 1.0, you must use build-cd >= v8.

Background

We are moving on up to spack v1! This update to spack contains many bug fixes, optimisations and new features that can be incorporated into build-cd.

This update also contains changes that make spack v0.X incompatible with this build-cd v8 update - the most prominent one being the splitting of spacks core codebase from it's builtin spack-packages repo. This means that for provenance, one to keep track of both builtin spack-packages and our own, renamed access-spack-packages. build-cd will centrally control the version of builtin spack-packages for a given instance, and config/versions.jsons spack-packages key is renamed to access-spack-packages.

As noted earlier, this major version of build-cd can only support spack v1, due to spack v1-specific commands in the infrastructure. If you still want to deploy to spack v0.X, you will need to change build-cd to v7, and update the config/versions.json file/spack.yaml.

Features

  • Support for spack v1: We're moving to a new, non-beta version of spack! It contains bug fixes, features and optimisations over the old version.

Prerequisites for Merging

  • Update build-cd entrypoints (this PR!)
  • Update config/versions.json with new inputs (also this PR!)
  • Validate compiler additions to the spack manifest (this PR, too!)
  • Test and Verify Model built with spack v1

🚀 The latest prerelease access-issm/pr32-14 at 0eb7e09 is here: #32 (comment) 🚀

@CodeGat CodeGat added the type:infra Related to CI/CD label Nov 18, 2025
@CodeGat CodeGat self-assigned this Nov 18, 2025
@github-actions
Copy link

🚀 Attempted to deploy access-issm Prerelease pr32-1 with commit 2bad7f8

🖥️ Gadi Deployment ❌

@github-actions
Copy link

🚀 Attempted to deploy access-issm Prerelease pr32-2 with commit bd331da

🖥️ Gadi Deployment ❌

@github-actions
Copy link

🚀 Attempted to deploy access-issm Prerelease pr32-3 with commit 1951b7d

🖥️ Gadi Deployment ❌

@harshula
Copy link
Collaborator

FYI, This PR is likely impacted by spack/spack-packages#3505

@github-actions
Copy link

🚀 Attempted to deploy access-issm Prerelease pr32-4 with commit fa86b1e

🖥️ Gadi Deployment ❌

@github-actions
Copy link

🚀 Attempted to deploy access-issm Prerelease pr32-5 with commit 78cc570

🖥️ Gadi Deployment ❌

@github-actions
Copy link

github-actions bot commented Mar 5, 2026

🚀 Attempted to deploy access-issm Prerelease pr32-8 with commit fc6fb2a

🖥️ Gadi Deployment ❌

@harshula
Copy link
Collaborator

harshula commented Mar 5, 2026

PR ACCESS-NRI/access-spack-packages#395 should fix missing compiler errors.

@github-actions
Copy link

github-actions bot commented Mar 5, 2026

🚀 Attempted to deploy access-issm Prerelease pr32-9 with commit d7c0552

🖥️ Gadi Deployment ❌

* Wokaround till spack/spack-packages#3644
  is included in an ACCESS-NRI release of `spack-config`.
@github-actions
Copy link

github-actions bot commented Mar 5, 2026

🚀 Attempted to deploy access-issm Prerelease pr32-10 with commit 71a5447

🖥️ Gadi Deployment ❌

@github-actions
Copy link

github-actions bot commented Mar 5, 2026

🚀 Attempted to deploy access-issm Prerelease pr32-11 with commit 2e65089

🖥️ Gadi Deployment ❌

@harshula
Copy link
Collaborator

harshula commented Mar 5, 2026

Hi @CodeGat , It looks like the zip timestamp build failure is not transient in Spack v1.1.

@CodeGat
Copy link
Member Author

CodeGat commented Mar 5, 2026

I'm looking into a fix into the package definition now.

EDIT: See ACCESS-NRI/access-spack-packages#396

@CodeGat CodeGat temporarily deployed to Gadi Prerelease March 6, 2026 02:52 — with GitHub Actions Inactive
@github-actions
Copy link

github-actions bot commented Mar 6, 2026

🚀 Attempted to deploy access-issm Prerelease pr32-12 with commit b979f82

🖥️ Gadi Deployment ✔️

Usage Instructions

access-issm, defined in spack.yaml, will be deployed to Gadi as:

  • 2025.11.0 as a Release (when merged).
  • pr32-12 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-issm/pr32-12

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/1.1/spack in the access-issm-pr32-12 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

If the above was not what was expected, commit changes to config/versions.json in this PR, or propose changes to build-cds config/settings.json.

@CodeGat
Copy link
Member Author

CodeGat commented Mar 6, 2026

The above access-spack-packages fix seems to work. When it's reviewed/merged, I can remove this temp commit.

@CodeGat CodeGat temporarily deployed to Gadi Prerelease March 6, 2026 04:27 — with GitHub Actions Inactive
@github-actions
Copy link

github-actions bot commented Mar 6, 2026

🚀 Attempted to deploy access-issm Prerelease pr32-13 with commit a1861a5

🖥️ Gadi Deployment ✔️

Usage Instructions

access-issm, defined in spack.yaml, will be deployed to Gadi as:

  • 2025.11.0 as a Release (when merged).
  • pr32-13 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-issm/pr32-13

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/1.1/spack in the access-issm-pr32-13 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

If the above was not what was expected, commit changes to config/versions.json in this PR, or propose changes to build-cds config/settings.json.

@CodeGat
Copy link
Member Author

CodeGat commented Mar 6, 2026

Fix was merged, and deployed successfully without error. I think we can consider this ready to merge, once it is ready for the ISM team.

@CodeGat CodeGat marked this pull request as ready for review March 6, 2026 04:35
@CodeGat CodeGat requested a review from headmetal March 6, 2026 04:35
@github-actions
Copy link

github-actions bot commented Mar 6, 2026

🚀 Attempted to deploy access-issm Prerelease pr32-14 with commit a1861a5

🖥️ Gadi Deployment ❌

@CodeGat CodeGat deployed to Gadi Prerelease March 6, 2026 04:42 — with GitHub Actions Active
@github-actions
Copy link

github-actions bot commented Mar 6, 2026

🚀 Attempted to deploy access-issm Prerelease pr32-14 with commit 0eb7e09

🖥️ Gadi Deployment ✔️

Usage Instructions

access-issm, defined in spack.yaml, will be deployed to Gadi as:

  • 2025.11.0 as a Release (when merged).
  • pr32-14 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-issm/pr32-14

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/1.1/spack in the access-issm-pr32-14 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

If the above was not what was expected, commit changes to config/versions.json in this PR, or propose changes to build-cds config/settings.json.

@CodeGat
Copy link
Member Author

CodeGat commented Mar 6, 2026

One question for the audience - is it okay if I explicitly require netlib-scalapack rather than the virtual package scalapack?

@harshula
Copy link
Collaborator

One question for the audience - is it okay if I explicitly require netlib-scalapack rather than the virtual package scalapack?

That's actually what is done by:

    scalapack:
      require:
      - 'netlib-scalapack@2.2.0'

The issm SPR contains:

depends_on("scalapack")

Or are you asking whether this can be changed to depends_on("netlib-scalapack")?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:infra Related to CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants