Add GIX_TEST_FIXTURE_HASH to gix-testtools#2377
Conversation
|
Thanks a lot, I am looking forward to sorting this out! There is also #2378, hopefully done in the next couple of days (it might conflict). |
|
Thanks a lot for starting this! This crate is special as it's the first one that parses a file-format that changes depending on the hash kind. It seems like one commit currently in the |
|
Assuming you’re referring to the one adding Apart from that, my plan is to go through the tests and, for every tests that currently uses |
Yes, that's the one, thanks 🙏.
Sorry for the confusion, that's the way. The only thing tests have to do is to actively set all hashes using feature toggles, and they should just set sha1/sha256 while they are at it. Once a crate has been converted, the tests should automatically test both hashes, and make sure they are compiled in. |
bde9c33 to
7bc1409
Compare
|
I added fixtures for SHA256 hashes to |
|
I think I’ve now gotten to a point where I need some feedback in order to be able to continue. (CI is still running, so I also might need to address any issue that comes up there.) Specifically, I’ve got the following questions:
|
Byron
left a comment
There was a problem hiding this comment.
I think this is great!
The PR runs tests for both hashes through cargo nextest run -p gix-commitgraph --features sha256 --no-fail-fast. Is that correct?
I made a note in Cargo.toml which should lead to the extra run in the justfile to be removed. Please take a look to see if it makes sense.
Last time CI ran it did not complain about any of the generated archives in gix-commitgraph, but when I delete both generated-archives as well as generated-do-not-edit in order to regenerate the archives, git status shows that most files have changed their size by 512 bytes and that one file has been both deleted and modified. (I’m running cargo test --features sha256 on Ubuntu 25.10.) What’s the best way to debug this?
generated-do-no-edit would be the folder to delete for it to restore from archive or regenerate the local cache in generated-do-not-edit. That's all there is to it. With the sha256 versions showing up, I'd expect it to want to create the sha256 versions which seems to have happened as well. It seems to work from my PoV, but I will take a closer look when reviewing properly.
PS: removing generated-archives will regenerate them, and they always appear changed. That's normal, and one would avoid deleting the tracked files.
Is there anything else I would need to change or include before I can mark this PR as ready?
It looks like it's ready for a non-armchair and final review after this one 🎉.
tests/tools/src/lib.rs
Outdated
| .env_remove("GIT_WORK_TREE") | ||
| .env_remove("GIT_COMMON_DIR") | ||
| .env_remove("GIT_ASKPASS") | ||
| .env_remove("GIT_DEFAULT_HASH") |
There was a problem hiding this comment.
I think the remove here isn't needed if we are always overriding. If there is something special, then let's keep this and document it though.
gix-commitgraph/tests/access/mod.rs
Outdated
| "overflow happened, can't represent huge dates" | ||
| ); | ||
| assert_eq!( | ||
| for hash_kind in gix_hash::Kind::all() { |
There was a problem hiding this comment.
Could all() rather return an iterator that hands out owned versions of hash_kind? I think that should work with impl Iterator<Item = Kind>.
gix-commitgraph/Cargo.toml
Outdated
|
|
||
| [features] | ||
| ## Support for SHA256 hashes and digests. | ||
| sha256 = ["gix-hash/sha256"] |
There was a problem hiding this comment.
I think we'd want to use dev-dependencies to always add sha256 to the features when testing. That way, there is only one way to run tests: all of them.
There was a problem hiding this comment.
That means removing the feature here, is that correct?
There was a problem hiding this comment.
Yes, let's start with that.
This means we say that plumbing will never set hash features, and it expects to be able to see the hash to be used at runtime via gix-hash::Kind. Most implementations do that already, and I think this should work for most cases.
There was a problem hiding this comment.
Makes sense, removed.
|
As I am looking at Git for how they tackle this I wonder this: Should we, rather than changing every test and looping over it, prefer to build with all hashes, but run tests for each hash selectively, maybe by setting an environment variable that controls which hash is under test? Performance-wise it should be very similar, even though development is a bit less convenient as one will have to run twice with different flags or environment variables set. There, having the tests run everything automatically, always, and what we have now, seems better. Also, how do we handle expectations on hashes, particularly those with insta? It basically requires to chose one hash and find/replace the actual output to match that. I.e. So I think it's fine to keep this PR as is as it's happy with this simple and convenient solution, and we can probably layer moer on top of that when a need arises. |
|
I’m glad you mention this other option because I’m still very hesitant to force every test to be rewritten to make it loop over hash kinds as well. What I could try instead is adding an environment variable, e. g. |
|
I’ve added a |
That's fair, and probably that's best whenever there is no actual assertions against a hash. The manual looping is always something one can implement if that's a good choice in specific cases. With such a system, we'd need just one more job to run the SHA256 version of the test suite, which just sets an environment variable.
Oh, right, the If In any case, it seems we'd have to find a way to get CI green. Something that I'd not want to lose is to see what the best possible size is, i.e. if just one hash is supported and that is the smallest hash. But then one would have to test the individual crate just with one hash kind which may also be difficult in future. So… maybe just change the size expectations to the new value, while also using |
|
Oh, and what happens if the crate is now tested individually? |
|
We could add the following line to This would duplicate the dependency, though. |
Yes, such a line would be needed via Lastly, it looks like there will never be a need to add
|
|
CI is now green! Quick summary of this PR’s changes:
|
Byron
left a comment
There was a problem hiding this comment.
Exciting!
It adds GIX_TEST_HASH which affects the default hash used in fixture scripts via GIT_DEFAULT_HASH. (We might want to consider renaming to GIX_TEST_FIXTURE_HASH or something that makes it clearer this is related to fixtures.)
Yes, I'd also like that.
I left a comment, but ideally it's applied to all the numbers so we keep track of the SHA1 versions.
gix-commitgraph/Cargo.toml
Outdated
|
|
||
| [features] | ||
| ## Support for SHA256 hashes and digests. | ||
| sha256 = ["gix-hash/sha256"] |
There was a problem hiding this comment.
Yes, let's start with that.
This means we say that plumbing will never set hash features, and it expects to be able to see the hash to be used at runtime via gix-hash::Kind. Most implementations do that already, and I think this should work for most cases.
There was a problem hiding this comment.
What surprises me is that there is no way to check what hash format the graph is using. Could we have an implementation of object_hash (which exists on File) for the Graph, and assert on the value.
But then we'd need a way to get the currently set hash, which had to come from gix-testtools.
I think this would be very useful to have and some assertion should definitely be made.
There was a problem hiding this comment.
Are you thinking about something like the following:
/// The kind of hash used in this `Graph`.
///
/// Note that it is always conforming to the hash used in the owning repository.
///
/// # Panics
///
/// If the graph does not contain any `File`.
pub fn object_hash(&self) -> gix_hash::Kind {
self.files
.first()
.map(super::File::object_hash)
.expect("graph to have at least one file")
}There was a problem hiding this comment.
Are you interested in a follow-up? There is the nonempty crate which I think would do wonders here (and in other places).
But besides that, yes, this is what I imagined.
There was a problem hiding this comment.
Absolutely! I’ll have a look at nonempty. I assume we want to wrap self.files? Or are you thinking about a more local use just in object_hash?
There was a problem hiding this comment.
I’ve now added Graph::object_hash.
|
@Byron I’ve merged |
There was a problem hiding this comment.
Pull request overview
This PR updates the test/fixture infrastructure and various dev-dependencies so SHA-256 object-format test runs can be supported (notably via hash-kind-specific fixtures and SHA-256-enabled gix-hash in test builds), and adds SHA-256 commitgraph fixtures.
Changes:
- Add
GIX_TEST_FIXTURE_HASH-driven fixture selection and pass hash kind into fixture scripts viaGIT_DEFAULT_HASH. - Enable
gix-hash’ssha256feature across multiple crates’ test/dev builds and adjust size ceiling tests accordingly. - Add SHA-256 commitgraph fixture archives and update a test expectation to reflect the new fixture directory layout.
Reviewed changes
Copilot reviewed 24 out of 31 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/tools/src/lib.rs | Add hash-kind-aware fixture selection and propagate hash kind into fixture scripts |
| tests/tools/Cargo.toml | Add gix-hash dependency required by new fixture selection logic |
| justfile | Run selected test packages under explicit hash-related env vars |
| gix/tests/gix/status.rs | Relax size expectations to account for SHA-256-enabled builds |
| gix/tests/gix/revision/spec/from_bytes/mod.rs | Update expected error path to include new sha1 fixture directory segment |
| gix/tests/gix/object/mod.rs | Adjust size ceilings for SHA-256-enabled builds |
| gix/src/id.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix/Cargo.toml | Enable gix-hash/sha256 for gix dev/test builds |
| gix-worktree/tests/Cargo.toml | Enable gix-hash/sha256 for test crate builds |
| gix-worktree-state/tests/Cargo.toml | Enable gix-hash/sha256 for test crate builds |
| gix-revwalk/tests/revwalk.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix-ref/src/raw.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix-pack/tests/pack/iter.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix-pack/tests/pack/data/output/mod.rs | Adjust size ceilings for SHA-256-enabled builds |
| gix-pack/tests/Cargo.toml | Update test crate metadata and enable gix-hash/sha256 |
| gix-pack/src/cache/delta/tree.rs | Adjust large-array size ceiling for SHA-256-enabled builds |
| gix-pack/Cargo.toml | Enable gix-hash/sha256 for gix-pack dev/test builds |
| gix-object/tests/object/main.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix-negotiate/tests/negotiate.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix-index/tests/index/mod.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix-index/tests/Cargo.toml | Enable gix-hash/sha256 for test crate builds |
| gix-index/src/extension/tree/mod.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix-diff/src/tree/visit.rs | Adjust size ceiling for SHA-256-enabled builds |
| gix-commitgraph/tests/fixtures/generated-archives/single_parent_sha256.tar | Add SHA-256 fixture archive for commitgraph tests |
| gix-commitgraph/tests/fixtures/generated-archives/single_commit_sha256.tar | Add SHA-256 fixture archive for commitgraph tests |
| gix-commitgraph/Cargo.toml | Add SHA-256-enabled gix-hash dev-dependency for commitgraph tests |
| Cargo.lock | Record new gix-hash dependency edge for tests/tools |
tests/tools/src/lib.rs
Outdated
| let gix_test_hash = env::var_os("GIX_TEST_FIXTURE_HASH") | ||
| .and_then(|os_string| os_string.into_string().ok()) | ||
| .unwrap_or_default(); | ||
| let hash_kind = gix_hash::Kind::from_str(&gix_test_hash).unwrap_or_default(); | ||
|
|
There was a problem hiding this comment.
GIX_TEST_FIXTURE_HASH values that fail to parse (including requesting sha256 when the gix-hash/sha256 feature isn’t enabled) currently fall back to the default hash via unwrap_or_default(). This can silently run tests/fixtures with SHA-1 while the caller believes SHA-256 is being exercised. Consider treating a non-empty env var that fails Kind::from_str() as an error (or at least emit a clear warning listing accepted values).
There was a problem hiding this comment.
I like that! Changed. (You might want to tune the specific behaviour.)
justfile
Outdated
| env GIX_TEST_HASH=sha1 cargo nextest run -p gix-worktree-state-tests --features gix-features-parallel --no-fail-fast | ||
| env GIX_TEST_HASH=sha1 cargo nextest run -p gix-worktree-tests --features gix-features-parallel --no-fail-fast |
There was a problem hiding this comment.
GIX_TEST_HASH appears to be unused in the repository (no code reads it), so setting it here likely has no effect and may be confusing. Either remove these env GIX_TEST_HASH=... prefixes or switch to the env var that the test tooling actually consumes (e.g. GIX_TEST_FIXTURE_HASH for fixture selection).
| env GIX_TEST_HASH=sha1 cargo nextest run -p gix-worktree-state-tests --features gix-features-parallel --no-fail-fast | |
| env GIX_TEST_HASH=sha1 cargo nextest run -p gix-worktree-tests --features gix-features-parallel --no-fail-fast | |
| env GIX_TEST_FIXTURE_HASH=sha1 cargo nextest run -p gix-worktree-state-tests --features gix-features-parallel --no-fail-fast | |
| env GIX_TEST_FIXTURE_HASH=sha1 cargo nextest run -p gix-worktree-tests --features gix-features-parallel --no-fail-fast |
There was a problem hiding this comment.
My oversight, fixed.
justfile
Outdated
| cargo nextest run -p gix-pack-tests --features all-features --no-fail-fast | ||
| cargo nextest run -p gix-pack-tests --features gix-features-parallel --no-fail-fast | ||
| cargo nextest run -p gix-index-tests --features gix-features-parallel --no-fail-fast | ||
| env GIX_TEST_HASH=sha1 cargo nextest run -p gix-index-tests --features gix-features-parallel --no-fail-fast |
There was a problem hiding this comment.
GIX_TEST_HASH appears to be unused in the repository (no code reads it), so setting it here likely has no effect and may be confusing. Either remove this env GIX_TEST_HASH=... prefix or switch to the env var that the test tooling actually consumes.
| env GIX_TEST_HASH=sha1 cargo nextest run -p gix-index-tests --features gix-features-parallel --no-fail-fast | |
| env GIX_TEST_FIXTURE_HASH=sha1 cargo nextest run -p gix-index-tests --features gix-features-parallel --no-fail-fast |
There was a problem hiding this comment.
My oversight, fixed.
|
Thanks a lot! Yes, I think it's ready. I gave it another quick look and had a comment about |
Awesome! I’ve tried to addressed the outstanding comments. I’ve also merged |
|
Uff, I think I should add to the contribution guidelines to never merge main, but always rebase. While the diff is still what it's supposed to be, I feel 'unsafe' when it's unclear which commits are actually added here. Would you fix that for me please? |
4f01761 to
ea0d591
Compare
|
I’ve now rebased onto |
Byron
left a comment
There was a problem hiding this comment.
Thanks a lot, it's so much better!
If you don't mind, in the very sense of the words, could you split the commits up so only gix-commitgraph gets the feat: … ? Everything could probably be Adapt to changes… then.
Otherwise, and I'd understand, please feel free to bring this PR out of draft mode and I will edit some commits.
Thanks again (I really want to merge this one, has been going on for too long 😅, as becomes obvious by the website sluggishness).
b05d9a7 to
c8bef8d
Compare
sha256 feature to gix-commitgraph|
I’ve now split the PRs focusing on the addition of |
For now, there is no direct way to set it, but that may change once we have tests that need it. Co-authored-by: Sebastian Thiel <sebastian.thiel@icloud.com>
c8bef8d to
b37c3a9
Compare
b37c3a9 to
12e6b7b
Compare
12e6b7b to
8ccd760
Compare
| env GIX_TEST_FIXTURE_HASH=sha1 cargo nextest run -p gix-commitgraph --no-fail-fast | ||
| env GIX_TEST_FIXTURE_HASH=sha256 cargo nextest run -p gix-commitgraph --no-fail-fast |
There was a problem hiding this comment.
@cruessler I have removed the other usages of gix-commitgraph as they just seemed to set the hash to SHA1 which doesn't seem necessary or effective anymore.
There was a problem hiding this comment.
Thanks a lot! Overall this took quite a while to review and I have changed gix-testtools quite significantly to make it non-breaking.
It would probably be advisable to build up a guide that helps AI to formalise this conversion (similar to what's happening in #2423 .
Thanks a lot for the review as well as for the patience! Out of curiosity: was there anything I could have done that would have made reviewing or working with this PR in general easier?
I’ll have a look! (Or are you already on it?) Are there any examples I could follow? |
Thanks for asking. Unfortunately I have no answer except for a learning for myself: If it's something 'fundamental', I should just dabble in it myself and get the basics going, and of course, also get an understanding myself first. This PR had a lot of rounds, and I think there are diminishing returns if it's not merged fast enough.
I am not on it, but have an example in |
This PR adds the
sha256feature togix-commitgraph. I opened it as a draft, so we have a space where we can discuss how to best approach testing this new feature.This PR was triggered by a comment in #2359.