Version Incompatability

[Tom-Finke]

Downgrade tests with OpenFHE v0.1.14 currently fail, see #100.
This is because we force minimum version but do not restrict maximum versions and openfhe introduces breaking changes in v1.x -> v1.y minor version updates.

Implementation idea: add weak dependency¹ for openfhe-julia.jll and openfhe.jll, both with a required minimum and maximum version. Every release of openfhe-julia / openfhe would then entail a new release of OpenFHE.jl / SecureArithmetic.jl with the new maximum version number of upstream packages, either as a patch patch or minor (in the future major) version update, depending on if the upstream changes are breaking.

Footnotes

1. https://pkgdocs.julialang.org/v1/creating-packages/#Weak-dependencies ↩

[Tom-Finke]

This can (currently) not be reproduced in the downgrade tests for OpenFHE.jl, because the current OpenFHE.jl v0.2 needs at least openfhe_julia_jll v0.6, which is compatible with the currently most recent version of OpenFHE_jl.jl, v1.5.1.

If a new minor version of OpenFHE_jl.jll was released which was not backwards compatible, then I would expect downgrade tests in OpenFHE_jl.jll to fail as well, because it will use openfhe_julia_jll v0.6 with a new incompatible version of OpenFHE_jll.jl, say v1.6.

The binary builder docs specifically mention this problem by giving an example:

CFL releases version 1.1.1 and version 1.1.2 that are incompatible. A real world example is Boost (which breaks the ABI in every single release because they embed the full version number in the soname of libraries). If you have a typical permissive semver-style compat section in a package that depends on CFL_jll, then your package will break whenever CFL_jll gets a new release. To solve this issue you have to use "hyphen style" compat bounds like "0.9.0 - 1.1.2". This leads to a separate problem: you need to change the compat bound every time there is a new CFL_jll release: this is the least bad option though – it causes more annoyance for developers but it ensures users never end up with broken installs. And bots like CompatHelper can mostly automate that issue.

https://docs.binarybuilder.org/stable/FAQ/#Now-that-I-have-a-published-and-versioned-jll-package,-what-compat-bounds-do-I-put-in-its-dependents?-What-if-the-upstream-does-not-follow-SemVer?

I would therefore suggest to not use weak dependencies in OpenFHE.jl, but use the hyphen style compatability bounds¹ directly in openfhe_julia. This also makes sense in my opinion, because every release of OpenFHE_jll means we have to manually check if openfhe_julia is still compatible anyways. If it is compatible, we can simply adjust the upper compat bound and make a patch release. If it is incompatible, update the lower and upper compat bound and make a minor (or major release when arriving at v1.x). Thus, openfhe_julia is our "compatability layer", which enforces semantic versioning around OpenFHE_jll.jl. For OpenFHE.jl, we can use default CompatHelper / Dependabot to keep versions up to date, because OpenFHE.jl only depends on openfhe_julia, not OpenFHE_jll, and openfhe_julia follows semantic versioning. The same then applies to SecureArithmetic.jl, which in the future can use any compatible OpenFHE version following semantic versioning, which in turn can use any compatible openfhe_julia version following semantic versioning, which enforces a compatible OpenFHE_jll.jl version by manual hyphen compat bounds.

Footnotes

1. For an example of the hyphen syntax in action in an Yggdrasil package see https://github.com/JuliaPackaging/Yggdrasil/blob/7bbb59b999752223da267f8e1b637e8dc923488d/C/Compose/build_tarballs.jl#L69. ↩

[Tom-Finke]

@sloede what do you think?

[sloede]

Haha, I think you lost me here. Let's discuss this tomorrow!

[Tom-Finke]

Deps are set

• https://github.com/JuliaPackaging/Yggdrasil/blob/master/O/openfhe_julia/build_tarballs.jl#L14
• https://github.com/hpsc-lab/OpenFHE.jl/blob/main/Project.toml#L18
• https://github.com/hpsc-lab/SecureArithmetic.jl/blob/main/Project.toml#L10

[Tom-Finke]

Problem with setting the upper bounds in Yggdrasil: we would also change the patch number. But that would mean having to change the patch number in openfhe-julia's VERSION file as well, without an actual code change.

Yggdrasil offers way of triggering a new build without changing the version number.

Open question: have breaking changes in openfhe-development happened in patch versions? If not, we could use the tilde modifier to fix openfhe-development to 1.5.x instead of explicit hyphen syntax.

If this option works: document it in openfhe-julia and / or OpenFHE.jl