Overriding the Elements jet set with the `--target` parameter

[ivanlele]

Project

compiler

Describe the feature

I propose adding a --target parameter to simc that specifies the jet set used for SimplicityHL compilation. Based on the passed argument, the compiler would load a different Jet trait implementation at runtime.

Such option would facilitate a development of projects which do not use standard jets. Practical exemple is SU.

The implementation can be carried out in a series of PRs:

• Create JetHL trait that defines the interfaces for all additional jet specific compilation-related computations introduced in Simplicity HL, and not available in on rust-simplicity Jet trait.
• Require ast::Program to accept a generic parameter for the implemented JetHL trait.
• Add a --target option that loads a dynamic library at runtime, whose methods would be used internally by a new CustomJetHL type.

This should be enough to make the minimal version work, although we might need to make additional changes for it to also work with codegen and LSP.

[apoelstra]

concept ACK. I think we will struggle to implement this though because so many of our types are parameterized by J: Jet and Rust makes it very hard to swap these at runtime.

[ivanlele]

concept ACK. I think we will struggle to implement this though because so many of our types are parameterized by J: Jet and Rust makes it very hard to swap these at runtime.

I think this might mainly be the case in rust-simplicity only. I searched for : Jet, Jet +, and + Jet, and I can count all the usages on one hand.

The main problem, I think, would be adding the required generic parameter to the SimplicityHL types, since almost everything currently depends on the Elements type.

And at the top level we can just do something like:

let program_bytes: Vec<u8> = match target {
    Some(target) => {
        CustomJet::load_from_dynamic_library(target)?;

        CompiledProgram::<CustomJet>::new(/* args */)
            .commit()
            .to_vec_without_witness()
    }
    None => {
        CompiledProgram::<Elements>::new(/* args */)
            .commit()
            .to_vec_without_witness()
    }
};

[ivanlele]

The main problem, I think, would be adding the required generic parameter to the SimplicityHL types, since almost everything currently depends on the Elements type.

I did a little experiment to see how it would turn out: ivanlele#1. It quickly turned into generics hell, one change in an enum somewhere deep in the type stack requires changing almost everything, or other stuff with similar spirit.

I was thinking about using a dummy jet implementation for logic branches that do not directly interact with Elements and are only affected because they are part of a type that depends on Elements elsewhere. But I dropped the idea because it felt inconsistent to mix generics with dummy implementation that only exists to fool generic checks.

I managed to isolated all logic that depends on the Elements jet into a new trait, and it turned out to be quite small.

Should I submit these changes as a upstream PR maybe? Or is ther a better approach that I haven't tought of?

[apoelstra]

@ivanlele sure, if you have a small trait then it's definitely worth PR'ing so we can discuss its merits.

What I would like to do is go upstream to rust-simplicity and attempt to remove the J: Jet parameter from the main Node type. It should instead live in types::Context (which should probably not live in types, but it's okay there for now), where it will prevent the user from mixing different jet types in a single program.

Then in Node, rather than having a generic J: Jet, we should have a Box<dyn Jet> (or maybe a struct Jet). However, the current Jet type isn't object-safe due to its associated types. So we will need to refactor the trait to invert the types, or something. Probably we should move this to a separate issue.

[apoelstra]

I opened BlockstreamResearch/rust-simplicity#349 but it's a bit rambly.