Funding a program: how to meet the min relay fee?

[egasimus]

Hi! As just discussed with Seth on office hours, here's an error when trying to broadcast a simple transaction.

400 (sendrawtransaction RPC error -26:
min relay fee not met, 0 < 41)

I'm not even evaluating any Simplicity in this transaction; I'm just trying to fund a program's P2TR from a P2WPKH. That's right, it's just a simple send transaction. And while I'm already aware that the min relay fee value (41) varies with network conditions; I can't for the life of me find the field or method which allows me to set the relay fee correctly on my end.

Here's the signed TX hex which reproduces the issue:

02000000010393ac8c206c71298ee9b564b9852d5396772f54e0d92f527b6b1c80a18b2c2ce90000000000ffffffffa229f4e61f48a23f21c77eb64479c851f7bb9c3ab7c0cb818d6d5989bca2e3790000000000ffffffff2815844c712c690ac08fecc32d0eff0028296b1eaf508f5c31fe26c561df15490000000000ffffffff0301a5502895799e276b4af246c821423b4ed5ec5e6b4e6df7a861606939d9a2fc3801000000000000006400225120a8227c2d878855cc3054775396e86df5c8028a714bf0511bce289ece2db5306f01a5502895799e276b4af246c821423b4ed5ec5e6b4e6df7a861606939d9a2fc38010000000000000b540016001479b000887626b294a914501a4cd226b58b23598301a5502895799e276b4af246c821423b4ed5ec5e6b4e6df7a861606939d9a2fc38010000000000002ee0000000000000000002473044022022febaab53c74861458ed05be880cd38c03f67227617454306f7affe1644e2f2022052bc73915fc1fddf758f49b214f9d3e82a2b5a67106b21171f7e8b9100584e3a0121031b84c5567b126440995d3ed5aaba0565d71e1834604819ff9c17f5e9d5dd078f00000002483045022100a5dfbd27860aa4daae6bbdab797cebc0b9a206205f4d65e19dca2a502a57b21b022053ec8038c0a0771478445ee15f6b6c35394d8f8f5cb36dfe639400bf5c2ee3690121031b84c5567b126440995d3ed5aaba0565d71e1834604819ff9c17f5e9d5dd078f0000000247304402207f0334409b149ecdff8a168c7a749d423b6d2067dd8a1673aed4806f02353ffc0220426f1441b4b520fa567e1f0ef8b63ba1883c950d511e637f6d8a203975968b890121031b84c5567b126440995d3ed5aaba0565d71e1834604819ff9c17f5e9d5dd078f00000000000000

Decoded ins and outs here (would be funny if what I'm missing is in the globals section, which is not currently included there, but I'm gonna look at it next)

[apoelstra]

The minimum relay fee is enforced not only by your node, but by other nodes on the network. Even if you configure your node not to enforce a minimum relay fee, the transaction won't propagate.

[egasimus]

going to fly the -v flag over here if that's cool. some things cannot be spoken of without adverbs. in any case, more discoursin' on SimplicityHL means more for the crawlers to index!

Not approaching this from the perspective of running a testnet node at all. Definitely would like to run one -- will likely need to -- at the same time I'd also like the users of my code to not have to.

So that's one thing I hoped to have abstracted away somewhat: the only testnet I was running is elementsregtest. Where, as you'd guess, min relay fee is indeed 0, and there is "no issue" -- until it bites on staging/actual testnet.

(UPDATE: my test suite now targets liquidtestnet as well; check it, it's succinct: https://github.com/hackbg/simf/blob/dev/src/test.ts)

What I see as someone who's not solved this (admittedly stupefyingly trivial) error before:

It would seem the 0 in the error message is not determined by the value of the fee output, as I surmised (or I'm providing the output wrong). I was hoping it'd turn out to be some other field or method of https://docs.rs/elements/latest/elements/pset/struct.Input.html or
https://docs.rs/elements/latest/elements/pset/struct.Output.html which I don't know about because has not floated out at me yet.

those PSETs, man...

FWIW, I do feel the original PSBT design does not follow principle of least astonishment, nor gestalt principle, but kitchen sink principle; meaning that the order in which I become aware of which PSBT/PSET feature I need to use when, remains all too nonlinear 🫠 While I fully agree with what Seth said about designs that go into the consensus mechanism -- that those are best considered immutable -- I think the pain points here are well worth pointing out. More generously, the difficulty of safely representing partially-constructed things in C/C++ has definitely informed the PST design, which is one of the core pieces adjacent to SimplicityHL -- hence the abstract "spooky action at a distance"...

Anyway what I was about to do next was grep through ElementsProject/elements, manually trace it from error site. Crude af, but reliably helped with the last ~5 not entirely unmisleading¹ error messages from the node, that I dun did battle with while trying to even see if I've constructed all correctly (thence a lot of the DX endeavor); but more on that in some next comment

Footnotes

1. Ran again into one of those I'm referring to, "make sure the transaction has at least one input", yeah there is, what about it is wrong though. Underlying CheckThis and VerifyThat C functions return booleans, which totally makes me want to check/verify that nothing will break by giving them some error codes... Some more looking at the node's source and I'm as likely as not to start pestering them with PRs as well. Soon as I enroll my GPG key 😅 ↩

[delta1]

trying to fund a program's P2TR from a P2WPK

Which wallet are you sending from? How are you creating the transaction?

I can't for the life of me find the field or method which allows me to set the relay fee correctly

If you can provide the above information about the sending wallet / transaction construction then I can try help

[stringhandler]

It was on this platform I think: https://github.com/hackbg/fadroma

[egasimus]

Enjoy, from wallet to signed hex: https://github.com/hackbg/simf/blob/dev/src/lib.rs#L129-L322

[egasimus]

@delta1 To save you the time of walking me through the entire TX construction (I just saw a thread where such an effort happened... and then it turned out to be React Native silently borking all BigInts) I'll try to paraphrase the key question here. Given the following message:

min relay fee not met, 0 < 41

I'm not asking what determines the 41 (which, as @apoelstra clarified, is not determined by the transaction construction) - but asking what exactly determines the 0 (which is where I'm messing it up, I would guess).

[egasimus]

EDIT: Well there's a thing to thank GH for - supporting the <details> tag. Collapsed long sections. A decoded form of the original TX is in that first one

Long decoding of transaction data, but it's from `std::fmt::Debug` and looks a bit unwieldy to me. The more screen area you give it, the easier it's to make sense of it I guess.

input: Input {
    non_witness_utxo: None,
    witness_utxo: Some(
        TxOut {
            asset: Explicit(
                0x38fca2d939696061a8f76d4e6b5eecd54e3b4221c846f24a6b279e79952850a5,
            ),
            value: Explicit(
                5000,
            ),
            nonce: Null,
            script_pubkey: Script(OP_0 OP_PUSHBYTES_20 79b000887626b294a914501a4cd226b58b235983),
            witness: TxOutWitness {
                surjection_proof: None,
                rangeproof: None,
            },
        },
    ),
    partial_sigs: {},
    sighash_type: None,
    redeem_script: None,
    witness_script: None,
    bip32_derivation: {},
    final_script_sig: Some(
        Script(),
    ),
    final_script_witness: Some(
        [
            [
                48,
                68,
                2,
                32,
                117,
                232,
                112,
                202,
                176,
                79,
                80,
                165,
                35,
                220,
                5,
                158,
                218,
                100,
                104,
                181,
                255,
                196,
                229,
                146,
                125,
                214,
                159,
                137,
                135,
                98,
                168,
                123,
                186,
                80,
                210,
                154,
                2,
                32,
                49,
                44,
                217,
                107,
                42,
                12,
                110,
                126,
                242,
                224,
                200,
                130,
                93,
                232,
                130,
                202,
                84,
                124,
                42,
                228,
                34,
                99,
                59,
                12,
                166,
                147,
                81,
                246,
                198,
                254,
                155,
                24,
                1,
            ],
            [
                3,
                27,
                132,
                197,
                86,
                123,
                18,
                100,
                64,
                153,
                93,
                62,
                213,
                170,
                186,
                5,
                101,
                215,
                30,
                24,
                52,
                96,
                72,
                25,
                255,
                156,
                23,
                245,
                233,
                213,
                221,
                7,
                143,
            ],
        ],
    ),
    ripemd160_preimages: {},
    sha256_preimages: {},
    hash160_preimages: {},
    hash256_preimages: {},
    previous_txid: 4915df61c526fe315c8f50af1e6b292800ff0e2dc3ec8fc00a692c714c841528,
    previous_output_index: 0,
    sequence: Some(
        Sequence(
            4294967295,
        ),
    ),
    required_time_locktime: None,
    required_height_locktime: None,
    tap_key_sig: None,
    tap_script_sigs: {},
    tap_scripts: {},
    tap_key_origins: {},
    tap_internal_key: None,
    tap_merkle_root: None,
    issuance_value_amount: None,
    issuance_value_comm: None,
    issuance_value_rangeproof: None,
    issuance_keys_rangeproof: None,
    pegin_tx: None,
    pegin_txout_proof: None,
    pegin_genesis_hash: None,
    pegin_claim_script: None,
    pegin_value: None,
    pegin_witness: None,
    issuance_inflation_keys: None,
    issuance_inflation_keys_comm: None,
    issuance_blinding_nonce: None,
    issuance_asset_entropy: None,
    in_utxo_rangeproof: None,
    in_issuance_blind_value_proof: None,
    in_issuance_blind_inflation_keys_proof: None,
    amount: None,
    blind_value_proof: None,
    asset: None,
    blind_asset_proof: None,
    blinded_issuance: None,
    proprietary: {},
    unknown: {},
}

input: Input {
    non_witness_utxo: None,
    witness_utxo: Some(
        TxOut {
            asset: Explicit(
                0x38fca2d939696061a8f76d4e6b5eecd54e3b4221c846f24a6b279e79952850a5,
            ),
            value: Explicit(
                5000,
            ),
            nonce: Null,
            script_pubkey: Script(OP_0 OP_PUSHBYTES_20 79b000887626b294a914501a4cd226b58b235983),
            witness: TxOutWitness {
                surjection_proof: None,
                rangeproof: None,
            },
        },
    ),
    partial_sigs: {},
    sighash_type: None,
    redeem_script: None,
    witness_script: None,
    bip32_derivation: {},
    final_script_sig: Some(
        Script(),
    ),
    final_script_witness: Some(
        [
            [
                48,
                69,
                2,
                33,
                0,
                168,
                104,
                24,
                199,
                30,
                157,
                3,
                54,
                107,
                229,
                117,
                21,
                6,
                140,
                225,
                10,
                57,
                57,
                45,
                221,
                132,
                168,
                82,
                103,
                186,
                221,
                236,
                164,
                34,
                161,
                107,
                164,
                2,
                32,
                36,
                79,
                50,
                246,
                32,
                234,
                201,
                199,
                190,
                5,
                241,
                169,
                98,
                229,
                90,
                24,
                240,
                28,
                55,
                46,
                74,
                65,
                239,
                88,
                17,
                235,
                16,
                183,
                147,
                248,
                16,
                163,
                1,
            ],
            [
                3,
                27,
                132,
                197,
                86,
                123,
                18,
                100,
                64,
                153,
                93,
                62,
                213,
                170,
                186,
                5,
                101,
                215,
                30,
                24,
                52,
                96,
                72,
                25,
                255,
                156,
                23,
                245,
                233,
                213,
                221,
                7,
                143,
            ],
        ],
    ),
    ripemd160_preimages: {},
    sha256_preimages: {},
    hash160_preimages: {},
    hash256_preimages: {},
    previous_txid: 79e3a2bc89596d8d81cbc0b73a9cbbf751c87944b67ec7213fa2481fe6f429a2,
    previous_output_index: 0,
    sequence: Some(
        Sequence(
            4294967295,
        ),
    ),
    required_time_locktime: None,
    required_height_locktime: None,
    tap_key_sig: None,
    tap_script_sigs: {},
    tap_scripts: {},
    tap_key_origins: {},
    tap_internal_key: None,
    tap_merkle_root: None,
    issuance_value_amount: None,
    issuance_value_comm: None,
    issuance_value_rangeproof: None,
    issuance_keys_rangeproof: None,
    pegin_tx: None,
    pegin_txout_proof: None,
    pegin_genesis_hash: None,
    pegin_claim_script: None,
    pegin_value: None,
    pegin_witness: None,
    issuance_inflation_keys: None,
    issuance_inflation_keys_comm: None,
    issuance_blinding_nonce: None,
    issuance_asset_entropy: None,
    in_utxo_rangeproof: None,
    in_issuance_blind_value_proof: None,
    in_issuance_blind_inflation_keys_proof: None,
    amount: None,
    blind_value_proof: None,
    asset: None,
    blind_asset_proof: None,
    blinded_issuance: None,
    proprietary: {},
    unknown: {},
}

input: Input {
    non_witness_utxo: None,
    witness_utxo: Some(
        TxOut {
            asset: Explicit(
                0x38fca2d939696061a8f76d4e6b5eecd54e3b4221c846f24a6b279e79952850a5,
            ),
            value: Explicit(
                5000,
            ),
            nonce: Null,
            script_pubkey: Script(OP_0 OP_PUSHBYTES_20 79b000887626b294a914501a4cd226b58b235983),
            witness: TxOutWitness {
                surjection_proof: None,
                rangeproof: None,
            },
        },
    ),
    partial_sigs: {},
    sighash_type: None,
    redeem_script: None,
    witness_script: None,
    bip32_derivation: {},
    final_script_sig: Some(
        Script(),
    ),
    final_script_witness: Some(
        [
            [
                48,
                69,
                2,
                33,
                0,
                232,
                153,
                241,
                139,
                183,
                50,
                251,
                6,
                165,
                205,
                115,
                207,
                223,
                180,
                183,
                230,
                122,
                157,
                16,
                145,
                36,
                165,
                46,
                151,
                177,
                205,
                129,
                232,
                252,
                111,
                23,
                81,
                2,
                32,
                10,
                5,
                218,
                154,
                84,
                166,
                106,
                179,
                248,
                220,
                139,
                124,
                253,
                151,
                28,
                108,
                188,
                201,
                45,
                174,
                123,
                21,
                215,
                166,
                23,
                143,
                238,
                102,
                215,
                120,
                100,
                16,
                1,
            ],
            [
                3,
                27,
                132,
                197,
                86,
                123,
                18,
                100,
                64,
                153,
                93,
                62,
                213,
                170,
                186,
                5,
                101,
                215,
                30,
                24,
                52,
                96,
                72,
                25,
                255,
                156,
                23,
                245,
                233,
                213,
                221,
                7,
                143,
            ],
        ],
    ),
    ripemd160_preimages: {},
    sha256_preimages: {},
    hash160_preimages: {},
    hash256_preimages: {},
    previous_txid: e92c2c8ba1801c6b7b522fd9e0542f7796532d85b964b5e98e29716c208cac93,
    previous_output_index: 0,
    sequence: Some(
        Sequence(
            4294967295,
        ),
    ),
    required_time_locktime: None,
    required_height_locktime: None,
    tap_key_sig: None,
    tap_script_sigs: {},
    tap_scripts: {},
    tap_key_origins: {},
    tap_internal_key: None,
    tap_merkle_root: None,
    issuance_value_amount: None,
    issuance_value_comm: None,
    issuance_value_rangeproof: None,
    issuance_keys_rangeproof: None,
    pegin_tx: None,
    pegin_txout_proof: None,
    pegin_genesis_hash: None,
    pegin_claim_script: None,
    pegin_value: None,
    pegin_witness: None,
    issuance_inflation_keys: None,
    issuance_inflation_keys_comm: None,
    issuance_blinding_nonce: None,
    issuance_asset_entropy: None,
    in_utxo_rangeproof: None,
    in_issuance_blind_value_proof: None,
    in_issuance_blind_inflation_keys_proof: None,
    amount: None,
    blind_value_proof: None,
    asset: None,
    blind_asset_proof: None,
    blinded_issuance: None,
    proprietary: {},
    unknown: {},
}

output 0: Output {
    redeem_script: None,
    witness_script: None,
    bip32_derivation: {},
    tap_internal_key: None,
    tap_tree: None,
    tap_key_origins: {},
    amount: Some(
        100,
    ),
    amount_comm: None,
    script_pubkey: Script(OP_PUSHNUM_1 OP_PUSHBYTES_32 a8227c2d878855cc3054775396e86df5c8028a714bf0511bce289ece2db5306f),
    asset: Some(
        0x38fca2d939696061a8f76d4e6b5eecd54e3b4221c846f24a6b279e79952850a5,
    ),
    asset_comm: None,
    value_rangeproof: None,
    asset_surjection_proof: None,
    blinding_key: None,
    ecdh_pubkey: None,
    blinder_index: None,
    blind_value_proof: None,
    blind_asset_proof: None,
    proprietary: {},
    unknown: {},
}

output 1: Output {
    redeem_script: None,
    witness_script: None,
    bip32_derivation: {},
    tap_internal_key: None,
    tap_tree: None,
    tap_key_origins: {},
    amount: Some(
        2900,
    ),
    amount_comm: None,
    script_pubkey: Script(OP_0 OP_PUSHBYTES_20 79b000887626b294a914501a4cd226b58b235983),
    asset: Some(
        0x38fca2d939696061a8f76d4e6b5eecd54e3b4221c846f24a6b279e79952850a5,
    ),
    asset_comm: None,
    value_rangeproof: None,
    asset_surjection_proof: None,
    blinding_key: None,
    ecdh_pubkey: None,
    blinder_index: None,
    blind_value_proof: None,
    blind_asset_proof: None,
    proprietary: {},
    unknown: {},
}

output 2: Output {
    redeem_script: None,
    witness_script: None,
    bip32_derivation: {},
    tap_internal_key: None,
    tap_tree: None,
    tap_key_origins: {},
    amount: Some(
        12000,
    ),
    amount_comm: None,
    script_pubkey: Script(),
    asset: Some(
        0x38fca2d939696061a8f76d4e6b5eecd54e3b4221c846f24a6b279e79952850a5,
    ),
    asset_comm: None,
    value_rangeproof: None,
    asset_surjection_proof: None,
    blinding_key: None,
    ecdh_pubkey: None,
    blinder_index: None,
    blind_value_proof: None,
    blind_asset_proof: None,
    proprietary: {},
    unknown: {},
}

(What's even more awkward than a 64-column byte string? Same string as 34-line byte array 😸)

Also long but not as much subjective thoughts; mainly just me being frustrated at states of things. Potentially interesting to anyone looking for DX feedback, but non-critical to fixing the actual case.

On reflection, I would after all venture to emphasize something about transaction construction with the elements crate: I sorely lacked both guard rails to preclude misconstructions, and visiblity to debug them should they occur. How exactly someone's constructing their transaction still matters; IMO it shouldn't.

Means I get to solve problems with that as I go -- but, considering the scale of the effort, and that what I'm working on is situated on its "last mile" so to speak, I'm still not 100% sure about the best form in which to contribute back what I've gleaned from all my looking-into stuff. So, for now, just putting these out there:

• Guard rails. I'd be surprised if folks here were entirely unfamiliar with "make invalid states unrepresentable". In an ideal world, the structs in elements would not permit to be constructed invalidly in the first place. (A particular point in that design space is represented by e.g. @scure/btc-signer -- that hasn't worked for me on Liquid so far but the README includes some choice critique besides the whole broad range of transaction primitives.)

  • But with elements rightly off limits for breaking changes, and the misconstruction possibly determined by network parameters (as In the case at hand - which means the solution should be RPC and/or Esplora aware...) I can't help but think how something that could not only solve the general issue, but also be expressive while looking familiar to all, would be a Rust SDK based on method chaining/builder pattern. It'd wrap elements, elements-miniscript, simplicityhl, rust-simplicity, etc, into a unified method chain for producing, broadcasting, and inspecting a valid TX hex. (Personally, I'm curious how the method chaining could be made to work across the JS/WASM boundary - but that's a matter for more prototyping...)
  • But maybe something similar already exists? Or maybe just around the corner.

• Improving legibility of details. Admittedly I felt a pang of guilt for not providing the problematic transaction in its decoded form right off the bat (now included below in std::fmt::Debug style, sans globals). I was secretly hoping I'm the only one who is not seeing the 100ft tall animated glowing TXHEX decoder+debugger on https://blockstream.info/ - but I think it really doesn't exist (unlike the visualizer for transactions that did broadcast, which is pretty neat!)

[egasimus]

by way of which, also UPDATE:

The problem with inability to specify sufficient fee in a transaction for testnet to accept the broadcast first cropped up when building Fadroma IDE's SimplicityHL PSET tester.

But after some fiddling with the testing setup I've also added a test case for it in here https://github.com/hackbg/simf/blob/dev/src/test.ts - now it takes fewer scrolls and jabs to get to it in the first place!

Anyone feeling curious, please feel welcome to check that out, glance for quick obvious markers of me having gotten it all wrong, etc etc. (Watch for the test suite failing to terminate elementsd on exit and make sure to pkill that then rm -rf /tmp/fadroma if it fails to start the second time; that's another one on the backlog while I'm trying to get this thing to also pass oracle attestations around; more on that also soon™️)

[stringhandler]

I noticed that all of your outputs are asset 0x38fca2d939696061a8f76d4e6b5eecd54e3b4221c846f24a6b279e79952850a5. Are there not supposed to be some LBTC outputs to pay for the fee?

[delta1]

correct you need a fee output with the liquid testnet asset ID (144c654344aa716d6f3abcc1ca90e5641e4e2a7f633bc09fe3baf64585819a49), and for your transaction to balance you'll need an input of the same asset (and likely a change output because your input won't have the exact fee value)

[delta1]

obviously that's for liquidtestnet, if it's for elementsregtest you need the native asset as a fee output in your transaction

[egasimus]

Thanks for checking my work guys 🙇

I think the concrete issue boiled down to decimals handling. One of the inputs was getting unneccessarily multiplied by 10^8 🤦 That was from my earlier attempt to uniformly support both BigInt and the Numbers that the remote APIs give out.

• For whatever reason, though, it appeared with the correct value in the debug print I'd added; while just using the serde_json implementation from elements there printed Asset::Explicit in unreadable format (wrong endianness?) so I couldn't immediately catch that I was debugging wrong, too.

• Plus, some other moving part (maybe order of fee output in transaction?) was making the error message I'm getting from the node alternate between "min relay fee not met" and "inputs do not equal outputs" (both, strictly speaking, true - but giving me very little to go by)

• And differences between the RPC API of elementsregtest and the Esplora API likely played some role in my getting it all wrong. Is there public testnet RPC somewhere, or does everyone just use Esplora? (I see ad-hoc handles to it in half of the SimplicityHL projects I've reviewed so far so I'm guessing the later)

All adding up to a rather confusing puzzle, ultimately solved by poring over the transaction construction code some more until every step is as simple as it gets (for now). 😅

Before going back to my next few hours of similar activities, I wanted to share this sketch for a Liquid transaction builder API. Nothing here is final-final but I'd say this is rather close to what I've been going for:

• plenty of room for pre-broadcast validation at every step of construction,
• first-class support for TX inpuit from SimplicityHL+witness sigs
• construction of witness data is exposed to the user as a single js callback, abstracting away that little "construct-sighash-witness-reconstruct" redeem dance

import { SimplicityHL, LiquidTestnet, ElementsRegtest } from '@hackbg/fadroma';

// esplora client
const chain = LiquidTestnet();

// lwk_signer can plug here
const signer = await SimplicityHL.Signer(new Uint8Array(32)); 

// compile a program
const program = await SimplicityHL.Program("name", `
  fn main () { /* ... */ }
`, {
  chain: "liquidtestnet", // represents hrp, blech32, prefixes
  genesis: "...", // really needed to compile?
  args: { "ARG": "VAL" }
});

// commit to program
const commitTx = await SimplicityHL.Spend()
  .input(utxo, signer)
  .output(p2tr, commitAmount)
  .fee(feeAmount)
  .broadcast(chain);

// redeem from program
const redeemTx = await SimplicityHL.Spend()
  .input(utxo, program, witness)
  .output(address, redeemAmount)
  .fee(feeAmount)
  .broadcast(chain);

// provide witness signatures
function witness ({ sighash }) {
  return { "WIT": "..." }
}