Polkadot update v0.9.42 by OverOrion · Pull Request #1337 · integritee-network/worker

OverOrion · 2023-05-31T15:42:18Z

Required PRs to be merged / issues to be solved first:

Closes #1303.
Closes #1379.

…default async

clangenb

Looks actually good to me, I don't know how the CI error can occur yet.

cli/src/base_cli/mod.rs

cli/src/benchmark/mod.rs

core-primitives/extrinsics-factory/Cargo.toml

app-libs/sgx-runtime/src/lib.rs

brenzi · 2023-07-18T06:43:36Z

I see these errors in the log, seem to be new:
offchain-worker-demo-indirect-invocation-demo-indirect-invocation-1 | [2023-07-17T15:05:54Z WARN substrate_api_client::rpc::ws_client] Expected subscription, but received an id response instead: Object {"id": String("1"), "jsonrpc": String("2.0"), "result": String("r1pAt7znUzbHt4Lp")}

see M6 offchain test

brenzi · 2023-07-19T14:21:57Z

started testing manually: locally with SW mode and skip-ra: https://hackmd.io/@brenzi5/HJ8Y2Sr5n

teeracle works as expected
sidechain single validateer
sidechain 2 validateers

brenzi · 2023-07-19T16:49:57Z

seems like our integritee-node is notcompatible with rpc behavior of api-client polkadot-0.9.42-tag-1.10
scs/substrate-api-client#620 (comment)

therefore, integritee-service thinks that extrinsics fail when they don't and block hashes are wrong (wrong hash is fixed with next commit)

not sure if this could help:
#1266

brenzi · 2023-07-20T07:40:05Z

service/src/main.rs

+		enclave_count_of_previous_block,
+		register_enclave_xt_header.parent_hash()
+	);
 	Ok(enclave_count_of_previous_block == 0)


I really don't get the logic of this. It's here for a while already: why so complicated?
why check that enclave count was zero in the previous block when we can check that it is 1 in the current block?
I didn't touch it beyond logging because we'll kick that out anyway and replace it with per-shard ShardStatus logic

I have no idea actually.

brenzi · 2023-07-20T16:02:44Z

I reproduced sending a transfer extrinsic with an api-client example and until_success works as expected (apart from the warning we observe as well.)

the successful jsnrpc log looks like this:

[2023-07-20T15:03:15Z INFO  substrate_api_client::rpc::ws_client] sending request: {"id":"1","jsonrpc":"2.0","method":"author_submitAndWatchExtrinsic","params":["0x3d028400d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d01421a7e29f0b5f430034d0e909a9e8a8073fcfb71888f721065e8832034259245b488cdd2adc6ae538a2f13a82f49e8a3208e549a9d3bf1734c013065be683a86001c000a00008eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48070010a5d4e8"]}
[2023-07-20T15:03:15Z INFO  substrate_api_client::rpc::ws_client] got on_subscription_msg {"jsonrpc":"2.0","result":"6soqlODDXRt58lf7","id":"1"}
[2023-07-20T15:03:15Z WARN  substrate_api_client::rpc::ws_client] Expected subscription, but received an id response instead: Object {"id": String("1"), "jsonrpc": String("2.0"), "result": String("6soqlODDXRt58lf7")}
[2023-07-20T15:03:15Z INFO  substrate_api_client::rpc::ws_client] got on_subscription_msg {"jsonrpc":"2.0","method":"author_extrinsicUpdate","params":{"subscription":"6soqlODDXRt58lf7","result":"ready"}}
[2023-07-20T15:03:24Z INFO  substrate_api_client::rpc::ws_client] got on_subscription_msg {"jsonrpc":"2.0","method":"author_extrinsicUpdate","params":{"subscription":"6soqlODDXRt58lf7","result":{"inBlock":"0xe8a977d95d199d8ed87fc7b14c41e05fb1faaaee2ba9feb0a324796a039dc90e"}}}
[2023-07-20T15:03:48Z INFO  substrate_api_client::rpc::ws_client] got on_subscription_msg {"jsonrpc":"2.0","method":"author_extrinsicUpdate","params":{"subscription":"6soqlODDXRt58lf7","result":{"finalized":"0xe8a977d95d199d8ed87fc7b14c41e05fb1faaaee2ba9feb0a324796a039dc90e"}}}
[2023-07-20T15:03:48Z INFO  ws] Queuing connection to ws://127.0.0.1:9944/
[2023-07-20T15:03:48Z INFO  substrate_api_client::rpc::ws_client] sending request: {"id":"1","jsonrpc":"2.0","method":"chain_getBlock","params":["0xe8a977d95d199d8ed87fc7b14c41e05fb1faaaee2ba9feb0a324796a039dc90e"]}
[2023-07-20T15:03:48Z INFO  substrate_api_client::rpc::ws_client] Got get_request_msg {"jsonrpc":"2.0","result":{"block":{"header":{"parentHash":"0x5d7003f5fe62ec7ba76bfd5f7bbb8b4824a72e760dd47ac0639b4a998e21f89f","number":"0x2f8","stateRoot":"0x1021a40886c6514173d2776d64c4c9d313ed59e69cecf3262a4aae133aff8730","extrinsicsRoot":"0x2d337d19d5ef693f5954aedca5b586e2344db208e6e0ba832f03e153fc59532a","digest":{"logs":["0x06617572612065c6640800000000","0x056175726101010cbfeda0e25e143b808ea3014fe117799e53dd7af22c0034f92a611ec4404d526e776617e7cec845b4ca2833e4bd86e3d6c84fd48db86b3ed17af29f9c3dc081"]}},"extrinsics":["0x280403000b61bed3738901","0x3d028400d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d01421a7e29f0b5f430034d0e909a9e8a8073fcfb71888f721065e8832034259245b488cdd2adc6ae538a2f13a82f49e8a3208e549a9d3bf1734c013065be683a86001c000a00008eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48070010a5d4e8"]},"justifications":null},"id":"1"}
[2023-07-20T15:03:49Z INFO  ws] Queuing connection to ws://127.0.0.1:9944/
[2023-07-20T15:03:49Z INFO  substrate_api_client::rpc::ws_client] sending request: {"id":"1","jsonrpc":"2.0","method":"state_getStorage","params":["0x26aa394eea5630e07c48ae0c9558cef780d41e5e16056765bc8461851072c9d7","0xe8a977d95d199d8ed87fc7b14c41e05fb1faaaee2ba9feb0a324796a039dc90e"]}
[2023-07-20T15:03:49Z INFO  substrate_api_client::rpc::ws_client] Got get_request_msg {"jsonrpc":"2.0","result":"0x1c00000000000000e27c01510002000000010000000a08d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27db0806a0f0000000000000000000000000000010000000a02d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d8eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a480010a5d4e800000000000000000000000000010000000a076d6f646c70792f747273727900000000000000000000000000000000000000000c7727000000000000000000000000000000010000001e060c7727000000000000000000000000000000010000000b00d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27db0806a0f00000000000000000000000000000000000000000000000000000000000001000000000082aef55800000000","id":"1"}
[+] Extrinsic got included. block hash: Some(0xe8a977d95d199d8ed87fc7b14c41e05fb1faaaee2ba9feb0a324796a039dc90e)
[+] Extrinsic got included. extrinsic hash: 0xb0117124699f03cd63eeb69fed470fe589803054d4523d169e28015e2ad338c9
[+] Extrinsic got included. status: Finalized(0xe8a977d95d199d8ed87fc7b14c41e05fb1faaaee2ba9feb0a324796a039dc90e)

I will now gather the same type of log for the failing case here

brenzi · 2023-07-20T17:14:35Z

ok. our issue was scs/substrate-api-client#624
workaround is to not verify extrinsic success in main thread
let's see if there are others....

…ection

brenzi · 2023-07-20T19:48:15Z

back to square one:

first validateer produces blocks
second validateer knows he's not primary and tries to sync. but won't accept blocks
sidechain.confirmImportedSidechainBlock is never called
sidechainWorkerForShard stays empty (local testing)

[2023-07-20T19:15:40Z INFO  its_consensus_common::peer_block_sync] Fetched 119 blocks from peer to import
[2023-07-20T19:15:40Z DEBUG its_consensus_common::block_import] Attempting to import sidechain block (number: 24, hash: 0xea21f835a72b5e369db3e50ddd8a23fa4889e6e5aa63ec3c1fab99098eda8e34, parentchain hash: 0x1074ecab5fd26c2bf223110a86012709262f60c67823b344ba26aeb51f1008ab)
[2023-07-20T19:15:41Z INFO  its_consensus_common::block_import] Applying state diff for block 24 of size 219 bytes
[2023-07-20T19:15:41Z ERROR its_consensus_common::peer_block_sync] Failed to import sidechain block that was fetched from peer: Other(InvalidAprioriHash)
[2023-07-20T19:15:41Z ERROR enclave_runtime::error] Returning error Consensus(Other(InvalidAprioriHash)) as sgx unexpected.
[2023-07-20T19:15:41Z ERROR integritee_service::sidechain_setup] Sgx(SGX_ERROR_UNEXPECTED)

brenzi · 2023-07-21T07:14:00Z

Hmm, very similar to #1366 which shall be fixed by #1370.

clangenb

Cool, I have revisited the PR, and I think it is ready when the CI says so! 🥳

cli/src/base_cli/commands/transfer.rs

cli/src/trusted_operation.rs

core-primitives/attestation-handler/src/cert.rs

docker/entrypoint.sh

clangenb · 2023-08-01T09:27:41Z

service/src/main.rs

+		enclave_count_of_previous_block,
+		register_enclave_xt_header.parent_hash()
+	);
 	Ok(enclave_count_of_previous_block == 0)


I have no idea actually.

clangenb · 2023-08-01T10:21:49Z

I will merge it, as the instability has been noted before and does not have to do with this PR, see #1403

OverOrion force-pushed the szp/polkadot-v0.9.42 branch 3 times, most recently from 7efc53d to 598e7d8 Compare June 2, 2023 12:36

OverOrion force-pushed the szp/polkadot-v0.9.42 branch from c64f36f to e51081d Compare June 8, 2023 12:27

OverOrion added A0-core Affects a core part B0-silent Item should not be mentioned in any change logs. (e.g. documentation updates) C1-low 📌 Does not elevate a release containing this beyond "low priority" E0-breaksnothing labels Jun 8, 2023

OverOrion marked this pull request as ready for review June 8, 2023 13:54

OverOrion changed the title ~~[WIP] Polkadot update v0.9.42~~ Polkadot update v0.9.42 Jun 8, 2023

OverOrion force-pushed the szp/polkadot-v0.9.42 branch from 0cde9df to 6c81611 Compare June 12, 2023 15:37

OverOrion added 16 commits June 13, 2023 09:22

polkadot update v0.9.42

606b993

followup for Keystore overhaul (final) (#13683)

6e8a291

implement missing trait methods

6f0aade

substrate-api-client: explicitly use synchronous mode instead of the …

a81fda6

…default async

susbtrate-api-client followup regarding keystore overhaul

e2167e2

taplo fmt

fece3a0

followup for sidechain-evm

6170c50

cargo clippy

2dc6a86

fixup! followup for sidechain-evm

9d483f0

update integritee-node image

b02f96a

integritee-node: update branch

34b1320

integritee-node: update branch

8bceae9

main: AddedEnclave change followup made in pallets/pull/180

68f6718

attesteer: fix warnings where DCAP was activated, but not with attesteer

e7c29c1

evm: followup made in pallets/pull/180

eea5d22

add debug! logs

bf1aee8

OverOrion force-pushed the szp/polkadot-v0.9.42 branch from 6c81611 to bf1aee8 Compare June 13, 2023 07:24

clangenb reviewed Jun 13, 2023

View reviewed changes

cli/src/base_cli/mod.rs Outdated Show resolved Hide resolved

cli/src/benchmark/mod.rs Outdated Show resolved Hide resolved

core-primitives/extrinsics-factory/Cargo.toml Outdated Show resolved Hide resolved

app-libs/sgx-runtime/src/lib.rs Outdated Show resolved Hide resolved

remove commented out code

47c2a91

BillyWooo mentioned this pull request Jul 8, 2023

Polkadot 0.9.43 cli breaking changes litentry/heima#1774

Closed

fix clippy

3f8df59

brenzi mentioned this pull request Jul 13, 2023

integrate pallet teerex refactoring #1390

Merged

brenzi and others added 3 commits July 17, 2023 09:59

Merge branch 'master' into szp/polkadot-v0.9.42

978fec6

Merge remote-tracking branch 'origin/master' into szp/polkadot-v0.9.42

183caca

hopefully fixcing evm build

8b3dd11

brenzi and others added 4 commits July 18, 2023 11:58

add debug log in entrypoint

9751046

use explicit sgx_default_qcnl.conf file

258f489

CI: healthy params for teeracle testing

ead3b93

more verbose logging for sidechain consensus

0494ffb

pimp logging

de2dda2

ensure worker panics immediately if registering enclave fails

f9dac57

brenzi reviewed Jul 20, 2023

View reviewed changes

brenzi added 2 commits July 20, 2023 19:55

bypass enclave registration success check. fix primary validateer det…

af613d3

…ection

fmt

a30ba5d

pimp logging to track shard

e287fda

clangenb mentioned this pull request Jul 31, 2023

Provision light-client in mutual remote attestation #1399

Merged

Merge branch 'master' into szp/polkadot-v0.9.42

ce03d1c

clangenb approved these changes Aug 1, 2023

View reviewed changes

fmt

5140a1a

clangenb merged commit d1b8cbe into master Aug 1, 2023

clangenb deleted the szp/polkadot-v0.9.42 branch August 14, 2023 14:49

Conversation

OverOrion commented May 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clangenb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brenzi commented Jul 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brenzi commented Jul 19, 2023

Uh oh!

brenzi commented Jul 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brenzi Jul 20, 2023

Choose a reason for hiding this comment

Uh oh!

clangenb Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

brenzi commented Jul 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brenzi commented Jul 20, 2023

Uh oh!

brenzi commented Jul 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brenzi commented Jul 21, 2023

Uh oh!

clangenb left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clangenb Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

clangenb commented Aug 1, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

OverOrion commented May 31, 2023 •

edited

Loading

brenzi commented Jul 18, 2023 •

edited

Loading

brenzi commented Jul 19, 2023 •

edited

Loading

brenzi commented Jul 20, 2023 •

edited

Loading

brenzi commented Jul 20, 2023 •

edited

Loading

clangenb left a comment •

edited

Loading