Skip to content

feat: add HTTP header invocation and retrieval specs#139

Merged
alanshaw merged 7 commits intomainfrom
ash/feat/add-http-header-inv-and-retrieval-specs
Sep 12, 2025
Merged

feat: add HTTP header invocation and retrieval specs#139
alanshaw merged 7 commits intomainfrom
ash/feat/add-http-header-inv-and-retrieval-specs

Conversation

@alanshaw
Copy link
Copy Markdown
Member

@alanshaw alanshaw commented Aug 19, 2025

Adds specs for UCAN invocations over HTTP headers and the related retrieval spec that uses it.

The UCAN invocations over HTTP headers spec is based on storacha/RFC#60

🎬 Implementation Demo of UCAN invocations over HTTP headers

📚 Preview UCAN invocations over HTTP headers spec

📚 Preview Retrieval spec

resolves storacha/project-tracking#504

@alanshaw alanshaw requested a review from a team August 19, 2025 11:00
Copy link
Copy Markdown
Contributor

@volmedo volmedo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL at some comments.

Regarding egress tracking, I think all the data we need is included in the original invocation. The only contentious point is the byte range, if it can be different from the one requested (see https://github.com/storacha/specs/pull/139/files#r2288087511). Other than that, I'll write an RFC to discuss the egress tracker implementation, but it looks like we are covered for now.

Comment thread http-header-ucan-invocation.md Outdated
Comment thread http-header-ucan-invocation.md
Comment thread http-header-ucan-invocation.md Outdated
Comment thread w3-retrieval.md
Comment thread w3-retrieval.md
@volmedo
Copy link
Copy Markdown
Contributor

volmedo commented Aug 21, 2025

giving the retrieval spec a second look while working in the egress tracking spec, I think we might need to add a nonce field to space/content/retrieve invocations. The goal of this is allowing a storage node to get paid for egress in the case a client requests exactly the same content more than once for some reason.

Co-authored-by: Vicente Olmedo <vicente@storacha.network>
Copy link
Copy Markdown
Member

@frrist frrist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I've left a couple of questions, but I suspect we can tease them out during implementation.

Comment thread http-header-ucan-invocation.md Outdated
Comment thread http-header-ucan-invocation.md Outdated
Comment thread http-header-ucan-invocation.md
Comment thread http-header-ucan-invocation.md
Comment thread w3-retrieval.md
@alanshaw
Copy link
Copy Markdown
Member Author

alanshaw commented Sep 9, 2025

giving the retrieval spec a second look while working in the egress tracking spec, I think we might need to add a nonce field to space/content/retrieve invocations. The goal of this is allowing a storage node to get paid for egress in the case a client requests exactly the same content more than once for some reason.

UCAN has nonce already!

@volmedo
Copy link
Copy Markdown
Contributor

volmedo commented Sep 9, 2025

UCAN has nonce already!

I know, but the spec says they are optional, so I assumed their inclusion is something that must be specified on a per-use-case basis. Are they always present in the implementations we use?

@alanshaw
Copy link
Copy Markdown
Member Author

Are they always present in the implementations we use?

No you use them as you need them. I'll add words to the spec...

Copy link
Copy Markdown
Contributor

@volmedo volmedo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

I left one more, non-blocking question.

Comment thread w3-retrieval.md
"digest": { "/": { "bytes": "mEi...sfKg" } }
},
// byte range to extract from the blob - start and end byte (both inclusive)
"range": [2097152, 2098788]
Copy link
Copy Markdown
Contributor

@volmedo volmedo Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one more question, sorry: can range be non-present to signal we want to retrieve the full thing?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case we'd have to include the range in the receipt...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to confirm my understanding, is it so so that we can then know the size in bytes when accounting for egress?

Anyway, I'm ok with having to set "range": [0, (size-1)]. Does the client always know the size of a blob it wants to retrieve?

Copy link
Copy Markdown
Member Author

@alanshaw alanshaw Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funny you should say that. storacha/piri#229

In the case of upload data you do because you retrieve bytes based off of the data in the index. However you make a request for the full index first, which you only know the range of if the location commitment includes that information.

Comment thread w3-retrieval.md Outdated
@alanshaw
Copy link
Copy Markdown
Member Author

alanshaw commented Sep 11, 2025

@volmedo @frrist please take a look at the latest tweaks. Hopefully they incorporate all the needed feedback.

Amongst changes based on your feedback:

  • I have clarified what happens if the server receives a request with an invocation it has already executed (return 422 status), this implies that each invocation needs to have a nonce or a different expiry in order to actually get data.
  • I've also clarified what happens when you request a range outside of the blob size limits.

6f24f63

@alanshaw alanshaw merged commit bb0292d into main Sep 12, 2025
2 checks passed
@alanshaw alanshaw deleted the ash/feat/add-http-header-inv-and-retrieval-specs branch September 12, 2025 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create a spec for space/content/retrieve

3 participants