feat(hub): secret intake links — secure secret submission from chat#318
feat(hub): secret intake links — secure secret submission from chat#318zeroasterisk wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request implements Secret Intake Links, enabling secure secret submission from chat channels via short-lived, one-time-use URLs. It adds a CLI command, backend handlers, an in-memory SecretIntakeService, and a standalone frontend page for anonymous submission. The review feedback identifies several critical issues: a concurrency data race on SecretIntake fields between handlers and the background cleanup loop; a client-side JWT decoding bug due to missing base64 padding and lack of UTF-8 support; a potential memory-exhaustion DoS vulnerability from a missing size limit on submitted values; a validation gap where scope_id is not verified for project-scoped secrets; loose path matching in the unauthenticated endpoint check; and a bug where trimming whitespace from submitted secrets can corrupt keys with significant whitespace.
| // GetIntake returns an intake by ID, or nil if not found. | ||
| func (s *SecretIntakeService) GetIntake(id string) *SecretIntake { | ||
| s.mu.Lock() | ||
| defer s.mu.Unlock() | ||
| return s.intakes[id] | ||
| } |
There was a problem hiding this comment.
Concurrency Bug: Data Race on SecretIntake Fields
Returning a direct pointer to SecretIntake via GetIntake and then reading/writing its fields directly in the HTTP handlers (e.g., handleSubmitIntake, handleConfirmIntake, handleRejectIntake) without synchronization causes a data race with the background cleanupLoop goroutine, which concurrently reads and writes the same fields (like Status and PendingValue).
In Go, concurrent read/write access to struct fields without synchronization is undefined behavior and can lead to memory corruption or crashes.
Recommended Solution
Encapsulate the state transitions inside SecretIntakeService methods under the protection of s.mu. This ensures that all reads and writes to SecretIntake fields are fully synchronized. Alternatively, return a copy of the struct value instead of a pointer to prevent external modification without synchronization.
func (s *SecretIntakeService) GetIntake(id string) (SecretIntake, bool) {
s.mu.Lock()
defer s.mu.Unlock()
intake, ok := s.intakes[id]
if !ok {
return SecretIntake{}, false
}
return *intake, true
}| const payload = parts[1] | ||
| .replace(/-/g, '+') | ||
| .replace(/_/g, '/'); | ||
| const decoded = atob(payload); |
There was a problem hiding this comment.
Bug: Missing Base64 Padding and UTF-8 Support in JWT Decoding
The client-side JWT decoding logic uses atob(payload) directly on the base64url-encoded payload segment. This has two major issues:
- Missing Padding: JWT payloads are base64url-encoded without padding. If the payload length is not a multiple of 4,
atobwill throw aDOMExceptionin most browsers, causing the page to fail to load and show the "Invalid Link" state. - UTF-8 Corruption:
atobdecodes strings as Latin-1. If the secret description or key contains non-ASCII UTF-8 characters, they will be corrupted (mojibake).
Recommended Solution
Add the necessary padding before calling atob, and decode the binary string as UTF-8.
| const payload = parts[1] | |
| .replace(/-/g, '+') | |
| .replace(/_/g, '/'); | |
| const decoded = atob(payload); | |
| const payload = parts[1] | |
| .replace(/-/g, '+') | |
| .replace(/_/g, '/'); | |
| const pad = payload.length % 4; | |
| const paddedPayload = pad ? payload + '='.repeat(4 - pad) : payload; | |
| const binaryString = atob(paddedPayload); | |
| const bytes = new Uint8Array(binaryString.length); | |
| for (let i = 0; i < binaryString.length; i++) { | |
| bytes[i] = binaryString.charCodeAt(i); | |
| } | |
| const decoded = new TextDecoder().decode(bytes); |
| if req.Token == "" || req.Value == "" { | ||
| writeError(w, http.StatusBadRequest, ErrCodeValidationError, "token and value are required", nil) | ||
| return | ||
| } |
There was a problem hiding this comment.
Security/Robustness: Missing Size Limit on Submitted Secret Value
There is no limit on the size of req.Value in handleSubmitIntake. Since the submitted secret is stored in memory (PendingValue) for up to an hour, an attacker could send extremely large values (e.g., tens of megabytes) to exhaust server memory and cause a Denial of Service (DoS).
Consider enforcing a reasonable maximum limit on the secret value size (e.g., 64 KB).
if req.Token == "" || req.Value == "" {
writeError(w, http.StatusBadRequest, ErrCodeValidationError, "token and value are required", nil)
return
}
if len(req.Value) > 65536 {
writeError(w, http.StatusBadRequest, ErrCodeValidationError, "secret value is too large (max 64KB)", nil)
return
}| // Validate required fields. | ||
| if req.Key == "" { | ||
| writeError(w, http.StatusBadRequest, ErrCodeValidationError, "key is required", nil) | ||
| return | ||
| } |
There was a problem hiding this comment.
Validation Gap: Missing scope_id Validation for Project Scope
When creating an intake link, if req.Scope is set to "project", there is no validation to ensure that req.ScopeID is also provided and non-empty. If a client creates an intake with scope: "project" but leaves scope_id empty, the secret will eventually be persisted with an empty project ID, which could lead to orphaned secrets or security bypasses.
Consider adding a validation check to ensure ScopeID is present when the scope is "project".
// Validate required fields.
if req.Key == "" {
writeError(w, http.StatusBadRequest, ErrCodeValidationError, "key is required", nil)
return
}
if req.Scope == "project" && req.ScopeID == "" {
writeError(w, http.StatusBadRequest, ErrCodeValidationError, "scope_id is required for project scope", nil)
return
}| // Secret intake submit endpoint uses JWT in request body, not auth header | ||
| if strings.HasPrefix(path, "/api/v1/secret-intake/") && strings.HasSuffix(path, "/submit") { | ||
| return true | ||
| } |
There was a problem hiding this comment.
Robustness: Loose Path Matching in isUnauthenticatedEndpoint
The path matching logic for the unauthenticated submit endpoint uses a simple prefix and suffix check. This is quite loose and could accidentally match nested or malformed paths (e.g., /api/v1/secret-intake/some/nested/path/submit) that are not intended to bypass authentication.
Consider using a more precise check by splitting the path segments or using a regular expression.
// Secret intake submit endpoint uses JWT in request body, not auth header
if strings.HasPrefix(path, "/api/v1/secret-intake/") {
parts := strings.Split(strings.TrimPrefix(path, "/api/v1/secret-intake/"), "/")
if len(parts) == 2 && parts[1] == "submit" {
return true
}
}| if (!this.context || !this.secretValue.trim()) return; | ||
|
|
||
| this.pageState = 'submitting'; | ||
|
|
||
| try { | ||
| const res = await fetch(`/api/v1/secret-intake/${this.context.jti}/submit`, { | ||
| method: 'POST', | ||
| headers: { 'Content-Type': 'application/json' }, | ||
| body: JSON.stringify({ | ||
| token: this.token, | ||
| value: this.secretValue.trim(), | ||
| }), |
There was a problem hiding this comment.
Bug: Trimming Secret Values Can Corrupt Secrets
The submission logic trims the secret value before sending it to the server. Trimming leading or trailing whitespace can corrupt certain types of secrets where whitespace is significant (such as multi-line PEM private keys, SSH keys, or specific API tokens that may contain or end with whitespace).
Recommended Solution
Only use .trim() to check if the input is empty, but submit the raw, untrimmed value.
| if (!this.context || !this.secretValue.trim()) return; | |
| this.pageState = 'submitting'; | |
| try { | |
| const res = await fetch(`/api/v1/secret-intake/${this.context.jti}/submit`, { | |
| method: 'POST', | |
| headers: { 'Content-Type': 'application/json' }, | |
| body: JSON.stringify({ | |
| token: this.token, | |
| value: this.secretValue.trim(), | |
| }), | |
| if (!this.context || !this.secretValue) return; | |
| this.pageState = 'submitting'; | |
| try { | |
| const res = await fetch(`/api/v1/secret-intake/${this.context.jti}/submit`, { | |
| method: 'POST', | |
| headers: { 'Content-Type': 'application/json' }, | |
| body: JSON.stringify({ | |
| token: this.token, | |
| value: this.secretValue, | |
| }), |
|
could said link basically just bring you to the new secret dialog in the linked project the secret intake with no login, then confirmation in chat seems like a mild security risk surface |
|
Thanks for the feedback! We were going for a more versatile and zero-friction approach (works even if the user isn't near a browser session), but you're right that the no-login surface is unnecessary risk. Switching to a deep-link into the Hub UI with a simplified, focused secret-entry screen. The user must be logged in, the link pre-fills the key/scope/project/type from a JWT, and we'll send a notification to chat when the secret is stored. Dropping the anonymous intake page and the confirmation step. |
…chat Agents in chat channels (Telegram, Discord) sometimes need users to provide secrets (API keys, tokens). Today users must either leave the chat to use the CLI/web UI, or paste secrets in plaintext in chat messages — a security anti-pattern. Secret intake links solve this: a short-lived JWT-secured URL that lets users paste a secret value directly into the Hub. The secret is stored in a pending state until the user confirms via the originating chat channel, where they can verify the submitter's IP and user-agent. Flow: 1. Agent calls POST /api/v1/secret-intake with key, scope, type 2. Hub returns a URL with JWT in fragment (never sent to server) 3. Agent sends URL in chat 4. User clicks, pastes value → stored as PENDING 5. Confirmation sent to chat with submitter IP/UA 6. User confirms → secret becomes ACTIVE Security: one-time use, 15 min TTL, rate limiting (5 attempts per intake + per-IP bucket), JWT signed with Hub signing key, pending value cleared on confirm/reject/expire. No login required on the intake page — the JWT is the authorization, confirmation is the gate. New files: - pkg/hub/secret_intake.go (716 lines) — handlers + service - pkg/hub/secret_intake_test.go (472 lines) — 12 tests - pkg/hubclient/secret_intake.go — SDK client - cmd/hub_secret_intake.go — CLI command - web/src/components/pages/secret-intake.ts — intake page - .design/secret-intake-link.md — design document Design doc: .design/secret-intake-link.md
Review findings and fixes: 1. TOCTOU race in submit handler: GetIntake + check status + update was not atomic. Concurrent submitters could both pass validation. Fix: new TrySubmit() method does all checks under a single lock. 2. GetIntake returned pointer to shared map entry. Callers could mutate the live record without holding the lock. Fix: GetIntake now returns a defensive copy. 3. Added 6 new tests: - TokenMismatch: JWT from one intake used on a different intake ID - DoubleConfirm: confirm an already-confirmed intake - SubmitMissingFields: missing token or value in submit body - RejectWrongStatus: reject intake that isn't pending confirmation - TTLCapping: TTL > 1 hour gets capped - ConfirmWrongStatus: confirm intake that isn't pending confirmation Total: 18 tests, all pass.
Redesign per maintainer feedback: replace the anonymous no-login intake page with an authenticated deep-link into the Hub web UI. Changes: - User MUST be logged in to submit a secret value (no anonymous endpoint) - Removed: pending/confirmation flow, OTP, PendingValue, SubmitterIP/UA, per-intake rate limiting, TrySubmit atomics - Simplified: intake record is just metadata + consumed flag - Secret is stored immediately on submit via secretBackend.Set() - Notification sent to originating chat channel on success - Auth exemption for submit endpoint removed from auth.go - Web intake page requires login, shows focused single-purpose form - Design doc updated to reflect simplified architecture Net: -450 lines. 12 tests covering the simplified flow.
3e3da3f to
9cbfdc9
Compare
|
Round 3 review complete — 3 clean cycles, no new findings. Rebased on upstream/main (resolved 4 merge conflicts in server.go from upstream Discord link service additions). All tests pass. Design simplified per maintainer feedback: authenticated deep-link, no anonymous endpoints, no OTP/confirmation flow. 12 tests. |
Why
When users interact with Scion agents via chat (Telegram, Discord), agents sometimes need secrets. Today users must leave the chat context to use CLI/web UI, or paste secrets in plaintext — a security anti-pattern that happened in the project's own history (raw GitHub PAT pasted into Telegram).
What
Secret intake links: a short-lived, JWT-secured URL that lets users paste a secret value directly into the Hub. The secret is stored as pending until the user confirms in the originating chat channel.
User flow
Security model
How
New files (1,859 lines)
pkg/hub/secret_intake.go— SecretIntakeService (in-memory, follows TelegramLinkService pattern), handlers for create/submit/confirm/reject, JWT generation/validation, rate limitingpkg/hub/secret_intake_test.go— 12 tests covering full lifecycle + security scenariospkg/hubclient/secret_intake.go— SDK client for the intake APIcmd/hub_secret_intake.go—scion hub secret intake KEYCLI commandweb/src/components/pages/secret-intake.ts— Lit component intake page (no login required).design/secret-intake-link.md— Full design document with security analysisModified files
pkg/hub/server.go— route registration + service lifecyclepkg/hub/auth.go— exempt intake submit endpoint from auth (JWT is the auth)pkg/hubclient/client.go— SecretIntake service accessorweb/src/client/main.ts— register /intake page routeTest plan
go build ./...passes