Skip to content

fix(http): isolate project cache entries by URL scheme (http vs https)#7137

Open
usernametooshort wants to merge 4 commits intoprojectdiscovery:devfrom
usernametooshort:fix/project-cache-scheme-isolation
Open

fix(http): isolate project cache entries by URL scheme (http vs https)#7137
usernametooshort wants to merge 4 commits intoprojectdiscovery:devfrom
usernametooshort:fix/project-cache-scheme-isolation

Conversation

@usernametooshort
Copy link

Summary

Fixes #6866 — the project cache (-project) serves HTTPS responses for HTTP requests because cache keys do not include the URL scheme.

Root Cause

dump() calls req.request.Clone().Dump() which produces raw HTTP/1.1 request bytes:

GET / HTTP/1.1
Host: example.com
...

The scheme (http:// vs https://) is NOT included. So ProjectFile.Get() hashes the dump and gets the same key for both http://example.com and https://example.com.

Fix

Prefix the dumped bytes with the full URL (including scheme) before returning from dump(). This makes the hash include the scheme, producing distinct cache entries.

Behaviour

  • Before: http://example.com with -project returns cached https://example.com response — false positive
  • After: separate cache entries per scheme, no cross-contamination

The second goroutine in runWorkflowStep was not cloning ctx.Input
before passing it to runWorkflowStep, unlike the first goroutine
which properly calls ctx.Input.Clone().

The comment on both paths says 'clone the Input so that other parallel
executions won't overwrite the shared variables' but only the first
path was actually doing it.

This causes a race condition where parallel subtemplate executions
can corrupt each other's input data when modifying shared variables.
…ure them

When a template variable like {{contact_id}} is not provided by the user
and gets passed through an encoding function (e.g. base64(rawhash) where
rawhash contains {{contact_id}}), the ContainsUnresolvedVariables check
on the final dumped request fails to detect it — the base64-encoded form
e3tjb250YWN0X2lkfX0= contains no {{...}} markers.

Fix: after EvaluateWithInteractsh builds the variablesMap, check each
variable value for unresolved {{...}} markers before allVars is assembled
and passed to expression evaluation. This catches self-referential
variables (e.g. contact_id: "{{contact_id}}") that were never provided
by the user via -var.

Fixes projectdiscovery#7032
…te install

When nuclei-templates are not installed and the GitHub API returns a
rate limit error (403), the SDK panics because NewNucleiEngineCtx
propagates the error up to the caller.

Fix: detect rate limit errors in FreshInstallIfNotExists and
UpdateIfOutdated, and log a warning instead of returning a fatal error.
This allows the engine to start even without templates (users can retry
later or set GITHUB_TOKEN for higher rate limits).

Fixes projectdiscovery#7118
The project file (-project) uses a hash of the dumped HTTP request as
the cache key. Since http.Request.Dump() produces only the raw HTTP/1.1
request line (e.g. 'GET / HTTP/1.1') without the scheme, requests to
http://example.com and https://example.com produce identical cache keys.

This causes HTTPS responses to be served for HTTP requests (and vice
versa) when using the project cache, leading to false positives.

Fix: prefix the dumped request with the full URL (including scheme)
before hashing, so http:// and https:// produce distinct cache entries.

Fixes projectdiscovery#6866
@neo-by-projectdiscovery-dev
Copy link

neo-by-projectdiscovery-dev bot commented Mar 6, 2026

Neo - PR Security Review

No security issues found

Highlights

  • Adds URL scheme (http:// vs https://) to project cache keys to prevent cross-scheme cache collisions
  • Improves template cleanup with orphaned file removal and metadata regeneration
  • Maintains existing path traversal protections in installer code
Hardening Notes
  • The cache fix only applies to standard HTTP requests (req.request != nil). Raw requests using the rawRequest path call rawhttp.DumpRequestRaw with reqURL parameter but don't explicitly prepend the scheme. This may be intentional if DumpRequestRaw includes the full URL in its output, but should be verified through integration testing.
  • The installer code maintains secure path handling with filepath.Clean() normalization and prefix validation to prevent path traversal (lines 297-299, 436, 449, 460, 485 in template.go).

Comment @pdneo help for available commands. · Open in Neo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Nuclei cache/project pulls HTTPS cache into HTTP targets

1 participant