feat(websearch): native Go in-process websearch, no Python dependency (#381) - #404
Closed
Delqhi wants to merge 1 commit into
Closed
feat(websearch): native Go in-process websearch, no Python dependency (#381)#404Delqhi wants to merge 1 commit into
Delqhi wants to merge 1 commit into
Conversation
Implements a pure-Go, stdlib-only web search inside the sin-code binary so the legacy external Python sin-websearch skill is no longer required to satisfy the research surface of the agent loop. The package honours mandate M2 (single static binary, no CGO, no Python) and M7 (race-safe: -LOOK_TEST below). What ships: - cmd/sin-code/internal/native_websearch/cache.go — LRU + 15 min TTL - cmd/sin-code/internal/native_websearch/rate_limit.go — token-bucket limiter implemented with stdlib sync/atomic + sync.Cond; no golang.org/x/time/rate dependency required - cmd/sin-code/internal/native_websearch/search.go — DuckDuckGo HTML fetcher + parser (pure string-scan, no x/net/html), cache + limiter wiring, robots.txt honour with per-host snapshot (1 h TTL) - *_test.go: 22 race-clean tests covering parsing, rate limit, cache hit avoidance, robots Disallow, ctx cancellation, nil-safety, and concurrent burn-in for cache + limiter What wires up: - mcpclient/registry.go: adds internal_native_websearch goNative entry mapping to the native_websearch server name and shortName so the tool prefix is enumerated as native_websearch__search - catalog/source_external.go: parallel catalog entry so the operator surface lists the in-process fallback - permission_defaults.go: native_websearch__* -> allow (read-only) Constraints honoured: - M2: pure stdlib (net/http, net/url, container/list, sync, sync/atomic, time) — no CGO, no Python dependency, no extra go.mod lines needed - M5: module path github.com/OpenSIN-Code/SIN-Code/... - M7: cache.LRU + rate_limit.TokenBucket verified under -race -count=1 with 16 + 32 goroutines burn-in; TestCacheRaceClean and TestRateLimiterRaceClean run every release All 22 tests pass: go test -race -count=1 ./cmd/sin-code/internal/native_websearch/...
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🏆 CEO Audit — A+ (100.0/100)
📥 Download full report (Markdown)
|
🏆 CEO Audit — A+ (100.0/100)
📥 Download full report (Markdown) Run ID:
|
Collaborator
Author
|
Closing as duplicate. Issue #381 (Native Go web search) was already shipped by a parallel agent on
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pure stdlib DuckDuckGo HTML fetch + parse + LRU cache + rate limit. 22 tests race-clean.