fix: shed load and survive shutdown instead of blocking and panicking in Submit

[GautamSharma99]

Fixes #58.

Branched from master and independent of my other open PRs — nothing else touches internal/worker.

The two failures

worker.Pool.Submit was a bare channel send, called synchronously from inside the Fiber request handlers.

It blocked instead of shedding load. Once JOB_BUFFER_SIZE jobs were queued and every worker was busy, the send parked the request goroutine, so POST /v1/scrape and POST /v1/url-scraper stopped responding. Clients saw connections hang until they timed out, with no log line to explain it.

It panicked during shutdown. Drain closes the job channel, so a Submit racing it died with send on closed channel, taking the process down. That window is real rather than theoretical: main.go bounds app.ShutdownWithContext at 30s, but a sync scrape can legitimately still be inside its handler at 120s since #12 raised the ceiling — so Drain can close the channel while a handler is live.

The fix

Submit now returns an error and never blocks: ErrQueueFull at capacity, ErrPoolClosed once draining has begun.

A single RWMutex covers both problems. Submit holds it for reading across the non-blocking send; Drain takes it for writing before closing. Taking the write lock has waited out every in-flight Submit, so no send can be in flight at the moment of the close — the race is eliminated by construction rather than narrowed. Drain is idempotent too. Because the send is select/default, the read lock is never held long enough to matter.

The handlers translate both errors into 503 with Retry-After, and mark the already-created job row failed — otherwise it sits pending forever with nothing to pick it up. A batch marks each rejected child individually and only returns 503 when nothing at all could be queued, so a partially-accepted batch still gets its id back.

The doc comment said "Non-blocking if buffer has space", which framed the safe case as the contract; it now states the refusal behaviour.

Verification

internal/worker had no test file. All three defects reproduce against the previous implementation:

=== full-queue behaviour ===
--- FAIL: TestPool_SubmitReturnsErrQueueFullInsteadOfBlocking (5.01s)
    worker_test.go:98: Submit blocked on a full queue instead of returning ErrQueueFull

=== Submit racing Drain ===
panic: send on closed channel

=== Drain idempotence ===
panic: close of closed channel [recovered, repanicked]

The race test runs 8 concurrent submitters against Drain over 20 attempts, so it isn't relying on one lucky interleaving.

I also ran the issue's own reproduction end to end. Server with JOB_BUFFER_SIZE=5 WORKER_POOL_SIZE=1 against a 10s origin, 20 concurrent submissions:

   6 x 201   (1 in flight + 5 buffered)
  14 x 503   all in under 3ms

Previously the 14 would have hung until a worker freed a slot. Now they are refused immediately with a backoff hint.

The handler-side tests use a pool with no workers and no buffer, so every Submit is refused deterministically — same state a real pool reaches under saturation, with no timing dependence.

Coverage: internal/worker 0% → 96.6%.

Full CI parity locally — gofmt -l clean, go build ./..., go vet ./..., go test -race ./... all pass.

Notes for the reviewer

• Submit now returns an error. All three call sites are in scraper.go and are updated; there are no others.
• Choice of 503 over queuing to disk or blocking with a timeout: backpressure that reaches the client is the only option that keeps the API responsive, and Retry-After gives clients something actionable. Happy to tune the retry hint or the message.
• The handler test file is scraper_queue_test.go and its fake is named stubStore, so it does not collide with the scraper_test.go/fakeStore added by fix: test the shipping sync-timeout clamp, and fix an overflow it hid #60.
• This is adjacent to Jobs are stuck in processing forever when they hit JOB_TIMEOUT — the failure is written with the already-expired job context #32 (terminal state written with an expired context). They touch the same bgCancel() / Drain() shutdown path, so fix: record terminal job state with a fresh context so timed-out jobs don't hang in processing #33 and this PR are worth landing near each other, but they fix different defects and do not overlap in code.
• Expected trivial conflict with my other open PRs on the ## Unreleased CHANGELOG heading, and possibly a small one with fix: test the shipping sync-timeout clamp, and fix an overflow it hid #60 in ScrapeSync — different hunks, so it should auto-merge. Happy to rebase whichever lands last.