feat(fetch): add --live flag to bypass the cache

[IlyaGusev]

Summary

• Add --live to keenable fetch — requests live page content instead of the cached copy, mapping to the live=true query param the API (and the MCP fetch_page_content tool) already supports.
• Thread live through DaemonRequest with #[serde(default)] so old/new client-daemon pairs stay interoperable (same skew precedent as the fetch POST→GET change in Fix fetch command using POST instead of GET #15), and through the direct HTTP fallback path.
• Update after_help examples, README, and add test_fetch_live to the e2e suite.

Testing

• cargo fmt / cargo clippy --all-targets -- -D warnings / cargo test all clean.
• Verified live=true against the real API (public endpoint) with curl before wiring.
• Ran the built binary end-to-end: fetch --live succeeds both through a running daemon (isolated KEENABLE_HOME, socket confirmed) and via the direct path.
• pytest tests/e2e/test_fetch.py — 8/8 pass, including the new test_fetch_live.

Note: nightly e2e CI tests the released binary, so test_fetch_live will fail there until the next release is tagged.

🤖 Generated with Claude Code

[qodo-code-review]

PR Summary by Qodo

Add --live flag to fetch to bypass cache

✨ Enhancement 🧪 Tests 📝 Documentation 🕐 20-40 Minutes

AI Description

• Add --live to keenable fetch to request uncached page content.
• Propagate live through the daemon protocol and direct HTTP fallback path.
• Update CLI/help docs and add an e2e test covering fetch --live.

Diagram

sequenceDiagram
  actor U as User
  participant C as CLI (keenable)
  participant D as Daemon
  participant A as Keenable API

  U->>C: keenable fetch <url> --live
  alt Daemon available (no --api-key override)
    C->>D: DaemonRequest{command:"fetch", urls:[url], live:true}
    D->>A: GET /v1/fetch?url=...&live=true
    A-->>D: content
    D-->>C: DaemonResponse{ok,data}
  else Direct HTTP
    C->>A: GET /v1/fetch?url=...&live=true
    A-->>C: content
  end
  C-->>U: YAML / pretty output

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Generic request options map (extensible params)

• ➕ Avoids adding new fields to DaemonRequest for each future fetch/search option
• ➕ Can reduce protocol churn when adding query/body toggles
• ➖ Less type-safe and self-documenting than explicit fields like live: bool
• ➖ Harder to validate/serialize consistently across commands
• ➖ May introduce ambiguity between command-specific vs shared options

Recommendation: The current approach (explicit live: bool with #[serde(default)]) is the best fit here: it’s type-safe, easy to reason about, and preserves old/new client↔daemon interoperability. If fetch accrues many more optional toggles, revisiting a generic options container could become worthwhile, but it’s unnecessary complexity for a single flag.

Files changed (5) +34 / -5

Enhancement (3) +25 / -5

search.rsThread 'live' through fetch request execution +9/-2

Thread 'live' through fetch request execution

• Extends the direct HTTP fetch path to include 'live=true' when requested. Updates the 'fetch()' helper signature to accept 'live', and initializes 'DaemonRequest.live' for non-fetch commands to keep construction explicit.

src/commands/search.rs

daemon.rsAdd 'live' to DaemonRequest and forward as query param +9/-1

Add 'live' to DaemonRequest and forward as query param

• Adds a 'live: bool' field to 'DaemonRequest' with '#[serde(default)]' for backward compatibility. Updates the daemon’s fetch handler to append 'live=true' to the '/v1/fetch' query when set.

src/daemon.rs

main.rsAdd '--live' flag to 'fetch' subcommand and wire to fetch() +7/-2

Add '--live' flag to 'fetch' subcommand and wire to fetch()

• Introduces a '--live' CLI flag for 'keenable fetch', updates help examples, and passes the flag through to the fetch command implementation.

src/main.rs

Tests (1) +8 / -0

test_fetch.pyAdd e2e coverage for 'fetch --live' +8/-0

Add e2e coverage for 'fetch --live'

• Adds 'test_fetch_live' to verify the CLI accepts '--live' and returns expected content for example.com.

tests/e2e/test_fetch.py

Documentation (1) +1 / -0

README.mdDocument 'fetch --live' usage example +1/-0

Document 'fetch --live' usage example

• Adds an example showing 'keenable fetch <url> --live' to clarify how to bypass cached content.

README.md

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

1. Old daemon ignores --live 🐞 Bug ≡ Correctness

Description

The CLI reuses any already-running daemon without checking capabilities, so after an upgrade an
older daemon can keep serving fetch requests but cannot forward live=true, causing `fetch
--live to return cached content until the daemon restarts. Because execute()` returns immediately
on successful daemon responses, there is no opportunity to fall back to the direct HTTP path for
--live freshness semantics.

Code

src/daemon.rs[R11-13]

+    /// Fetch only: request live content instead of the cached copy.
+    #[serde(default)]
+    pub live: bool,

Evidence

The CLI will reuse an already-running daemon and return its successful response immediately, while
the daemon only forwards live=true when it has the new req.live logic. Therefore, if a
pre-upgrade daemon is still running, it can’t honor --live freshness semantics and the client
won’t fall back to direct HTTP to compensate.

src/commands/search.rs[96-111]
src/daemon.rs[341-388]
src/daemon.rs[390-420]
src/daemon.rs[278-294]
src/daemon.rs[316-339]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`fetch --live` depends on daemon support to append `live=true`, but the CLI will reuse an already-running daemon process without any version/feature negotiation. If the user upgrades the CLI while an older daemon is still running, the request can succeed yet not be “live” (cached content returned) until the daemon exits/restarts.

### Issue Context
- The CLI tries the daemon first and returns the daemon’s successful response immediately.
- The daemon only appends the `live=true` query param when it understands/implements `req.live`.
- There’s explicit precedent in the codebase that older daemons can remain running and behave differently (see comments in `kill_daemon`).

### Fix Focus Areas
- src/commands/search.rs[96-170]
- src/commands/search.rs[388-420]
- src/daemon.rs[383-420]
- src/daemon.rs[260-314]
- src/daemon.rs[316-339]

### Suggested fix
Choose one (or combine):
1) **Bypass daemon when `req.live == true`**: in `execute()`, skip the daemon fast-path for fetch-live so the direct HTTP request always includes `live=true`.
2) **Force daemon refresh on `--live`**: if `req.live` is true and a daemon is running, call `kill_daemon()` then `ensure_daemon()` to guarantee a daemon with live support.
3) **Add a capability handshake**: extend `ping` to return a version/feature set; when `--live` is requested, only use daemon if it advertises support, otherwise fall back to direct HTTP.

(Option 1 is the simplest/least invasive and makes `--live` semantics reliable immediately after an upgrade.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[qodo-code-review]

1. Old daemon ignores --live 🐞 Bug ≡ Correctness

The CLI reuses any already-running daemon without checking capabilities, so after an upgrade an
older daemon can keep serving fetch requests but cannot forward live=true, causing `fetch
--live to return cached content until the daemon restarts. Because execute()` returns immediately
on successful daemon responses, there is no opportunity to fall back to the direct HTTP path for
--live freshness semantics.

Agent Prompt

### Issue description
`fetch --live` depends on daemon support to append `live=true`, but the CLI will reuse an already-running daemon process without any version/feature negotiation. If the user upgrades the CLI while an older daemon is still running, the request can succeed yet not be “live” (cached content returned) until the daemon exits/restarts.

### Issue Context
- The CLI tries the daemon first and returns the daemon’s successful response immediately.
- The daemon only appends the `live=true` query param when it understands/implements `req.live`.
- There’s explicit precedent in the codebase that older daemons can remain running and behave differently (see comments in `kill_daemon`).

### Fix Focus Areas
- src/commands/search.rs[96-170]
- src/commands/search.rs[388-420]
- src/daemon.rs[383-420]
- src/daemon.rs[260-314]
- src/daemon.rs[316-339]

### Suggested fix
Choose one (or combine):
1) **Bypass daemon when `req.live == true`**: in `execute()`, skip the daemon fast-path for fetch-live so the direct HTTP request always includes `live=true`.
2) **Force daemon refresh on `--live`**: if `req.live` is true and a daemon is running, call `kill_daemon()` then `ensure_daemon()` to guarantee a daemon with live support.
3) **Add a capability handshake**: extend `ping` to return a version/feature set; when `--live` is requested, only use daemon if it advertises support, otherwise fall back to direct HTTP.

(Option 1 is the simplest/least invasive and makes `--live` semantics reliable immediately after an upgrade.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools