Skip to content

feat(mobile): swipe to ask — bring the ask verb to the capture screen #321

Description

@jayesh-keychain

Goal

Bring the ask verb to mobile: a horizontal swipe switches the capture screen between capture mode (default, unchanged) and ask mode, where the user types a question and gets a grounded answer with citations.

Why now

Mobile currently implements only one of the four verbs. It calls exactly four endpoints — /auth/login, /memory/capture, /notifications, /notifications/register-token — and there is no reference to /memory/ask anywhere in apps/mobile. A user can put memories in from their phone but cannot ask anything of them; the "inform" screen is a notification feed, not a recall surface. Recall works end-to-end on the API and in Admin, so this is purely a missing mobile surface.

Scope — bigger than a gesture

This is not just mode-switching. It needs:

  1. api.ask() in apps/mobile/src/lib/api.ts (the AskRequest/AskResponse contracts already exist in @thebrain/types).
  2. An ask UI: query input, answer, citations, loading + honest-miss states.
  3. The gesture + mode state.

How to do it

Axis matters. apps/mobile/app/index.tsx already drives BOTH hold-to-talk and slide-up-to-type from a single vertical Pan (see beginHold / slidingRef / dragY). The vertical axis is fully taken, so mode switching MUST be horizontal — a left/right swipe — or it will fight capture. Concretely this has already bitten us: adb vertical swipes during testing kept starting voice recordings instead of opening the composer.

  • Add a horizontal Pan (or Gesture.Race with the existing one) with a clear activation threshold and activeOffsetX, so a mostly-vertical drag never switches modes and vice versa.
  • Keep capture the default mode on every launch — asking is deliberate, capturing is reflexive. Do NOT persist the last mode.
  • Reuse the existing composer input rather than building a second text surface.
  • Ask goes through request() in src/lib/api.ts, so it inherits the 401 refresh-and-retry from fix(mobile): refresh the expired session instead of losing the capture #318 for free.
  • Render citations from AskResponse.citations; an honest miss (citations: []) must read as "I don't have a memory about that", NOT as an error.

Out of scope

  • Voice-driven ask (hold-to-talk stays capture-only).
  • Multi-turn threads / sessionId — one-shot ask first.
  • Any change to the recall pipeline or confidence bands.

Acceptance criteria

  • App opens in capture mode every launch; existing hold-to-talk and slide-up-to-type behave exactly as before (no regression).
  • A horizontal swipe switches to ask mode and back, with a visible mode indicator.
  • A mostly-VERTICAL drag never switches mode; a mostly-HORIZONTAL drag never starts a recording or opens the text composer.
  • Asking a question hits POST /memory/ask and renders the answer plus its citations.
  • A grounded miss renders as an honest "no memory" state, not an error.
  • A 401 mid-ask refreshes and retries once (inherited from fix(mobile): refresh the expired session instead of losing the capture #318) rather than dropping the question.
  • Verified on a real emulator run, not just type-check — note that gesture surfaces are hard to drive via adb (a swipe registers as hold-to-talk), so this likely needs a manual pass or a test seam.

Notes

Blocked-ish on nothing, but see #319 (asymmetric query embedding): until that merges, ask on main returns "I don't have a memory about..." for most paraphrased questions, which would make this feature look broken when it isn't.

Metadata

Metadata

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions