diff --git a/docs/mobile-chat/9a-citations/00-index.md b/docs/mobile-chat/9a-citations/00-index.md new file mode 100644 index 0000000000..349e7dc1e8 --- /dev/null +++ b/docs/mobile-chat/9a-citations/00-index.md @@ -0,0 +1,24 @@ +# Mobile Chat 9a — Citations & Cited Sources + +> Status: active — spec complete, ready for execution. + +Sub-phase 9a of the deferred rich-chat work in `../05-pr-roadmap.md` (the parent mobile-chat +roadmap). Ports web's chat citations + cited-sources experience to the React Native app. + +## Artifacts + +1. [01-research.md](01-research.md) — requirement, clarifications, codebase + backend + web + + industry findings, the three approaches, and the chosen one (C + A's inline simplification). +2. [02-high-level-design.md](02-high-level-design.md) — end-to-end walkthrough, component + interaction, key decisions. +3. [03-detailed-design.md](03-detailed-design.md) — interfaces, new files, file tree, per-file + contents, integration points, implementation notes. +4. [04-implementation-plan.md](04-implementation-plan.md) — CLAUDE.md-format plan + the appended + plan-challenge results. +5. [05-pr-roadmap.md](05-pr-roadmap.md) — the single-PR delivery plan. + +## One-line summary + +Inline `[N]` markers become styled tappable links (open in the in-app browser) and a "Sources" +button + bottom sheet lists cited/found documents — built on a small reusable packet-processing + +source-UI foundation that PR 9b (agent timeline) extends. No backend/DB changes. diff --git a/docs/mobile-chat/9a-citations/01-research.md b/docs/mobile-chat/9a-citations/01-research.md new file mode 100644 index 0000000000..7716dc2a68 --- /dev/null +++ b/docs/mobile-chat/9a-citations/01-research.md @@ -0,0 +1,287 @@ +> Status: draft · Task: 9a-citations + +# Mobile Chat 9a — Citations & Cited Sources — Research + +## Requirement + +Port Onyx **web's chat citations + cited-sources experience** to the mobile React Native app +(`mobile/`): type + process the citation/document streaming packets, render inline `[N]` +citation markers in the streamed answer, and render a cited-sources surface (a "Sources" +affordance + a list of source rows), matching web's behaviour and — where the platform allows — +its structure. This is sub-phase **9a** of the deferred rich-chat work in +`docs/mobile-chat/05-pr-roadmap.md` (PR 9a–9e). Web is the design source of truth. + +## Clarifications (Q&A) + +**Q1 — Which PR 9 sub-phase?** → **9a (citations/sources).** (Roadmap's recommended-first; the +agent-timeline work we analysed is 9b, deferred.) + +**Q2 — Kick-off process?** → Full feature-flow (research → approaches → HLD → detailed → plan), +gated docs under `docs/mobile-chat/`, grilled before any code. + +**Q3 (the load-bearing one) — Inline `[N]` marker treatment, given the RN markdown renderer has +NO custom-node hook?** → **Styled, tappable `[N]` links** (tap → open the source via +`onLinkPress`); the rich, web-faithful UI goes in the **Sources button + cited-sources sheet** +(which have no renderer constraint). Superscript styling of the marker is optional. Real inline +`SourceTag`-style chips (web's look) would require heavy text-segmentation that fights the +streaming markdown renderer — **rejected** as too fragile for the value. Hiding markers entirely — +**rejected** (loses the claim→source bond). + +### Why the constraint exists (recorded so it isn't re-litigated) +- **Web** renders the answer with `react-markdown` (a JS element tree); Onyx overrides the `a` + node (`MemoizedAnchor`) to swap each `[[N]](url)` link for a custom `SourceTag` chip + hover + card. Every node is a React element it can replace. +- **Mobile** uses `react-native-streamdown` → `react-native-enriched-markdown`, a **native** + renderer (md4c + native text primitives, chosen to avoid the WebView tax + streaming jank). Its + entire public customization surface is `markdownStyle` (per-node colour/weight/underline) + + `onLinkPress`/`onLinkLongPress` tap callbacks. **No custom node renderers / render overrides** + (confirmed against the published API reference). So a `[N]` link cannot be replaced by a custom + component inline; it can only be styled + made tappable. The **hover card** can't port literally + regardless (no hover on touch → tap-to-open is the equivalent). + +## Current status & reuse (codebase scan — exact paths) + +### Render path & the dispatch seam +- `mobile/src/components/chat/MessageList.tsx` → `MessageRow` per item; no per-packet logic. +- `mobile/src/components/chat/MessageRow.tsx` — `AssistantMessage` calls `usePacketDisplay(node)` + → `{renderer, packets, isComplete}`, renders `` (shell) then + `` in a `px-12` inset. Memoized on `node.packets.length`. +- `mobile/src/hooks/usePacketDisplay.ts` — **the "one group" stub**; header comment: "Core = one + group; PR 9 adds real grouping." Passes ALL `node.packets` to the single matched renderer. +- `mobile/src/components/chat/renderers/registry.ts` — dispatch seam: + `MessageRenderer { matches(packets):boolean; Component }`, + `MessageRendererProps { packets: Packet[]; isComplete: boolean }`, `findRenderer` first-match. + Only `MessageTextRenderer` registered ("PR 9 adds rich renderers"). +- `mobile/src/components/chat/renderers/MessageTextRenderer.tsx` — + `matches: packets.some(isChatPacket)`; Component: `accumulateContent(packets)` (concat + `content` of `message_start`/`message_delta`) → `useTypewriter` → ``. + +### Markdown renderer +- `mobile/src/components/chat/StreamingMarkdown.tsx` — wraps + ``. Today passes NO tap + callback. `StreamdownText` **inherits all `EnrichedMarkdownText` props** (incl. + `onLinkPress`/`onLinkLongPress` → `event.url`) **plus** `remendConfig`. `markdownStyle.link` = + `--action-link-05` + underline (global; no per-link styling). +- `mobile/src/hooks/useTypewriter.ts` — reveals `target` by growing char-prefix slice + (~180cps mid-stream). Markers pass through as chars → a partial `[1` can momentarily show; + citation parsing must run on the **full accumulated content**, not `displayed`. + +### Packet contracts & ingestion +- `mobile/src/chat/streamingModels.ts` — `PacketType` enum (`message_start/delta/end`, `stop`, + `section_end`, `error`), `Packet {placement, obj}`, `Placement {turn_index, tab_index?, + sub_turn_index?, model_index?}`, `ObjTypes` union. **Extension point** for new packet types. + Two-tier discrimination convention (wrapped `obj.type` vs root-by-field) enforced in + `mobile/src/api/chat/stream.ts` (`isPacket = "obj" in e && "placement" in e`). +- `mobile/src/chat/contracts/projects.ts` — project read-models; candidate home for a + `SearchDoc`/document contract (`mobile/src/chat/contracts/`). +- `mobile/src/hooks/useChatController.ts` — appends **every** wrapped packet to `node.packets` + via a debounced flush (`{...node, packets:[...node.packets, ...pending]}`); **no `obj.type` + inspection** → new citation/document packet types reach the renderer with zero controller change. +- `mobile/src/chat/chatHistory.ts` — `processRawChatHistory` aligns `packets[agentIdx]` per + assistant turn → historical citations load through here. + +### Reuse primitives +- `mobile/src/components/ui/card.tsx` — `Card` (`variant`: primary/secondary/tertiary/borderless; + `rounded-12 p-16`; optional `onPress`). +- `mobile/src/components/ui/line-item-button.tsx` — `LineItemButton` (full-width tappable row: + icon + title + description + `rightChildren`; no snippet slot). +- `mobile/src/components/chat/FilePickerSheet.tsx` — **the bottom-sheet Modal pattern** + (RN `Modal` transparent slide, scrim `Pressable`, inner `rounded-t-24`, safe-area bottom inset). +- `mobile/src/components/chat/{ProjectFileList,FileCard}.tsx` — row/card shapes. +- `mobile/src/components/ui/BearerImage.tsx` — auth'd `expo-image` (for API images). **Note:** + public web favicons are NOT auth'd → use plain `expo-image`, not `BearerImage`. +- `mobile/src/components/ui/{spinner,text,icon,button,separator,content}.tsx` — `Spinner`, `Text` + (all text must use this), `Icon`, `Button`, `Separator`, `Content`/`ContentAction`. +- `mobile/src/hooks/useToast.ts` — `useToasts()` toast host (for "preview unavailable" etc.). +- Opening links: **`expo-web-browser`** `WebBrowser.openBrowserAsync(url)` (already a dep; + nothing opens content links today — only auth SSO uses the browser). + +### Confirmed gaps / collisions +- **Source-icon gap:** mobile has **no** `SourceIcon` / `WebResultIcon` / favicon anywhere + (grep-confirmed). Web uses `SourceIcon` (source_type→icon) + `WebResultIcon` (favicon from URL). + 9a must add a minimal version; a full ~40-connector logo set is out of scope. +- **No document-preview surface** on mobile (web opens File/UserFile docs in a `PreviewModal`). +- **Route `/sources/[id]` is taken** — it's the PROJECT-FILES screen + (`mobile/src/app/(app)/sources/[id].tsx`, reads `id` as projectId, reuses `useProjectFiles`). + `deriveFocus` (`mobile/src/chat/chatFocus.ts`) only matches `/`, `/chat/…`, `/projects/…`. A + chat-citations surface must be a **bottom sheet** (or a new, distinct route), not `/sources`. +- **No renderer / `usePacketDisplay` tests** exist. `mobile/src/chat/__tests__/fixtures.ts` has + only `makeProjectFile`; the de-facto packet fixture is inline in `ndjson.test.ts`. Add + `makePacket` / `makeCitationPacket` / `makeSearchDoc(Packet)` helpers. + +## Backend packet spec (authoritative wire shapes) + +NDJSON, one `Packet` per line: `{ placement:{turn_index, tab_index?, sub_turn_index?, +model_index?}, obj:{ type, ... } }`. No `data:` SSE prefix. + +- **Citation — the ONLY citation packet:** `CitationInfo` + `{ type:"citation_info", citation_number:int, document_id:str }` + (`backend/onyx/server/query_and_chat/streaming_models.py:144`). Web's enum also declares + `citation_start`/`citation_end` — **the backend never emits them; ignore.** +- **Document packets** (each `{ type, documents: SearchDoc[] }`): + - `SearchToolDocumentsDelta` — `type:"search_tool_documents_delta"` (internal + web search). + - `OpenUrlDocuments` — `type:"open_url_documents"` (URL-fetch tool; web calls it + `FetchToolDocuments` — same wire string). +- **`message_start`** carries `final_documents: SearchDoc[] | null` (authoritative cited-doc set) + + `pre_answer_processing_seconds`. (Backend `AgentResponseStart` does NOT send web's `id`/ + `content` on this packet.) +- **`SearchDoc`** (`backend/onyx/context/search/models.py:283`): `document_id:str`, + `semantic_identifier:str`, `link:str|null`, `blurb:str`, `source_type:str`, `score:number|null`, + `updated_at:str(ISO)|null`, `match_highlights:str[]`, `metadata:Record`, + `is_internet:bool`, `chunk_ind:int`, `boost:int`, `hidden:bool`, plus nullable + `primary_owners/secondary_owners/is_relevant/relevance_explanation/file_id`. (No `db_doc_id` on + the wire — it's `SearchDoc`, not `SavedSearchDoc`.) +- **Terminal packet is `stop`** (`{type:"stop", stop_reason}`) — there is **no `message_end`**. + Mobile's `isComplete` already keys on `STOP`. + +### The pivotal inline-marker fact +`backend/onyx/chat/citation_processor.py:496,506`: the inline marker is emitted as +`[[{num}]]({link})` where **`link = search_doc.link or ""`**. So: +- **Web/linked docs** → the marker URL **is** the document link → `onLinkPress(event.url)` can open + it directly, **no citation-state resolution needed for the tap**. +- **File/internal docs (no link)** → the marker is **`[[n]]()`** (empty parens) → may not render as + a tappable link at all (the real edge case). + +Ordering per answer turn: document packets precede the answer (in the search tool's turn) → +`message_start` (`final_documents`) → repeating(`citation_info` immediately **before** the +`message_delta` carrying its `[[n]](url)` text) → `section_end` → `stop`. Dedup: a `document_id` +emits `citation_info` only on first cite. + +## Web structure to mirror (parity target) + +`packetProcessor.ts` (`handleCitationPacket`/`handleDocumentPacket`) builds three pieces of state, +mutated in place, incrementally (process-only-new via `nextPacketIndex`, reset-on-shrink): +`citationMap` (`{[n]: document_id}`), `citations[]` (deduped `{citation_num, document_id}` via a +seen-set, first-cite order), `documentMap` (`Map`). Three surfaces, all fed from +this state: +- **(A) inline marker** in the answer markdown → `MemoizedAnchor` resolves `[N]` → `citationMap` → + doc → `SourceTag` chip + hover card. (Mobile: styled tappable link instead.) +- **(B) "Sources" toolbar button** (`MessageToolbar` → `SourceTag variant="button"`): an IconStack + of ≤3 source icons + "Sources" label, shown when `citations.length>0 || documentMap.size>0`, + tap → opens (C). +- **(C) cited-sources list** (`DocumentsSidebar`): **on mobile web already renders a bottom + `Modal` titled "Sources"**. Sections: **Cited Sources** (sorted by citation order) / **More** + (or "Found Sources") / **User Files**. Each row = `ChatDocumentDisplay`: [source-type icon or + favicon + title(`semantic_identifier`) truncated] / [metadata: updated-at badge + ≤3 metadata + chips] / [snippet from `match_highlights || blurb`]. Row tap → `openDocument`. +- **`openDocument`** (`web/src/lib/search/utils.ts`): `link` → open in browser; File/UserFile (no + link) → preview modal; else no-op. + +## Industry best practices (mobile citation UX) + +- **Keep numeric `[N]` markers** (the RAG-grade pattern; Perplexity/Claude use them) — the work is + styling + tap-target, not switching to inline hyperlinks. — + https://medium.com/@shuimuzhisou/how-ai-engines-cite-sources-patterns-across-chatgpt-claude-perplexity-and-sge-8c317777c71d +- **Markers must be real focusable/tappable elements** with a tap-to-open behaviour (no hover + fallback on touch). — https://www.aydesign.ai/blog/ai-citation-source-ui-patterns-2026 +- **Tap target ≥ 44×44 pt** (Apple HIG) / WCAG 2.5.8 24px floor → expand the hit area + (`hitSlop`/padding) beyond the tiny glyph. — https://www.designmonks.co/blog/perfect-mobile-button-size +- **Sources = a "Sources (N)" collapsible / bottom sheet / horizontal card scroller**, NOT a + desktop side rail; card shows favicon + title + domain + ≤200-char snippet. — + https://www.aydesign.ai/blog/ai-citation-source-ui-patterns-2026 +- **Tap-to-open default = in-app browser** (`expo-web-browser` `openBrowserAsync` → + SFSafariViewController / Chrome Custom Tabs), not external Safari/Chrome; a preview sheet for + internal docs with no URL. — https://docs.expo.dev/versions/latest/sdk/webbrowser/ +- **Streaming:** parse markers off the **cumulative** answer text (a `[12]` can split across + deltas); tolerate forward-references (marker before its source) — render inert until resolved. — + https://docs.perplexity.ai/docs/cookbook/articles/streaming-citations/README +- **Accessibility:** each marker needs a descriptive `accessibilityLabel` ("Source N: {title}"), + not the bare number; `accessibilityRole="link"`. — https://www.deque.com/blog/text-links-practices-screen-readers/ +- **NN/g:** users rarely click citations but their presence drives (over)trust — lead source cards + with title + domain (meaningful labels), place markers adjacent to the claim. — + https://www.nngroup.com/articles/explainable-ai/ + +## Approaches + +### Approach A — Simplicity-First: extend `MessageTextRenderer`, no new framework +Citations ride the existing `node.packets`; `usePacketDisplay` already hands all packets to the one +matched `MessageTextRenderer`, so **no registry / `usePacketDisplay` / controller / history +change**. Extend `MessageTextRenderer` to (a) pass `onLinkPress` through `StreamingMarkdown` and +(b) fold packets via one pure `deriveCitations(packets)` (re-run each flush via `useMemo`, like +`accumulateContent`) that drives a Sources bar + bottom-sheet Modal. **Inline routing needs no +citation state** — the marker URL is pre-baked, so `onLinkPress(event.url)` opens it directly; +empty-URL file markers route to opening the Sources sheet. New: `contracts/search.ts` (`SearchDoc`), +`chat/citations.ts` (`deriveCitations` + `faviconUrlForLink`/`domainOf`), `SourceIcon.tsx`, +`CitedSources.tsx` (bar + sheet + row). ~4 new files, 4 small edits. +- **Trade-offs:** fewest moving parts, ships fast, lowest risk; but citation logic is trapped in + `MessageTextRenderer` (9b's grouping will have to re-home it), `deriveCitations` re-scans the full + array each flush (fine at chat scale), and it builds no shared source layer for 9b to reuse. +- **Risks:** empty-parens `[[n]]()` file markers may not parse as a link (mitigation: 1-line + normalize `]]()` → `]](#cited)`); source-icon gap filled by a public-favicon `expo-image` + + generic fallback. + +### Approach B — Streaming-Robustness / Performance-First: incremental processor + marker transform +Center on a pure, incremental `citationProcessor.ts` (ref-based `useCitationProcessor`, process-only- +new via `nextPacketIndex`, in-place mutation, primitive change-proxies `citationCount`/`docCount`/ +`resolveVersion`) so a 2000-token / 30-citation answer never re-parses; the Sources bar/sheet are +memoized siblings that re-render only when counts change, never per token. A pure +`transformCitationMarkers(displayed, resolve)` rewrites `[[n]](url)` → a controlled `onyxcite://n` +href when resolved, elides the partial trailing marker, and downgrades forward-refs to inert plain +text; tap routing resolves `n → doc → openDocument`. +- **Trade-offs:** deterministic streaming edges + O(new-packets) processing + memoized UI; but the + robustness machinery is **largely undercut by the pre-baked-URL fact** (doc packets precede the + answer and the marker carries its own URL, so forward-ref gating buys little), and the + `onyxcite://` rewrite adds a fragile per-frame transform. +- **Risks:** renderer may filter a custom `onyxcite://` scheme in `onLinkPress`; partial-marker + regex edge cases (`[not a cite]`, real `[[x]](y)` links); superscript may not be expressible. + +### Approach C — Flexibility / Reusable-Foundation: one processor + a `processed` channel + shared source layer +Introduce the minimal durable seams 9b–9e will reuse: (a) a pure incremental `messageProcessor.ts` +(mobile port of web's single `packetProcessor`: cursor + reset-on-shrink) that 9a fills with +`citationMap`/`citations[]`/`documentMap` and **9b extends** with grouping/steps; (b) extend +`usePacketDisplay` to host the processor and `MessageRendererProps` to carry `processed` state (not +just raw packets) — the channel 9b's timeline/search/tool renderers read; (c) a shared source layer +— `SearchDoc` contract, `SourceIcon`/`WebResultIcon`, `SourceRow`, `openSource` router — reused by +9a's sheet **and** 9b's search/fetch sub-renderers. Grouping itself is **not** built now (that's 9b); +9a state stays flat/grouping-free so 9b's grouping design stays unconstrained. Inline uses the same +`onLinkPress` → `resolveCitationHref` → `openSource`. +- **Trade-offs:** 9b plugs in with zero renderer-plumbing rewrite (web proves the one-processor + path) and `SourceRow`/`SourceIcon`/`SearchDoc`/`openSource` are each reused 3–4× across 9a–9b; + upfront cost is one small pure module + a hook change + a prop addition + a few components. Each + seam is justified by a concrete, imminent 9b consumer (not speculative). +- **Risks:** a wrong guess about 9b's grouping shape (minimized — 9a commits nothing about + turn/tab grouping); `SearchDoc` under-modelling (mitigation: port the full shape now); + `resolveCitationHref` is the single fragile mapping (isolated + unit-tested). + +## Cross-comparison + +- **Inline tap-path:** A's insight (pre-baked marker URL + doc-packets-precede-answer) makes the + common web-doc case trivial and **undercuts B's forward-ref/`onyxcite://` machinery** — B's + genuinely useful core (an incremental cursor) is already present in C (and in web). So B's extras + mostly don't pay for themselves at chat scale. +- **Alignment with the roadmap:** 9b is **next** and explicitly needs a packet processor to extend + with grouping + search/fetch renderers that render source rows of `SearchDoc`s — i.e. exactly C's + shared layer + `processed` channel. A ports citations ad-hoc into `MessageTextRenderer`, + diverging from web's one-processor structure and forcing a re-home when 9b lands. +- **Extraction policy tension:** the mobile policy is "extract-on-proven-reuse, not upfront." C's + reuses are **named and imminent** (9b), not speculative — so C is not over-engineering; but if 9b + were far off, A would be the honest call. +- **Failure modes:** all three share the empty-parens `[[n]]()` file-marker risk and the + source-icon gap; C additionally carries a (minimized) grouping-shape guess; B additionally + carries the custom-scheme + regex risks. + +## Chosen approach + +**Approach C (Reusable Foundation), adopting Approach A's inline simplification.** Selected at +GATE 1. + +Rationale: C matches web's actual structure — one incremental packet processor + a `processed` +state channel to renderers — and **9b (agentic timeline) is the very next phase** and needs +exactly this: a processor to extend with turn/tab grouping, and a shared source layer +(`SearchDoc` + `SourceIcon`/`WebResultIcon`/`SourceRow`/`openSource`) that 9b's search/fetch +sub-renderers reuse. A would force a re-home of the citation logic when 9b lands; B builds +streaming-robustness machinery that the pre-baked-marker-URL fact largely renders unnecessary. + +**Grafted from A (the inline simplification):** the inline tap-path does **not** resolve citation +state — since the marker URL is pre-baked (`[[n]](link)`, `link = search_doc.link or ""`), +`onLinkPress(event.url)` opens web/linked docs directly, and empty-URL file markers degrade to +opening the Sources sheet. We therefore **drop B's forward-reference gating and the +`onyxcite://` marker-rewrite transform** entirely. + +Net shape: C's reusable seams (processor + `processed` channel + shared source layer) + A's lean +inline path + none of B's rewrite/gating complexity. + +**Deferred to 9b (explicitly NOT built now):** turn/tab packet **grouping**, timeline steps, and +the reasoning/search/tool sub-renderers. 9a's processor state stays flat (`citationMap`, +`citations[]`, `documentMap`, completion) so 9b's grouping design remains unconstrained. + diff --git a/docs/mobile-chat/9a-citations/02-high-level-design.md b/docs/mobile-chat/9a-citations/02-high-level-design.md new file mode 100644 index 0000000000..459212f9bd --- /dev/null +++ b/docs/mobile-chat/9a-citations/02-high-level-design.md @@ -0,0 +1,170 @@ +> Status: active · Task: 9a-citations · Approach: C (Reusable Foundation) + A's inline simplification + +# Mobile Chat 9a — Citations & Cited Sources — High-Level Design + +## What it does + +When the assistant answers using retrieved documents, the answer contains inline `[N]` citation +markers and the turn has a set of cited/found source documents. This feature makes the mobile chat +(1) render those `[N]` markers as tappable links that open the cited document, and (2) show a +"Sources" button under a completed answer that opens a bottom sheet listing the source documents +(title, source, snippet), each tappable to open. It also lays the small, reusable packet-processing ++ source-UI foundation that the next rich-chat phase (9b, the agent timeline) builds on. + +## How it works (end-to-end walkthrough) + +The backend already streams the pieces we need, mixed into the same NDJSON packet stream the chat +already consumes: +- **`citation_info`** packets — `{citation_number, document_id}` — one per first-cite, arriving just + before the answer text that contains the marker. +- **document** packets — `search_tool_documents_delta` and `open_url_documents` — each carrying a + list of `SearchDoc`s, emitted by the search/URL tools *before* the answer. +- **`message_start.final_documents`** — the authoritative cited-doc set for the turn. +- The inline marker itself is already baked into the answer text as a markdown link + **`[[N]](link)`** where `link = search_doc.link or ""` — so for a normal web/document source the + marker's URL *is* the document link. + +The mobile chat controller already stores **every** packet on the assistant message +(`node.packets`) without inspecting its type, and `usePacketDisplay` already hands all of a node's +packets to the one matched renderer. So nothing new is needed to *receive* citation/document +packets — the work is to *process* and *render* them: + +1. **Type the new packets.** Add the three packet shapes + a `SearchDoc` contract so the stream is + strongly typed (purely additive to `streamingModels.ts`). +2. **Process them into state.** A new pure processor (`messageProcessor.ts`, a small mobile port of + web's `packetProcessor`) folds the packets into a `ProcessedMessageState`: a `citationMap` + (`{[N]: document_id}`), a deduped ordered `citations[]`, a `documentMap` + (`document_id → SearchDoc`), and completion (`isComplete`/`stopReason`). The processor is + incremental-capable (a cursor + reset when the packet array shrinks), but 9a drives it with a + full pass per flush (see step 3), so the cursor isn't relied upon yet — 9b can host it + incrementally. +3. **Host the processor.** `usePacketDisplay` is the mobile analog of web's `usePacketProcessor`: + it recomputes `processed` with a `useMemo` (from a fresh `createInitialState`, a full pass over + `node.packets`) whenever the packet array changes — cheap at chat scale, and lint-clean (the + `react-hooks/refs` rule forbids the ref-during-render pattern web uses). It returns + `{ renderer, packets, processed }`; this `processed` value is the **channel** that both this + phase's Sources UI and 9b's timeline renderers read. +4. **Render inline markers.** The answer keeps flowing through the existing + `MessageTextRenderer → useTypewriter → StreamingMarkdown` path. `StreamingMarkdown` gains an + `onLinkPress` prop wired to the native markdown renderer's tap callback. When a `[N]` link (or + any markdown link) is tapped, the handler opens its URL in the in-app browser + (`expo-web-browser`); an empty-URL marker (a file source with no link) is a no-op (those sources + are reachable via the Sources sheet). No custom inline component is needed — the marker is a + styled, tappable link. +5. **Render the Sources surface.** Once the answer is complete and the turn has any + citations/documents, `MessageRow` renders a **"Sources" button** (a small stack of ≤3 source + icons + a count) under the answer. Tapping it opens a **bottom-sheet Modal** listing the sources + in three sections — **Cited Sources** (in citation order), **More** (found-but-not-cited), and + **User Files** — each a **source row** (icon/favicon + title + source/updated-at + snippet). A + row tap opens that document (browser for linked docs; a graceful no-op/toast for file docs, which + have no mobile preview yet). + +## Component interaction + +``` + NDJSON stream ─► useChatController (stores every packet on node.packets) [UNCHANGED] + │ + ▼ + MessageRow.AssistantMessage + │ const { renderer, packets, processed } = usePacketDisplay(node) [MODIFIED: hosts processor] + │ └─ useMemo: processPackets(createInitialState(nodeId), packets) [NEW · FOUNDATION] + │ → processed { citationMap, citations[], documentMap, isComplete, stopReason } + │ + ├─► [shell, unchanged] + │ + ├─► [contract +processed] + │ └─ MessageTextRenderer [MODIFIED] + │ accumulateContent → useTypewriter → [MODIFIED] + │ └─ onLinkPress(url) → openUrl(url) + │ + └─► if processed.isComplete && hasSources(processed): [NEW · 9a] + + + └─ sections Cited / More / User Files + └─ openSource(doc)}/> [NEW · FOUNDATION] + └─ (favicon | file-text) [NEW · FOUNDATION] + └─ openSource(doc): link→browser | file→toast/no-op | none→no-op [NEW · FOUNDATION] +``` + +## Key components + +- **`messageProcessor.ts`** — pure incremental packet→state processor (NEW · FOUNDATION; 9b extends + it with grouping/steps). +- **`contracts/documents.ts`** — `SearchDoc`, `StreamingCitation`, `CitationMap` types (NEW · + FOUNDATION; 9b's search/fetch renderers reuse `SearchDoc`). +- **`usePacketDisplay.ts`** — hosts the processor, returns `processed` (MODIFIED · FOUNDATION — the + channel 9b renderers read). +- **`registry.ts`** — `MessageRendererProps` carries `processed` instead of bare `isComplete` + (MODIFIED · FOUNDATION). +- **`openSource.ts`** — resolve a `SearchDoc` to an action + execute it (NEW · FOUNDATION; 9b result + rows + 9c "view source" reuse). +- **`SourceIcon.tsx` / `SourceRow.tsx`** — a source's icon and list row (NEW · FOUNDATION; 9b renders + the same rows inline). +- **`citations.ts`** — pure `selectSources(processed)` (Cited/More/Files split) + `domainOf`/ + `faviconUrl` helpers (NEW · 9a). +- **`CitedSources.tsx`** — the "Sources" bar + the bottom-sheet list (NEW · 9a). +- **`MessageTextRenderer.tsx` / `StreamingMarkdown.tsx`** — inline `onLinkPress` wiring (MODIFIED · 9a). +- **`MessageRow.tsx`** — reads `processed`; renders the Sources footer (MODIFIED · 9a). +- **`streamingModels.ts`** — the three new packet types + `MessageStart.final_documents` (MODIFIED). + +## End-to-end scenario + +1. User asks a question that triggers a search. The search tool emits `search_tool_documents_delta` + with 6 `SearchDoc`s → the processor fills `documentMap` with all 6. +2. `message_start` arrives with `final_documents` (the cited subset) → processor upserts them + (already present) and marks the answer as coming. +3. The answer streams: `citation_info {citation_number:1, document_id:"d1"}` then the delta text + `… as reported [[1]](https://acme.com/report) …`. The processor sets `citationMap[1]="d1"` and + pushes `{citation_num:1, document_id:"d1"}` to `citations[]`. The typewriter reveals the text; + `[1]` shows as a styled link. +4. User taps `[1]` → `onLinkPress("https://acme.com/report")` → opens the in-app browser on that + page. (No citation-state lookup needed — the URL was baked into the marker.) +5. `stop` arrives → `processed.isComplete = true`. `MessageRow` now renders the **Sources** button + showing 2 stacked favicons + "Sources · 6". +6. User taps **Sources** → the bottom sheet opens: **Cited Sources** lists the 1–2 docs referenced + in the answer (citation order); **More** lists the other found docs; each row shows a favicon, + the document title, its domain + updated date, and a 2-line snippet. Tapping a row opens that + document in the in-app browser. + +## Sequence of key operations + +1. Controller appends each streamed packet to `node.packets` (unchanged). +2. `MessageRow.AssistantMessage` calls `usePacketDisplay(node)`. +3. `usePacketDisplay` recomputes `processed` via `useMemo` — a full pass of `messageProcessor` over + `node.packets` (citationMap, citations, documentMap, isComplete) whenever the array changes. +4. The text renderer renders the answer markdown with an `onLinkPress` handler. +5. On marker/link tap → open the URL in the in-app browser (or no-op for empty-URL file markers). +6. On `stop`/complete → `MessageRow` shows the Sources button when `hasSources(processed)`. +7. On Sources tap → open the sheet; `selectSources(processed)` splits docs into Cited/More/Files. +8. On a source-row tap → `openSource(doc)` opens the doc (browser / toast). + +## Key decisions & why + +- **Mobile builds its own small processor (not shared with web).** Web's `usePacketProcessor`/ + `packetProcessor` stays web-only (per roadmap); the mobile chat layer is native by decision. We + port only the incremental-cursor + reset-on-shrink shape — the part 9b needs to extend. (Ref: + `01-research.md` web-structure + roadmap PR 2 decision.) +- **Inline markers are styled tappable links, not custom chips.** The native markdown renderer + (`react-native-enriched-markdown`) exposes only `markdownStyle` + `onLinkPress` — no custom node + hook — so inline chip components are impossible without heavy, fragile text-segmentation. The + backend bakes the doc URL into the marker, so a plain `onLinkPress(url)` gives full behavior + parity. (Ref: `01-research.md` Q3 + the `citation_processor.py:496,506` fact.) +- **The rich UI lives in the Sources sheet.** Custom components (icons, rows, sections) have no + renderer constraint there, so that's where web-faithful structure goes — mirroring web's own + *mobile* DocumentsSidebar, which is already a bottom Modal titled "Sources." +- **Establish the `processed` channel + shared source layer now (Approach C).** 9b (the very next + phase) needs a processor to extend with grouping and needs `SearchDoc`/`SourceRow`/`SourceIcon`/ + `openSource` for its search/fetch sub-renderers. Building these minimal seams now (each justified + by a concrete 9b consumer) avoids a rewrite; grouping itself is deliberately **not** built yet. +- **Drop the streaming-robustness machinery (reject B).** Because document packets precede the + answer and the marker carries its own URL, forward-reference gating and a marker-rewrite transform + buy almost nothing at chat scale — so they're omitted. + +## What existing behavior changes + +- Inline `[N]` markers (and any markdown links) in assistant answers become **tappable** and open in + the in-app browser; today nothing opens content links. No change to how text/markdown looks + otherwise. +- A **"Sources" button + sheet** appears under completed answers that have citations/documents. +- No change to user messages, errors, the timeline shell, streaming, or any non-answer surface. +- No backend, DB, or API changes. diff --git a/docs/mobile-chat/9a-citations/03-detailed-design.md b/docs/mobile-chat/9a-citations/03-detailed-design.md new file mode 100644 index 0000000000..ba48d22388 --- /dev/null +++ b/docs/mobile-chat/9a-citations/03-detailed-design.md @@ -0,0 +1,269 @@ +> Status: active · Task: 9a-citations + +# Mobile Chat 9a — Citations & Cited Sources — Detailed Design + +## Database design + +**N/A — client-only feature.** No backend, schema, or API changes. All packet types already exist +on the wire; 9a only consumes them on the mobile client. + +## Class / interface design + +### `ProcessedMessageState` (new · FOUNDATION) — `mobile/src/chat/messageProcessor.ts` +The incremental processor state for one assistant message. 9a fields only; deliberately +grouping-free so 9b can add `groupedPacketsMap`/`toolGroups`/steps without reshaping what exists. + +```ts +interface ProcessedMessageState { + nodeId: number; + nextPacketIndex: number; // cursor — process-only-new + citationMap: Record; // { citation_number: document_id } + citations: StreamingCitation[]; // deduped, first-cite order + seenCitationDocIds: Set; // dedup guard + documentMap: Map; // document_id → doc + isComplete: boolean; // saw MESSAGE_END or STOP + stopReason?: StopReason; +} + +function createInitialState(nodeId: number): ProcessedMessageState; +function processPackets(state: ProcessedMessageState, rawPackets: Packet[]): ProcessedMessageState; +``` + +- `processPackets`: if `nextPacketIndex > rawPackets.length` → `createInitialState(nodeId)` + (array replaced by a *shorter* one); else loop `[nextPacketIndex, len)` and dispatch each, then set + `nextPacketIndex = len`. Mutates `citationMap`/`documentMap`/`citations` in place (web pattern); + returns the same object. + - **Reset caveat:** this shrink check only catches a *shorter* replacement. A same-length or + longer regenerate/history-load would reuse stale state under an incremental host. 9a sidesteps + this entirely — `usePacketDisplay` passes a *fresh* `createInitialState` every render (useMemo), + so nothing is reused. A future incremental host (9b) must reset on packet-array *identity* + change, not just length. +- Per-packet dispatch (by `obj.type`): + - `CITATION_INFO` → `citationMap[n] = document_id`; if `!seenCitationDocIds.has(document_id)` → + add + push `{ citation_num, document_id }`. + - `SEARCH_TOOL_DOCUMENTS_DELTA` / `OPEN_URL_DOCUMENTS` → for each doc with a `document_id`, + `documentMap.set(document_id, doc)`. + - `MESSAGE_START` → if `final_documents`, upsert each into `documentMap`. + - `MESSAGE_END` / `STOP` → `isComplete = true`; `STOP` also captures `stopReason`. + - anything else → ignored (text/section_end/error handled elsewhere). + +### `SearchDoc` / `StreamingCitation` / `CitationMap` (new · FOUNDATION) — `mobile/src/chat/contracts/documents.ts` +Mobile-native port of the backend `SearchDoc` (full field set — 9b's search renderer needs the +extras). Types only. + +```ts +interface SearchDoc { + document_id: string; + semantic_identifier: string; + link: string | null; + blurb: string; + source_type: string; + score: number | null; + updated_at: string | null; // ISO-8601 + match_highlights: string[]; + metadata: Record; + is_internet: boolean; + chunk_ind: number; + boost: number; + hidden: boolean; + primary_owners: string[] | null; + secondary_owners: string[] | null; + is_relevant: boolean | null; + relevance_explanation: string | null; + file_id: string | null; +} +interface StreamingCitation { citation_num: number; document_id: string; } +type CitationMap = Record; +``` + +### `SourceTarget` + `openSource` (new · FOUNDATION) — `mobile/src/chat/openSource.ts` +Pure resolver + thin effectful executor. One router for inline taps and source rows. + +```ts +type SourceTarget = + | { kind: "browser"; url: string } + | { kind: "file"; fileId: string } + | { kind: "none" }; + +function documentTarget(doc: SearchDoc): SourceTarget; // link → browser; file_id & !link → file; else none +function openUrl(url: string): void; // WebBrowser.openBrowserAsync (in-app) +function openSource(doc: SearchDoc): void; +``` + +- `openSource`: `documentTarget(doc)` → `browser` = `openUrl`; `file` = a heads-up toast via the + global `toast` helper ("Preview isn't available on mobile yet") — no mobile doc viewer in 9a; + `none` = no-op. (No caller callback — the toast is fired internally, so `SourceRow` just calls + `openSource(doc)`.) + +### `PacketDisplay` (modified · FOUNDATION) — `mobile/src/hooks/usePacketDisplay.ts` +```ts +interface PacketDisplay { + renderer: MessageRenderer | null; + packets: Packet[]; + processed: ProcessedMessageState; // replaces the old top-level `isComplete` +} +``` +Hosts the processor and returns `processed`; `renderer` via `findRenderer(packets)`. + +> **Implementation note (deviation from the ref design):** the mobile `react-hooks/refs` lint rule +> forbids reading/writing `ref.current` during render (web's `usePacketProcessor` does exactly +> that). So 9a hosts the processor via `useMemo(() => processPackets(createInitialState(nodeId), +> packets), [nodeId, packets])` — a full pass per flush (cheap at chat scale) instead of a +> render-mutated incremental ref. The `messageProcessor` module stays incremental-capable +> (cursor + reset-on-shrink) for 9b, which can host it via a lint-compatible incremental pattern. + +### `MessageRendererProps` (modified · FOUNDATION) — `.../renderers/registry.ts` +```ts +interface MessageRendererProps { + packets: Packet[]; + processed: ProcessedMessageState; // was: isComplete: boolean → now processed.isComplete +} +``` + +## New files + +| File | Responsibility | +|------|----------------| +| `mobile/src/chat/messageProcessor.ts` | Pure incremental packet→`ProcessedMessageState` processor (FOUNDATION). | +| `mobile/src/chat/contracts/documents.ts` | `SearchDoc` / `StreamingCitation` / `CitationMap` types (FOUNDATION). | +| `mobile/src/chat/openSource.ts` | `documentTarget` + `openSource`/`openUrl` (FOUNDATION). | +| `mobile/src/chat/citations.ts` | `selectSources(processed)` split + `domainOf`/`faviconUrl` (9a). | +| `mobile/src/components/chat/SourceIcon.tsx` | Favicon (public `expo-image`) or `file-text` fallback for a doc (FOUNDATION). | +| `mobile/src/components/chat/SourceRow.tsx` | Tappable source row: icon + title + meta + snippet (FOUNDATION). | +| `mobile/src/components/chat/CitedSources.tsx` | `CitedSourcesBar` (button) + `CitedSourcesSheet` (bottom sheet) (9a). | +| `mobile/src/chat/__tests__/messageProcessor.test.ts` | Processor unit tests. | +| `mobile/src/chat/__tests__/citations.test.ts` | `selectSources` + helpers. | +| `mobile/src/chat/__tests__/openSource.test.ts` | `documentTarget` branches. | +| `mobile/src/components/chat/__tests__/SourceRow.test.tsx` | Row render + onPress. | +| `mobile/src/components/chat/__tests__/CitedSources.test.tsx` | Sheet sections + bar visibility. | + +## File structure (tree) + +``` +mobile/src/ +├── chat/ +│ ├── streamingModels.ts (modified: +3 packet types, +MessageStart.final_documents, +ObjTypes) +│ ├── messageProcessor.ts (new · FOUNDATION) +│ ├── openSource.ts (new · FOUNDATION) +│ ├── citations.ts (new · 9a) +│ ├── contracts/ +│ │ └── documents.ts (new · FOUNDATION) +│ └── __tests__/ +│ ├── fixtures.ts (modified: +makePacket/makeCitationPacket/makeSearchDoc) +│ ├── messageProcessor.test.ts (new) +│ ├── citations.test.ts (new) +│ └── openSource.test.ts (new) +├── hooks/ +│ └── usePacketDisplay.ts (modified: host processor, return `processed`) +└── components/chat/ + ├── renderers/ + │ ├── registry.ts (modified: MessageRendererProps → {packets, processed}) + │ └── MessageTextRenderer.tsx (modified: processed.isComplete, onLinkPress) + ├── StreamingMarkdown.tsx (modified: +onLinkPress passthrough) + ├── MessageRow.tsx (modified: read processed, render Sources footer) + ├── SourceIcon.tsx (new · FOUNDATION) + ├── SourceRow.tsx (new · FOUNDATION) + ├── CitedSources.tsx (new · 9a) + └── __tests__/ + ├── SourceRow.test.tsx (new) + └── CitedSources.test.tsx (new) +``` + +## What each file will contain + +- **`streamingModels.ts`** — add `PacketType.CITATION_INFO="citation_info"`, + `SEARCH_TOOL_DOCUMENTS_DELTA="search_tool_documents_delta"`, `OPEN_URL_DOCUMENTS="open_url_documents"`; + interfaces `CitationInfo {citation_number:number; document_id:string}`, + `SearchToolDocumentsDelta {documents: SearchDoc[]}`, `OpenUrlDocuments {documents: SearchDoc[]}`; + add `final_documents?: SearchDoc[] | null` to `MessageStart`; add the three to the `ObjTypes` + union. `import { SearchDoc } from "@/chat/contracts/documents"`. (Do NOT add `citation_start/end` + — backend never emits them.) +- **`messageProcessor.ts`** — `createInitialState`, `processPackets`, the per-type handlers above. + Pure; no React. Discriminates on `obj.type`. +- **`contracts/documents.ts`** — the three types above. No logic. +- **`openSource.ts`** — `documentTarget`, `openUrl` (`expo-web-browser`), `openSource`. +- **`citations.ts`** — `selectSources(processed): { cited: SearchDoc[]; more: SearchDoc[]; + files: SearchDoc[]; hasSources: boolean; iconDocs: SearchDoc[] }`: + - `cited` = `citations` mapped through `documentMap` (skip missing), in citation order, deduped. + - `files` = docs with `file_id` (pulled out of cited/more into their own section). + - `more` = remaining `documentMap` values not in `cited` and not files. + - `iconDocs` = first ≤3 of `cited` (fallback `more`) for the bar's icon stack. + - `hasSources` = `citations.length > 0 || documentMap.size > 0`. + - `domainOf(link)`, `faviconUrl(link)` (public favicon service URL; `null` if no host). +- **`SourceIcon.tsx`** — `SourceIcon({ doc, size=18 })`: if `doc.link` yields an http host → + `` (`expo-image`, public, `onError` → `file-text`); else + ``. (Minimal — full per-connector logo map is out of scope, flagged.) +- **`SourceRow.tsx`** — `SourceRow({ doc, onPress })`: `Card variant="secondary" onPress`; layout: + `[ + {semantic_identifier}]`, + `{domainOf(link) ?? source_type} · {timeAgo(updated_at)}`, + `{(match_highlights[0] ?? blurb).slice(0,200)}`. +- **`CitedSources.tsx`** — two exports: + - `CitedSourcesBar({ iconDocs, count, onPress })` — a `Pressable`/`Button`-style pill: a stack of + ≤3 `SourceIcon`s (overlapping) + `Text` "Sources · {count}". `accessibilityRole="button"`. + - `CitedSourcesSheet({ visible, onClose, processed })` — bottom-sheet `Modal` mirroring + `FilePickerSheet` chrome (scrim `Pressable`, inner `rounded-t-24 … px-16 pt-16`, safe-area + bottom, header "Sources" + close `SvgX`, `ScrollView max-h`). Body = `selectSources(processed)` + → up to three labeled sections (`Cited Sources` / `More` / `User Files`) each a `Separator` + + `Text` header + `SourceRow`s; a row's `onPress` = `openSource(doc)`. +- **`usePacketDisplay.ts`** — replace the `useMemo` body: `const stateRef = + useRef(createInitialState(node.nodeId))`; if `stateRef.current.nodeId !== node.nodeId` or shrink → + reseed; `stateRef.current = processPackets(stateRef.current, node.packets)`; `renderer = + useMemo(() => findRenderer(node.packets), [node.packets])`; return `{ renderer, packets: + node.packets, processed: stateRef.current }`. +- **`registry.ts`** — change `MessageRendererProps` to `{ packets; processed }`. +- **`MessageTextRenderer.tsx`** — read `processed.isComplete` (was `isComplete`); build + `onLinkPress = useCallback((url) => { if (url) openUrl(url); }, [])`; pass to `StreamingMarkdown`. + (`matches` unchanged.) +- **`StreamingMarkdown.tsx`** — add `onLinkPress?: (url: string) => void`; pass + `onLinkPress={(e) => onLinkPress?.(e.url)}` to ``. +- **`MessageRow.tsx`** — `AssistantMessage`: destructure `processed` from `usePacketDisplay`; use + `processed.isComplete` for `hasContent`/`AgentTimeline isLoading`; after ``, when + `processed.isComplete && hasSources`, render `` + `` with a + local `sheetVisible` state. Update the `memo` comparator only if needed (still keyed on + `packets.length`, which advances with new citation/doc packets). +- **`fixtures.ts`** — `makePacket(obj, placement?)`, `makeCitationPacket(n, docId)`, + `makeSearchDoc(overrides)`, `makeSearchDocsPacket(docs, type?)`. + +## Integration points + +- **`useChatController.ts`** — unchanged; already stores every wrapped packet on `node.packets`. +- **`chatHistory.ts`** — unchanged; `processRawChatHistory` already loads historical `packets` per + assistant turn → the processor rebuilds citation state on load. +- **`api/chat/stream.ts`** — unchanged; `isPacket` already routes wrapped citation/document packets. +- **`registry.ts` `RENDERERS`** — unchanged (still `[MessageTextRenderer]`); citation/doc packets + ride the same array and `MessageTextRenderer.matches` still fires. 9b appends new renderer entries. +- **`expo-web-browser`** — already a dependency (used by auth SSO); new usage in `openSource`. +- **`timeAgo`** — reuse `mobile/src/lib/time.ts` (already used by project files) for the row date. + +## Important notes before implementation + +- **The empty-parens `[[n]]()` file marker is the key edge case.** For a file/internal source + (`link == ""`), the marker may not render as a tappable link in enriched-markdown. 9a's behavior: + the inline tap is best-effort (no-op if no URL); the file source is reliably reachable in the + **Sources sheet** under "User Files." **Verify on a device build** whether `[[n]]()` renders as + text or a link; if a tappable-but-empty link is desired, a 1-line normalize (`]]()` → a sentinel + href) is the fallback — but keep it out of the default path. +- **`onLinkPress` fires for ALL links**, not just citations (ordinary markdown links in answers too). + That's intended — open them in the in-app browser. Confirm the exact `event` payload shape + (`{ url }`) on device. +- **Favicons are public** — use plain `expo-image`, NOT `BearerImage` (favicon URLs aren't + auth-keyed). `onError` must fall back to the generic icon so a missing favicon never breaks a row. +- **Source-icon coverage is intentionally minimal** (favicon or `file-text`). Mobile has no + per-connector logo set; porting the full ~40-source map is out of scope for 9a (flag as a + follow-up; 9b may expand the map). +- **Bar visibility mirrors web**: show only when `processed.isComplete && hasSources` — avoids + mid-stream layout shift; sources are populated by the time the answer completes. +- **In-place mutation + stable ref**: the processor mutates its state and `usePacketDisplay` returns + the same ref each render. `MessageRow`/`AssistantMessage` re-render on each packet flush (memo on + `packets.length`), so `processed` reads are fresh; do NOT `React.memo` a Sources child on + `processed` identity — compare primitive proxies (`citations.length`, `documentMap.size`) if + memoization is later needed (the 9b perf path). +- **Reset semantics**: reset the processor on `nodeId` change (new message) and on packet-array + shrink (regenerate / history replace) — mirrors web; without it a regenerate would double-count. +- **Route collision**: do NOT touch `/sources/[id]` (project files). The cited-sources surface is a + Modal, no route. +- **Testing**: processor + selector + `documentTarget` are pure → unit tests (dedup, ordering, + `final_documents` seeding, reset-on-shrink, file/link/none targets). `SourceRow`/`CitedSourcesSheet` + → RN Testing Library (render sections, onPress → openSource). Mock `expo-web-browser` + + `expo-image`. Import jest globals from `@jest/globals`; use `@/components/ui/text` in test + assertions. diff --git a/docs/mobile-chat/9a-citations/04-implementation-plan.md b/docs/mobile-chat/9a-citations/04-implementation-plan.md new file mode 100644 index 0000000000..ebb7d997ec --- /dev/null +++ b/docs/mobile-chat/9a-citations/04-implementation-plan.md @@ -0,0 +1,170 @@ +> Status: active · Task: 9a-citations + +# Mobile Chat 9a — Citations & Cited Sources — Implementation Plan + +## Issues to Address + +The mobile chat renders assistant answers as plain markdown with **no citation support**: inline +`[N]` markers are inert text and there is no cited-sources surface. Web shows inline citation chips +plus a "Sources" panel; mobile shows neither (confirmed clean slate — no citation packet types, no +processing, no sources UI). This change ports web's citation **behavior** to mobile: + +1. Type + process the citation/document streaming packets already arriving in `node.packets`. +2. Make inline `[N]` markers (and ordinary answer links) **tappable**, opening the source in the + in-app browser. +3. Render a **"Sources" button** under completed answers that opens a **bottom sheet** listing the + cited/found source documents. Tapping a linked doc opens it in the in-app browser; a file / + link-less doc has no mobile preview yet, so it shows a heads-up toast (files) or no-ops. +4. Establish the minimal, reusable packet-processing + source-UI **foundation** that PR 9b (agent + timeline) extends — without building grouping now. + +Out of scope (later phases): inline chip components / hover cards (platform-blocked — see Notes), +turn/tab packet grouping + timeline steps (9b), a per-connector source-logo set, an in-app document +preview for file sources. + +## Important Notes + +Pulled from `01-research.md` / `02-high-level-design.md` / `03-detailed-design.md`: + +- **Native markdown renderer has no custom-node hook.** `react-native-streamdown` → + `react-native-enriched-markdown` exposes only `markdownStyle` + `onLinkPress`/`onLinkLongPress` + (`StreamdownText` inherits all `EnrichedMarkdownText` props + `remendConfig`). So inline citations + are **styled tappable links**, not custom chips. Wire `onLinkPress` through + `mobile/src/components/chat/StreamingMarkdown.tsx`. +- **The marker URL is pre-baked.** Backend emits `[[{num}]]({link})` with `link = search_doc.link + or ""` (`backend/onyx/chat/citation_processor.py:496,506`). So `onLinkPress(event.url)` opens the + doc directly — **no citation-state lookup for the tap**. Empty-URL `[[n]]()` = file source with no + link (the edge case; reachable via the Sources sheet). +- **Only one citation packet exists:** `CitationInfo {citation_number, document_id}` (`citation_info`). + Web's `citation_start`/`citation_end` are **never emitted** — do not add them. Document packets: + `search_tool_documents_delta` + `open_url_documents`, each `{documents: SearchDoc[]}`. + `message_start` carries `final_documents: SearchDoc[] | null`. Terminal is `stop` (no + `message_end`). Docs arrive **before** the answer; `citation_info` arrives just before its marker + text; dedup per `document_id`. +- **Nothing new is needed to receive packets.** `useChatController` stores every wrapped packet on + `node.packets`; `processRawChatHistory` loads historical packets; `usePacketDisplay` already hands + all packets to the one matched `MessageTextRenderer`. So the controller/history/stream/registry- + list are untouched. +- **Reuse:** `FilePickerSheet.tsx` = the bottom-sheet Modal pattern to mirror; `Card`/`LineItemButton`/ + `Separator`/`Spinner`/`Text`/`Icon`/`Button` primitives; `timeAgo` from `mobile/src/lib/time.ts`; + `expo-web-browser` `openBrowserAsync` (already a dep) for opening links; `expo-image` for + **public** favicons (NOT `BearerImage`). **Gap:** no `SourceIcon`/`WebResultIcon`/source-logo set + on mobile — build a minimal favicon-or-`file-text` version. +- **Foundation seams (Approach C), each justified by a concrete 9b consumer:** the incremental + `messageProcessor` (9b extends with grouping), the `processed` channel on `usePacketDisplay` + + `MessageRendererProps` (9b's renderers read it), and `SearchDoc`/`SourceRow`/`SourceIcon`/ + `openSource` (9b's search/fetch sub-renderers reuse them). Grouping itself is deferred to 9b; 9a + state stays flat. +- **Constraints:** mobile spacing classes are pixels; all text via `@/components/ui/text`; icons via + `@/icons/*` + `Icon`; semantic color classes only; no `dark:`. Do not touch `/sources/[id]` (that + route is the project-files screen) — the cited-sources surface is a Modal, no route. + +## Implementation Strategy + +Ordered so each step is a coherent, compilable change; the pure/foundation layers land before the UI +that consumes them. + +1. **Packet contracts.** Add `mobile/src/chat/contracts/documents.ts` (`SearchDoc`, + `StreamingCitation`, `CitationMap`). Extend `mobile/src/chat/streamingModels.ts`: `PacketType` + members `CITATION_INFO` / `SEARCH_TOOL_DOCUMENTS_DELTA` / `OPEN_URL_DOCUMENTS`; interfaces + `CitationInfo` / `SearchToolDocumentsDelta` / `OpenUrlDocuments`; `MessageStart.final_documents?`; + add all three to the `ObjTypes` union. +2. **Processor (foundation).** Add `mobile/src/chat/messageProcessor.ts` — `ProcessedMessageState`, + `createInitialState`, `processPackets` (incremental cursor, reset-on-shrink, in-place mutation; + handlers for citation/document/message_start/stop). Pure, no React. +3. **Host the processor.** Rework `mobile/src/hooks/usePacketDisplay.ts` into the mobile analog of + web's `usePacketProcessor`: ref-held state, reset on `nodeId` change + array shrink, process new + packets each render, return `{ renderer, packets, processed }` (drop top-level `isComplete`). +4. **Renderer contract.** Change `MessageRendererProps` in + `mobile/src/components/chat/renderers/registry.ts` to `{ packets, processed }`. +5. **Inline tap-routing.** Add `mobile/src/chat/openSource.ts` (`documentTarget`, `openUrl`, + `openSource`). Add `onLinkPress` passthrough to `StreamingMarkdown.tsx`. In + `MessageTextRenderer.tsx`, read `processed.isComplete` and pass an `onLinkPress` that opens the + URL in-browser. +6. **Source presentation layer (foundation).** Add `SourceIcon.tsx` (favicon via `expo-image` + + `file-text` fallback) and `SourceRow.tsx` (icon + title + domain·date + 2-line snippet, `onPress` + → `openSource`). Add `mobile/src/chat/citations.ts` (`selectSources(processed)` split into + Cited/More/User-Files + `iconDocs`; `domainOf`/`faviconUrl`). +7. **Sources surface (9a).** Add `CitedSources.tsx` — `CitedSourcesBar` (icon stack + "Sources · N") + and `CitedSourcesSheet` (bottom-sheet Modal mirroring `FilePickerSheet`, three sections). +8. **Wire into the message.** In `MessageRow.tsx` `AssistantMessage`: consume `processed`, use + `processed.isComplete` for the `AgentTimeline` loading + `hasContent`, and render the Sources bar + + sheet (local `visible` state) after the renderer when `processed.isComplete && hasSources`. +9. **Fixtures + tests** (see Tests). + +## Tests + +Primary type: **unit tests (jest-expo, RN Testing Library)** — this is a client rendering feature +with pure processing logic; no backend/integration surface. Add `makePacket` / `makeCitationPacket` +/ `makeSearchDoc(+Packet)` to `mobile/src/chat/__tests__/fixtures.ts`, then: + +- **`messageProcessor.test.ts`** — citation dedup + first-cite ordering; `citationMap` population; + `documentMap` upsert from both document packet types + `message_start.final_documents`; `isComplete` + on `stop`; incremental cursor (no double-count across flushes); reset-on-array-shrink. +- **`citations.test.ts`** — `selectSources` splits Cited (citation order) / More / User Files + correctly and dedupes; `hasSources`/`iconDocs`; `domainOf`/`faviconUrl` edge cases (no host, no + link). +- **`openSource.test.ts`** — `documentTarget` branches: link → browser, file_id+no-link → file, + neither → none (mock `expo-web-browser`). +- **`SourceRow.test.tsx`** — renders title/domain/snippet; favicon vs fallback icon; `onPress` → + `openSource`. +- **`CitedSources.test.tsx`** — bar visibility gate (`isComplete && hasSources`); sheet renders the + three sections from a processed state; row tap routes to `openSource`. + +Gate: `bun run typecheck`, `bun run lint`, `bunx jest`. Device-verify (owner, post-merge) the two +platform unknowns: the `onLinkPress` `event` payload shape and whether `[[n]]()` renders tappable. + +## Plan Challenge Results + +### 1. Extendability & Scalability: PASS +Foundation seams are sized to a named 9b consumer each (processor→grouping, `processed` channel→9b +renderers, `SearchDoc`/`SourceRow`/`SourceIcon`/`openSource`→9b search/fetch rows). The incremental +cursor (web-proven) scales to long answers/many citations without re-parse; a new packet type = +extend the union + one handler + optionally one renderer. 9a state is grouping-free, so 9b *adds* +fields rather than reshaping — no rewrite. + +### 2. Fragility: CONCERN (isolated + mitigated) +The one real brittle point is the inline `onLinkPress` path — it depends on `StreamdownText` +forwarding `onLinkPress` and the `event.url` payload shape (documented, but device-unverified), plus +the empty-parens `[[n]]()` file marker possibly not rendering tappable. Both are **isolated** in +`StreamingMarkdown` + a one-line handler, have the **Sources sheet as a reliable fallback path** to +every source, and are flagged for device-verify (with a kept-out-of-default 1-line normalize +fallback). Secondary: in-place processor mutation + stable ref (documented: use primitive proxies, +never memoize on `processed` identity) and reset-before-process ordering on regenerate (called out). + +### 3. Industry Standard: VERIFIED +Searched RN in-app-browser guidance and RAG streaming-citation patterns. `WebBrowser.openBrowserAsync` +(SFSafariViewController / Chrome Custom Tabs) is the **official Expo recommendation** for opening a +webpage in-app (`Linking.openURL` is for ejecting to the system browser) — matches the plan. The +citation flow (accumulate text, collect search results with rich metadata, map numbered markers to +sources, show the reference list on completion) is the standard pattern across Perplexity / Cohere / +OpenAI Agent API. Sources: https://docs.expo.dev/versions/latest/sdk/webbrowser/ · +https://docs.perplexity.ai/docs/cookbook/articles/streaming-citations/README · +https://docs.cohere.com/docs/rag-citations +- Conscious divergence: industry often shows a **live citation counter during** streaming; we gate + the Sources bar on `isComplete` to avoid mid-stream layout shift (web-parity). Easy to revisit. + +### 4. Fact Check: PASS +Every "standard/recommended" claim is verified: in-app browser via `openBrowserAsync` (Expo docs, +above); "no custom-node hook in enriched-markdown, only `markdownStyle` + `onLinkPress`" (published +API reference, Phase 1); "marker URL pre-baked as `[[n]](link)`, `link = search_doc.link or ""`" +(code-verified `backend/onyx/chat/citation_processor.py:496,506`); "only `citation_info` emitted, no +`citation_start/end`" (code-verified `streaming_models.py`). No unverified assertions remain. + +### 5. Maintainability: PASS +Mirrors web's citation architecture (`packetProcessor → usePacketProcessor → renderer props`) so a +web-familiar dev recognizes it immediately; reuses existing mobile primitives (the `FilePickerSheet` +sheet pattern, `Card`, `LineItemButton`, `Text`/`Icon`/semantic colors); clear file boundaries +(contracts / processor / hook / source layer / sheet). The one ripple is `MessageRendererProps` +gaining `processed` (replacing bare `isComplete`) — a single, well-named contract change. + +### 6. Patch vs. Fix: PROPER FIX +Greenfield feature, not a workaround: no timeout/retry bumps, no error suppression, no loading-state- +for-backend-perf. "Styled tappable links instead of inline chips" is the correct adaptation to a real +platform constraint (native renderer, no custom-node hook) with behavior parity preserved — the +segmentation alternative was evaluated and rejected as fragile, not ignored. The only patch-shaped +item (the `[[n]]()` normalize) is explicitly kept out of the default path. **No patch-vs-fix decision +needed.** + +**Verdict:** all six pass; the sole CONCERN (device-unverified `onLinkPress`) is inherent to any RN +markdown-tap feature, isolated, and covered by the sheet fallback + a flagged device-verify. Proceed. diff --git a/docs/mobile-chat/9a-citations/05-pr-roadmap.md b/docs/mobile-chat/9a-citations/05-pr-roadmap.md new file mode 100644 index 0000000000..81c5f12a1b --- /dev/null +++ b/docs/mobile-chat/9a-citations/05-pr-roadmap.md @@ -0,0 +1,84 @@ +> Status: active · Task: 9a-citations · Source plan: 04-implementation-plan.md + +# Mobile Chat 9a — Citations & Cited Sources — PR Roadmap + +## Overview + +**Single PR** (owner decision at GATE 3 — ship 9a as one chunk rather than splitting the ~1150 LOC +into two). It exceeds the ~500-700 target band but is one coherent vertical feature; the owner +opted to review it together. + +| PR | Title | Est. LOC | Depends on | Key deliverable | +|----|-------|----------|------------|-----------------| +| 1 | `feat(mobile): chat citations + cited sources` | ~1050-1250 | — | Type + process citation/document packets; inline `[N]` (and answer links) open in the in-app browser; a "Sources" button + bottom sheet lists cited/found documents. Establishes the reusable processing + source-UI foundation for 9b. | + +## Sequence (internal build order within the one PR) + +``` +1. contracts/documents.ts + streamingModels.ts (types) +2. messageProcessor.ts (pure processor · FOUNDATION) +3. usePacketDisplay.ts + registry.ts (host processor, `processed` channel · FOUNDATION) +4. openSource.ts + StreamingMarkdown + MessageTextRenderer (inline tap-routing) +5. citations.ts + SourceIcon + SourceRow (source layer · FOUNDATION) +6. CitedSources.tsx + MessageRow wiring (Sources bar & sheet · 9a UI) +7. fixtures + tests +``` + +Build the pure/foundation layers before the UI that consumes them. 9b (agent timeline) later extends +`messageProcessor` with grouping and reuses `SearchDoc`/`SourceRow`/`SourceIcon`/`openSource`. + +--- + +## PR 1 — `feat(mobile): chat citations + cited sources` + +- **Goal:** Ship the full 9a slice — process the citation/document packets already arriving on + `node.packets`, make inline `[N]` markers (and answer links) tappable → in-app browser, and render + a "Sources" button under completed answers that opens a bottom sheet listing the source documents. +- **Scope (in):** + - Packet contracts + types (`SearchDoc`, `CitationInfo`, document packets, + `MessageStart.final_documents`). + - The incremental `messageProcessor` (`citationMap` / `citations[]` / `documentMap` / completion), + hosted in `usePacketDisplay` → `processed`; `MessageRendererProps` carries `processed`. + - `openSource`/`openUrl` + `onLinkPress` passthrough in `StreamingMarkdown`, wired in + `MessageTextRenderer`. + - `selectSources` selector; `SourceIcon` / `SourceRow`; `CitedSourcesBar` + `CitedSourcesSheet`; + footer wiring in `MessageRow`. + - Fixtures + unit/component tests. +- **Out of scope (deferred):** inline chip components / hover cards (platform-blocked); turn/tab + packet **grouping** + timeline steps (9b); a per-connector source-logo set; an in-app document + preview for file sources (they degrade to a toast/no-op); a live mid-stream citation counter. +- **Files:** + | File | New/Modified | Slice | + |------|--------------|-------| + | `mobile/src/chat/contracts/documents.ts` | new | `SearchDoc` / `StreamingCitation` / `CitationMap` | + | `mobile/src/chat/streamingModels.ts` | modified | 3 packet types + `MessageStart.final_documents` + `ObjTypes` | + | `mobile/src/chat/messageProcessor.ts` | new | `ProcessedMessageState`, `createInitialState`, `processPackets` | + | `mobile/src/chat/openSource.ts` | new | `documentTarget` / `openUrl` / `openSource` | + | `mobile/src/chat/citations.ts` | new | `selectSources` + `domainOf` / `faviconUrl` | + | `mobile/src/hooks/usePacketDisplay.ts` | modified | host processor; return `{ renderer, packets, processed }` | + | `mobile/src/components/chat/renderers/registry.ts` | modified | `MessageRendererProps → { packets, processed }` | + | `mobile/src/components/chat/renderers/MessageTextRenderer.tsx` | modified | `processed.isComplete`; `onLinkPress` | + | `mobile/src/components/chat/StreamingMarkdown.tsx` | modified | `onLinkPress` passthrough | + | `mobile/src/components/chat/SourceIcon.tsx` | new | favicon / `file-text` fallback | + | `mobile/src/components/chat/SourceRow.tsx` | new | tappable source row | + | `mobile/src/components/chat/CitedSources.tsx` | new | `CitedSourcesBar` + `CitedSourcesSheet` | + | `mobile/src/components/chat/MessageRow.tsx` | modified | read `processed`; render Sources footer + sheet | + | `mobile/src/chat/__tests__/fixtures.ts` | modified | `makePacket` / `makeCitationPacket` / `makeSearchDoc(+Packet)` | + | `mobile/src/chat/__tests__/messageProcessor.test.ts` | new | dedup/order, documentMap, final_documents, reset, cursor, isComplete | + | `mobile/src/chat/__tests__/citations.test.ts` | new | `selectSources` split/order/dedup + helpers | + | `mobile/src/chat/__tests__/openSource.test.ts` | new | `documentTarget` branches | + | `mobile/src/components/chat/__tests__/SourceRow.test.tsx` | new | render + `onPress` → `openSource` | + | `mobile/src/components/chat/__tests__/CitedSources.test.tsx` | new | sheet sections + bar visibility | +- **Est. size:** ~1050-1250 LOC (production + tests). +- **Depends on:** — +- **Feature-flag state:** N/A — additive; `main` stays releasable. +- **Tests on merge:** unit (jest) + RN Testing Library. Directly covered: citation/document state + building from a mocked packet stream (`messageProcessor`), the `selectSources` split, `documentTarget` + routing, a `SourceRow` tap → `openSource`, and the sheet's section rendering. **Not** exercised by + the unit suite (pending on-device verification): the native `onLinkPress` event shape and the + inline-marker → `openUrl` plumbing through `StreamdownText`, and the `MessageRow` completed-answer + footer gating. Gate: `bun run typecheck`, `bun run lint`, `bunx jest`. +- **Drift checkpoint:** Before/at implementation, **device-verify the `react-native-enriched-markdown` + `onLinkPress` `event` shape** (expected `{ url }`) — the one platform unknown the inline path rests + on (unit tests mock it; a dev build confirms it). Also confirm whether `[[n]]()` file markers render + tappable (the sheet is the reliable fallback regardless). diff --git a/mobile/src/chat/__tests__/citations.test.ts b/mobile/src/chat/__tests__/citations.test.ts new file mode 100644 index 0000000000..d34dc46d21 --- /dev/null +++ b/mobile/src/chat/__tests__/citations.test.ts @@ -0,0 +1,91 @@ +import { describe, expect, it } from "@jest/globals"; + +import { domainOf, faviconUrl, selectSources } from "@/chat/citations"; +import { createInitialState, processPackets } from "@/chat/messageProcessor"; + +import { + makeCitationPacket, + makeSearchDoc, + makeSearchDocsPacket, +} from "./fixtures"; + +describe("domainOf / faviconUrl", () => { + it("extracts the host without a www prefix", () => { + expect(domainOf("https://www.acme.com/a/b")).toBe("acme.com"); + expect(domainOf("http://docs.example.org")).toBe("docs.example.org"); + }); + + it("returns null for empty / non-url input", () => { + expect(domainOf(null)).toBeNull(); + expect(domainOf("")).toBeNull(); + expect(faviconUrl(null)).toBeNull(); + }); + + it("builds a favicon url from the host", () => { + expect(faviconUrl("https://www.acme.com")).toContain("acme.com"); + }); +}); + +describe("selectSources", () => { + it("splits cited (citation order) / more / files and dedupes", () => { + let state = createInitialState(1); + state = processPackets(state, [ + makeSearchDocsPacket([ + makeSearchDoc({ document_id: "d3" }), // uncited + makeSearchDoc({ document_id: "d1" }), + makeSearchDoc({ document_id: "d2" }), + makeSearchDoc({ + document_id: "f1", + file_id: "f1", + link: null, + source_type: "user_file", + }), + ]), + makeCitationPacket(1, "d2"), + makeCitationPacket(2, "d1"), + ]); + + const selected = selectSources(state); + expect(selected.cited.map((d) => d.document_id)).toEqual(["d2", "d1"]); + expect(selected.more.map((d) => d.document_id)).toEqual(["d3"]); + expect(selected.files.map((d) => d.document_id)).toEqual(["f1"]); + expect(selected.count).toBe(4); + expect(selected.hasSources).toBe(true); + expect(selected.iconDocs.map((d) => d.document_id)).toEqual(["d2", "d1"]); + }); + + it("falls back to file docs for the icon stack when there are no web sources", () => { + let state = createInitialState(1); + state = processPackets(state, [ + makeSearchDocsPacket([ + makeSearchDoc({ + document_id: "f1", + file_id: "f1", + link: null, + source_type: "user_file", + }), + ]), + makeCitationPacket(1, "f1"), + ]); + + const selected = selectSources(state); + expect(selected.cited).toEqual([]); + expect(selected.more).toEqual([]); + expect(selected.files.map((d) => d.document_id)).toEqual(["f1"]); + expect(selected.iconDocs.map((d) => d.document_id)).toEqual(["f1"]); + }); + + it("reports no sources for an empty state", () => { + const selected = selectSources(createInitialState(1)); + expect(selected.hasSources).toBe(false); + expect(selected.count).toBe(0); + }); + + it("hides sources when citations have no matching documents", () => { + let state = createInitialState(1); + state = processPackets(state, [makeCitationPacket(1, "missing-doc")]); + const selected = selectSources(state); + expect(selected.count).toBe(0); + expect(selected.hasSources).toBe(false); + }); +}); diff --git a/mobile/src/chat/__tests__/fixtures.ts b/mobile/src/chat/__tests__/fixtures.ts index f6cfc161e3..c3ba246915 100644 --- a/mobile/src/chat/__tests__/fixtures.ts +++ b/mobile/src/chat/__tests__/fixtures.ts @@ -1,5 +1,7 @@ +import { type SearchDoc } from "@/chat/contracts/documents"; import { UserFileStatus, type ProjectFile } from "@/chat/contracts/projects"; import { ChatFileType } from "@/chat/interfaces"; +import { type ObjTypes, type Packet } from "@/chat/streamingModels"; // Shared ProjectFile builder so the file's shape lives in one place across tests. export function makeProjectFile( @@ -16,3 +18,66 @@ export function makeProjectFile( ...overrides, }; } + +export function makePacket(obj: ObjTypes, turnIndex = 0): Packet { + return { placement: { turn_index: turnIndex }, obj }; +} + +export function makeCitationPacket( + citationNumber: number, + documentId: string, +): Packet { + return makePacket({ + type: "citation_info", + citation_number: citationNumber, + document_id: documentId, + }); +} + +export function makeSearchDoc(overrides: Partial = {}): SearchDoc { + return { + document_id: "d1", + semantic_identifier: "Doc One", + link: "https://example.com/doc-one", + blurb: "A blurb.", + source_type: "web", + score: 0.9, + updated_at: "2026-01-01T00:00:00Z", + match_highlights: [], + metadata: {}, + is_internet: true, + chunk_ind: 0, + boost: 0, + hidden: false, + primary_owners: null, + secondary_owners: null, + is_relevant: null, + relevance_explanation: null, + file_id: null, + ...overrides, + }; +} + +export function makeSearchDocsPacket( + docs: SearchDoc[], + kind: "search" | "open_url" = "search", +): Packet { + return makePacket( + kind === "open_url" + ? { type: "open_url_documents", documents: docs } + : { type: "search_tool_documents_delta", documents: docs }, + ); +} + +export function makeMessageStartPacket(finalDocuments?: SearchDoc[]): Packet { + return makePacket({ + type: "message_start", + id: "m", + content: "", + final_documents: finalDocuments ?? null, + }); +} + +export function makeStopPacket(): Packet { + return makePacket({ type: "stop" }); +} diff --git a/mobile/src/chat/__tests__/messageProcessor.test.ts b/mobile/src/chat/__tests__/messageProcessor.test.ts new file mode 100644 index 0000000000..ad9293159f --- /dev/null +++ b/mobile/src/chat/__tests__/messageProcessor.test.ts @@ -0,0 +1,70 @@ +import { describe, expect, it } from "@jest/globals"; + +import { createInitialState, processPackets } from "@/chat/messageProcessor"; + +import { + makeCitationPacket, + makeMessageStartPacket, + makeSearchDoc, + makeSearchDocsPacket, + makeStopPacket, +} from "./fixtures"; + +describe("messageProcessor", () => { + it("builds citationMap and deduped citations in first-cite order", () => { + let state = createInitialState(1); + state = processPackets(state, [ + makeCitationPacket(1, "d1"), + makeCitationPacket(2, "d2"), + makeCitationPacket(1, "d1"), // repeat — deduped + ]); + expect(state.citationMap).toEqual({ 1: "d1", 2: "d2" }); + expect(state.citations).toEqual([ + { citation_num: 1, document_id: "d1" }, + { citation_num: 2, document_id: "d2" }, + ]); + }); + + it("upserts documentMap from both document packet types and final_documents", () => { + let state = createInitialState(1); + state = processPackets(state, [ + makeSearchDocsPacket([makeSearchDoc({ document_id: "d1" })], "search"), + makeSearchDocsPacket([makeSearchDoc({ document_id: "d2" })], "open_url"), + makeMessageStartPacket([makeSearchDoc({ document_id: "d3" })]), + ]); + expect(Array.from(state.documentMap.keys()).sort()).toEqual([ + "d1", + "d2", + "d3", + ]); + }); + + it("marks complete on stop", () => { + let state = createInitialState(1); + expect(state.isComplete).toBe(false); + state = processPackets(state, [makeStopPacket()]); + expect(state.isComplete).toBe(true); + }); + + it("processes only new packets across flushes (no double count)", () => { + let state = createInitialState(1); + const packets = [makeCitationPacket(1, "d1")]; + state = processPackets(state, packets); + state = processPackets(state, packets); // same array, no growth + expect(state.citations).toHaveLength(1); + expect(state.nextPacketIndex).toBe(1); + }); + + it("resets when the packet array shrinks (regenerate / reload)", () => { + let state = createInitialState(1); + state = processPackets(state, [ + makeCitationPacket(1, "d1"), + makeCitationPacket(2, "d2"), + ]); + expect(state.citations).toHaveLength(2); + + state = processPackets(state, [makeCitationPacket(3, "d3")]); // shorter array + expect(state.citations).toEqual([{ citation_num: 3, document_id: "d3" }]); + expect(state.citationMap).toEqual({ 3: "d3" }); + }); +}); diff --git a/mobile/src/chat/__tests__/openSource.test.ts b/mobile/src/chat/__tests__/openSource.test.ts new file mode 100644 index 0000000000..2d5dc6ec44 --- /dev/null +++ b/mobile/src/chat/__tests__/openSource.test.ts @@ -0,0 +1,66 @@ +import { beforeEach, describe, expect, it, jest } from "@jest/globals"; +import * as WebBrowser from "expo-web-browser"; + +import { documentTarget, openSource } from "@/chat/openSource"; +import { toast } from "@/hooks/useToast"; + +import { makeSearchDoc } from "./fixtures"; + +jest.mock("expo-web-browser", () => ({ + openBrowserAsync: jest.fn(() => Promise.resolve()), +})); + +jest.mock("@/hooks/useToast", () => ({ + toast: Object.assign(jest.fn(), { + info: jest.fn(), + error: jest.fn(), + success: jest.fn(), + warning: jest.fn(), + dismiss: jest.fn(), + clearAll: jest.fn(), + }), +})); + +beforeEach(() => { + jest.clearAllMocks(); +}); + +describe("documentTarget", () => { + it("routes a linked doc to the browser", () => { + expect(documentTarget(makeSearchDoc({ link: "https://a.com" }))).toEqual({ + kind: "browser", + url: "https://a.com", + }); + }); + + it("routes a file doc (no link) to file", () => { + expect( + documentTarget(makeSearchDoc({ link: null, file_id: "f1" })), + ).toEqual({ kind: "file", fileId: "f1" }); + }); + + it("routes a doc with neither link nor file to none", () => { + expect( + documentTarget(makeSearchDoc({ link: null, file_id: null })), + ).toEqual({ kind: "none" }); + }); + + it("treats a non-http link as not browsable", () => { + expect( + documentTarget(makeSearchDoc({ link: "ftp://x", file_id: null })), + ).toEqual({ kind: "none" }); + }); +}); + +describe("openSource", () => { + it("opens a linked doc in the in-app browser", () => { + openSource(makeSearchDoc({ link: "https://a.com" })); + expect(WebBrowser.openBrowserAsync).toHaveBeenCalledWith("https://a.com"); + }); + + it("toasts for a file doc with no link", () => { + openSource(makeSearchDoc({ link: null, file_id: "f1" })); + expect(WebBrowser.openBrowserAsync).not.toHaveBeenCalled(); + expect(toast.info).toHaveBeenCalledTimes(1); + }); +}); diff --git a/mobile/src/chat/citations.ts b/mobile/src/chat/citations.ts new file mode 100644 index 0000000000..ebd6619ba8 --- /dev/null +++ b/mobile/src/chat/citations.ts @@ -0,0 +1,63 @@ +// Pure selectors over processed citation state: split the turn's documents into the Sources-sheet +// sections and derive host/favicon strings. No React. + +import { SearchDoc } from "@/chat/contracts/documents"; +import { ProcessedMessageState } from "@/chat/messageProcessor"; + +export interface SelectedSources { + cited: SearchDoc[]; // referenced by the answer, in citation order (non-file) + more: SearchDoc[]; // found but not cited (non-file) + files: SearchDoc[]; // user-uploaded files + iconDocs: SearchDoc[]; // up to 3 for the Sources button icon stack + count: number; // total sources listed in the sheet + hasSources: boolean; +} + +const HOST_RE = /^https?:\/\/([^/?#]+)/i; + +export function domainOf(link: string | null): string | null { + if (!link) return null; + const match = HOST_RE.exec(link); + if (!match) return null; + return match[1].replace(/^www\./i, ""); +} + +// Public favicon service — the URL isn't auth'd, so a plain (non-bearer) image fetch is fine. +export function faviconUrl(link: string | null): string | null { + const host = domainOf(link); + if (!host) return null; + return `https://www.google.com/s2/favicons?sz=64&domain=${host}`; +} + +function isFileDoc(doc: SearchDoc): boolean { + return doc.file_id != null; +} + +export function selectSources(state: ProcessedMessageState): SelectedSources { + const all = Array.from(state.documentMap.values()); + const files = all.filter(isFileDoc); + const fileIds = new Set(files.map((doc) => doc.document_id)); + const citedIds = new Set(state.citations.map((c) => c.document_id)); + + // Cited (non-file) docs, in the order they were first cited. + const cited: SearchDoc[] = []; + for (const citation of state.citations) { + const doc = state.documentMap.get(citation.document_id); + if (doc && !fileIds.has(doc.document_id)) cited.push(doc); + } + const more = all.filter( + (doc) => !isFileDoc(doc) && !citedIds.has(doc.document_id), + ); + + // Fall through cited -> more -> files so a file-only answer still shows a (file) glyph in the bar. + const iconDocs = (cited.length ? cited : more.length ? more : files).slice( + 0, + 3, + ); + const count = cited.length + more.length + files.length; + // From count, not raw citations/docs: citations whose documents never arrived must not render an + // empty "Sources · 0". + const hasSources = count > 0; + + return { cited, more, files, iconDocs, count, hasSources }; +} diff --git a/mobile/src/chat/contracts/documents.ts b/mobile/src/chat/contracts/documents.ts new file mode 100644 index 0000000000..84f8f108db --- /dev/null +++ b/mobile/src/chat/contracts/documents.ts @@ -0,0 +1,34 @@ +// Mobile-native port of the backend `SearchDoc` (the doc object carried by search/fetch document +// packets and `message_start.final_documents`) plus the citation read-models. Full field set — 9b's +// search/fetch sub-renderers consume the extras, so model it once here. + +export interface SearchDoc { + document_id: string; + semantic_identifier: string; + link: string | null; + blurb: string; + source_type: string; + score: number | null; + updated_at: string | null; // ISO-8601 + match_highlights: string[]; + metadata: Record; + is_internet: boolean; + chunk_ind: number; + boost: number; + hidden: boolean; + primary_owners: string[] | null; + secondary_owners: string[] | null; + is_relevant: boolean | null; + relevance_explanation: string | null; + file_id: string | null; +} + +// The deduped, first-cite-ordered citation record (mirrors web's StreamingCitation). Distinct from +// the wire `CitationInfo` packet, which uses `citation_number` (not `citation_num`). +export interface StreamingCitation { + citation_num: number; + document_id: string; +} + +// citation_number -> document_id +export type CitationMap = Record; diff --git a/mobile/src/chat/messageProcessor.ts b/mobile/src/chat/messageProcessor.ts new file mode 100644 index 0000000000..d2cb0a01f8 --- /dev/null +++ b/mobile/src/chat/messageProcessor.ts @@ -0,0 +1,108 @@ +// Pure, incremental packet -> state processor for one assistant message. A small mobile port of +// web's `packetProcessor`: it advances a cursor (`nextPacketIndex`), processes only NEW packets each +// call, and mutates its state in place. 9a fills it with citations + documents; 9b extends it with +// turn/tab grouping + timeline steps, so the shape here stays deliberately flat (grouping-free). + +import { + CitationMap, + SearchDoc, + StreamingCitation, +} from "@/chat/contracts/documents"; +import { + CitationInfo, + MessageStart, + OpenUrlDocuments, + Packet, + PacketType, + SearchToolDocumentsDelta, + Stop, + StopReason, +} from "@/chat/streamingModels"; + +export interface ProcessedMessageState { + nodeId: number; + nextPacketIndex: number; // cursor — process only packets past this index + citationMap: CitationMap; + citations: StreamingCitation[]; // deduped, first-cite order + seenCitationDocIds: Set; // dedups `citations` + documentMap: Map; + isComplete: boolean; // saw MESSAGE_END or STOP + stopReason?: StopReason; +} + +export function createInitialState(nodeId: number): ProcessedMessageState { + return { + nodeId, + nextPacketIndex: 0, + citationMap: {}, + citations: [], + seenCitationDocIds: new Set(), + documentMap: new Map(), + isComplete: false, + stopReason: undefined, + }; +} + +function upsertDocuments( + state: ProcessedMessageState, + documents: SearchDoc[], +): void { + for (const doc of documents) { + if (doc.document_id) state.documentMap.set(doc.document_id, doc); + } +} + +function processPacket(state: ProcessedMessageState, packet: Packet): void { + const obj = packet.obj; + switch (obj.type) { + case PacketType.CITATION_INFO: { + const citation = obj as CitationInfo; + state.citationMap[citation.citation_number] = citation.document_id; + if (!state.seenCitationDocIds.has(citation.document_id)) { + state.seenCitationDocIds.add(citation.document_id); + state.citations.push({ + citation_num: citation.citation_number, + document_id: citation.document_id, + }); + } + break; + } + case PacketType.SEARCH_TOOL_DOCUMENTS_DELTA: + upsertDocuments(state, (obj as SearchToolDocumentsDelta).documents ?? []); + break; + case PacketType.OPEN_URL_DOCUMENTS: + upsertDocuments(state, (obj as OpenUrlDocuments).documents ?? []); + break; + case PacketType.MESSAGE_START: { + const documents = (obj as MessageStart).final_documents; + if (documents) upsertDocuments(state, documents); + break; + } + case PacketType.MESSAGE_END: + state.isComplete = true; + break; + case PacketType.STOP: + state.isComplete = true; + state.stopReason = (obj as Stop).stop_reason; + break; + default: + break; // message_delta / section_end / error / heartbeat: not our concern + } +} + +export function processPackets( + state: ProcessedMessageState, + rawPackets: Packet[], +): ProcessedMessageState { + // Array replaced by a shorter list (regenerate / history reload) -> rebuild from scratch so we + // never double-count a re-streamed turn. + if (state.nextPacketIndex > rawPackets.length) { + state = createInitialState(state.nodeId); + } + for (let i = state.nextPacketIndex; i < rawPackets.length; i++) { + const packet = rawPackets[i]; + if (packet) processPacket(state, packet); + } + state.nextPacketIndex = rawPackets.length; + return state; +} diff --git a/mobile/src/chat/openSource.ts b/mobile/src/chat/openSource.ts new file mode 100644 index 0000000000..8a0052989f --- /dev/null +++ b/mobile/src/chat/openSource.ts @@ -0,0 +1,48 @@ +// Where a source opens. Inline marker taps use `openUrl` directly (the `[[n]](url)` marker already +// carries the doc URL); source-row taps use `openSource`, which also handles link-less file docs. + +import * as WebBrowser from "expo-web-browser"; + +import { SearchDoc } from "@/chat/contracts/documents"; +import { toast } from "@/hooks/useToast"; + +export type SourceTarget = + | { kind: "browser"; url: string } + | { kind: "file"; fileId: string } + | { kind: "none" }; + +const HTTP_RE = /^https?:\/\//i; + +export function isHttpUrl(url: string): boolean { + return HTTP_RE.test(url); +} + +export function documentTarget(doc: SearchDoc): SourceTarget { + if (doc.link && isHttpUrl(doc.link)) + return { kind: "browser", url: doc.link }; + if (doc.file_id) return { kind: "file", fileId: doc.file_id }; + return { kind: "none" }; +} + +// In-app browser (SFSafariViewController / Chrome Custom Tabs) — keeps the user in the app. +export function openUrl(url: string): void { + if (!isHttpUrl(url)) return; + void WebBrowser.openBrowserAsync(url).catch(() => { + toast.error("Couldn't open this link."); + }); +} + +export function openSource(doc: SearchDoc): void { + const target = documentTarget(doc); + switch (target.kind) { + case "browser": + openUrl(target.url); + break; + case "file": + // File/internal source with no public link; no mobile doc preview yet. + toast.info("Preview isn't available on mobile yet."); + break; + case "none": + break; + } +} diff --git a/mobile/src/chat/streamingModels.ts b/mobile/src/chat/streamingModels.ts index 370f67f931..bfec6288df 100644 --- a/mobile/src/chat/streamingModels.ts +++ b/mobile/src/chat/streamingModels.ts @@ -1,5 +1,7 @@ // Core streaming-packet contracts (NDJSON wire shapes). +import type { SearchDoc } from "@/chat/contracts/documents"; + interface BaseObj { type: string; } @@ -11,6 +13,11 @@ export enum PacketType { STOP = "stop", SECTION_END = "section_end", ERROR = "error", + // Rich-chat (9a): citations + the documents that back them. `citation_info` is the ONLY citation + // packet the backend emits (web's citation_start/end are declared but never sent). + CITATION_INFO = "citation_info", + SEARCH_TOOL_DOCUMENTS_DELTA = "search_tool_documents_delta", + OPEN_URL_DOCUMENTS = "open_url_documents", } export interface MessageStart extends BaseObj { @@ -18,6 +25,24 @@ export interface MessageStart extends BaseObj { type: "message_start"; content: string; pre_answer_processing_seconds?: number; + // Authoritative cited-doc set for the turn (present once the answer starts). + final_documents?: SearchDoc[] | null; +} + +export interface CitationInfo extends BaseObj { + type: "citation_info"; + citation_number: number; + document_id: string; +} + +export interface SearchToolDocumentsDelta extends BaseObj { + type: "search_tool_documents_delta"; + documents: SearchDoc[]; +} + +export interface OpenUrlDocuments extends BaseObj { + type: "open_url_documents"; + documents: SearchDoc[]; } export interface MessageDelta extends BaseObj { @@ -60,7 +85,10 @@ export type ObjTypes = | Stop | SectionEnd | PacketError - | ChatHeartbeat; + | ChatHeartbeat + | CitationInfo + | SearchToolDocumentsDelta + | OpenUrlDocuments; export interface Placement { turn_index: number; diff --git a/mobile/src/components/chat/CitedSources.tsx b/mobile/src/components/chat/CitedSources.tsx new file mode 100644 index 0000000000..8e09c73e40 --- /dev/null +++ b/mobile/src/components/chat/CitedSources.tsx @@ -0,0 +1,132 @@ +// The cited-sources surface: a "Sources · N" button under a completed answer, and the bottom-sheet +// list it opens (mirrors web's mobile DocumentsSidebar Modal). Sections: Cited / More / User Files. +import { Modal, Pressable, ScrollView, View } from "react-native"; +import { useSafeAreaInsets } from "react-native-safe-area-context"; + +import { SelectedSources } from "@/chat/citations"; +import { SearchDoc } from "@/chat/contracts/documents"; +import { openSource } from "@/chat/openSource"; +import { SourceIcon } from "@/components/chat/SourceIcon"; +import { SourceRow } from "@/components/chat/SourceRow"; +import { Button } from "@/components/ui/button"; +import { Separator } from "@/components/ui/separator"; +import { Text } from "@/components/ui/text"; +import SvgX from "@/icons/x"; + +interface CitedSourcesBarProps { + iconDocs: SearchDoc[]; + count: number; + onPress: () => void; +} + +export function CitedSourcesBar({ + iconDocs, + count, + onPress, +}: CitedSourcesBarProps) { + return ( + + {iconDocs.length > 0 ? ( + + {iconDocs.map((doc, index) => ( + 0 ? { marginLeft: -6 } : undefined} + > + + + ))} + + ) : null} + + Sources · {count} + + + ); +} + +interface CitedSourcesSheetProps { + visible: boolean; + onClose: () => void; + sources: SelectedSources; +} + +export function CitedSourcesSheet({ + visible, + onClose, + sources, +}: CitedSourcesSheetProps) { + const insets = useSafeAreaInsets(); + const { cited, more, files } = sources; + + const sections = [ + { title: "Cited Sources", docs: cited }, + { title: cited.length ? "More" : "Found Sources", docs: more }, + { title: "User Files", docs: files }, + ].filter((section) => section.docs.length > 0); + + return ( + + {/* Semantic color tokens can't express translucency (bare CSS vars, no alpha channel), so the + dimming scrim uses a raw rgba — the one place a non-token color is warranted. */} + + {/* Stop taps inside the sheet from dismissing it. */} + {}} + className="rounded-t-20 border-t border-border-01 bg-background-tint-00 px-16 pt-16" + style={{ paddingBottom: insets.bottom + 16 }} + > + + + Sources + +