09 — Dual datastore (MongoDB for mobile, DuckLake for web/analytics)

Decision (2026-05-20): the mobile app persists reports to MongoDB; the hosted website and the analytics/knowledge-graph backend stay on DuckLake. Both write paths go through FastAPI (chosen over direct device→Realm), so the privacy contract stays enforced in one place. Mobile reports are then synced into DuckLake so the agents, MCP servers, and cluster detection see a unified dataset.

This plan slots into plan/08-mobile-ux-revamp.md Phase 3 (backend wiring): the polished UI keeps calling createReport() → POST /v1/reports; only the server-side sink changes by channel.

Architecture

            ┌─────────── same api/openapi.yaml contract ───────────┐
 Web build  │  POST /v1/reports                                     │
 (static) ──┤      X-Client-Channel: web    → FastAPI → DuckLake ───┼─► knowledge graph
            │                                   │  (kg_writer)      │   (agents, MCP,
 Mobile     │      X-Client-Channel: mobile  → FastAPI → MongoDB ───┤    cluster scan)
 (Capacitor)│                                   │  (mongo_writer)   │        ▲
            └───────────────────────────────────┼───────────────────┘        │
                       privacy enforcement       │                            │
                       (validate/coarsen/EXIF/   └── mongo→ducklake sync ──────┘
                        triage-guard) runs BEFORE     (idempotent on observation_id)
                        the sink, for BOTH channels

Why Mongo for mobile (given it still goes through FastAPI)

Operational separation: a flexible document store for an evolving mobile payload and a separate high-write ingest path, decoupled from the analytical lakehouse. Analytics unification is preserved by the sync. (If true offline-first ever becomes the priority, the alternative is direct device→Atlas Device Sync — but that moves privacy enforcement off the server and is explicitly not this plan.)

Hosting + sync — recommendation (you’ll confirm)

Recommended: self-hosted MongoDB Community on the VM (new mongodb Ansible role), bound to 127.0.0.1, alongside Postgres. Rationale: the stack is already self-hosted (one-command VM deploy), data stays on the box (privacy), and — because we chose server-mediated writes — we don’t need Atlas Device Sync. Atlas (managed) stays the alternative if you later want managed ops or Device Sync.

Sync: a scheduled mongo_to_ducklake job (systemd timer, every N minutes) that reads new Mongo report docs past a stored watermark (created_at/_id) and writes each into DuckLake via the existing kg_writer.persist_observation (idempotent on observation_id, so re-runs are safe). A change-stream consumer is the lower-latency upgrade path later.

Components to build

FastAPI / agents

Mongo → DuckLake sync

Ansible

App

Phases

Guardrails (unchanged from the contract)

Open decision (please confirm)

Hosting: self-hosted MongoDB CE on the VM (recommended) vs MongoDB Atlas. Everything else above is independent of this choice; Phase B is the only part that branches on it.