Insight
TL;DR: Insight is an offline-first iOS and Android app for people who read physical books. You photograph a page, swipe your finger across the lines you want, and the passage is recognised, tagged, summarised and filed against the book it came from. It all lives on your phone — and sentence embeddings turn a pile of highlights into something you can search by meaning, browse by theme, and ask questions of.
The problem I'm actually solving
A highlight in a paper book is write-only.
You underline a sentence that stops you mid-page, you close the book, and that's the last you ever see of it. The marking felt like saving something, but nothing was saved — there's no index, no search, no way to ask "what was that bit about attention I read two years ago?" The better the book, the more highlights you make, and the more thoroughly they're buried.
Digital reading solved this badly. Kindle gives you a flat, chronological export nobody reads twice. Note-taking apps ask you to retype the passage, and the tax on that is high enough that you stop. The physical book — still the format I actually read in — got nothing at all.
Insight is my attempt at the thing I wanted to exist: capture that costs one gesture, and a library that gets more useful as it grows instead of less.
What it does
Capture. Photograph a page. On-device OCR gives back word-level boxes, and you swipe your finger across the lines you want — contiguous span selection, hit-tested per word, exactly the way selecting text on a screen feels. Tap for a whole line. Hyphenated line-breaks get rejoined. The selection canvas runs in Reanimated worklets on the UI thread, because that gesture is the product and it has to stay at 60fps.
File it. The passage goes to a language model that returns the book title, author, chapter, a short summary and tags as strict JSON. Insight fuzzy-matches that title against books you already have so a second capture from the same book lands in the same place.
Find it again. Every highlight is embedded as a 384-dimension vector. That gives semantic search, a "related" view on any passage, self-organising themes, and an "Ask" mode that retrieves the relevant passages and answers with citations back to your own library.
The architecture, and why it's shaped this way
save(clip) ──► embed(text) ──► clips.embedding (all-MiniLM-L6-v2, int8 ONNX, 384-d, L2-normalised)
└─► FTS5 clips_fts(text, summary, tags, chapter, note) [via triggers]
search(q) ──► qv = embed(q)
──► score = relevance(cosine(qv, emb))
+ keyword hit (FTS5)
+ tag match
+ book title / author match
──► threshold, sort, top 50
ask(q) ──► search(q) top 6 ──► LLM with numbered citations
└─► no LLM or error: return the hits themselves, with a plain reason
related(id) ──► cosine over all clips, above threshold
themes ──► greedy single-pass clustering, named from shared tags
Three decisions carry most of the weight.
Retrieval is hybrid, and the embedding channel is the one that matters. The product thesis is oblique recall — remembering a passage by its shape rather than its words. "That bit about people quietly putting up with a life they never picked" should find "The mass of men lead lives of quiet desperation." Those two strings share no content words at all. I tested that exact query against a real FTS5 index and it returns zero rows, which is the whole argument against the tempting architecture of "keyword search first, rerank with embeddings second" — first-stage recall is a hard ceiling, and you cannot rerank what was never retrieved. Keyword search is a bonus channel here, never the gate.
The embedder earns its place on related and themes, not on search. This is the question I kept interrogating, because an online LLM could plausibly do semantic search on its own. But related and themes are cross-passage questions — which of my three thousand highlights resemble this one, and what clusters do they form — and no single LLM call computes that at any library size without seeing every passage at once. The capture pipeline produces per-passage outputs; the embedder is what makes the library more than the sum of its captures.
Node and the phone run the same 23 MB model file. MiniLM as int8 ONNX, pinned by sha256, with one hand-written pure-TypeScript WordPiece tokenizer and one pooling function shared by both sides. Tokenizer parity is a Jest test against a fixture from HuggingFace's own tokenizer, not a hope. This sounds like fussiness and isn't: an offline harness measuring retrieval quality is only meaningful if its vectors are the vectors the phone will actually produce. Different weights or a different tokenizer, and every number the harness reports is fiction.
That last point is why I'm building an evaluation harness before tuning anything. Insight's four similarity thresholds — the search floor, "related", theme clustering, book-snapping — were originally hand-calibrated to Universal Sentence Encoder's compressed cosine band. Moving to MiniLM invalidated all four at once, silently, because MiniLM's distribution is far wider. A related question/passage pair scores about 0.25 where USE gave 0.80. One model swap quietly changes four features simultaneously, and none of them throws an error — search just gets worse. So the thresholds get derived from a measured distribution over a frozen 150-highlight, 450-question library, scored on recall, MRR, and separation between gold and noise. Not eyeballed.
The constraints I refuse to relax
Private by default. No accounts, no social features, no telemetry, no analytics, no ads. Your library is on your device. The only thing that ever leaves is the single passage being analysed — and the app says so plainly rather than burying it.
The library is local. Search, related, themes and the entire capture flow work with no network. Not "degraded offline" — fully working offline. That's a hard requirement, and it's the reason retrieval is pure TypeScript that runs under Jest in plain Node with no device attached.
The language model is a librarian, not a chatbot. It files things for you. It is never required. Every LLM path falls back to heuristics, and no remote failure or blown quota is allowed to break a user flow.
Those three rules kill a lot of otherwise reasonable features, and that's the point. "Send the whole library to the model" would genuinely produce better answers on a small library — I costed it out at roughly 12k tokens for 100 highlights — but it breaks all three rules at once, and it breaks precisely as the library grows, which is the opposite of when retrieval starts to matter.
Where it is
One React Native and TypeScript codebase targeting iOS and Android, built on Expo with a dev client. The core — OCR mapping, retrieval, vector math, prompts — imports nothing platform-specific, which is what keeps it testable in Node and is the single most important structural rule in the codebase. Native access is quarantined at the edges.
Currently: the capture, selection and storage path works, retrieval evaluation is specified and being built, and cloud LLM support behind an anonymous-auth quota is designed and not yet shipped. It started as a Kotlin and Compose Android app, which now survives only as reference.
What I'm ultimately after is narrower than "a reading app". I want the twenty years of reading I've already done to be queryable — for the book I read in 2019 to surface itself when I'm thinking about something adjacent today, without me having remembered that it was relevant. That's a retrieval problem wearing a reading app's clothes, and getting the retrieval genuinely right is most of the work.