Concepts — associations, contexts, two lanes

Knowledge is an association

The smallest unit is the five-tuple (subject, label, object, weight, source).

A context is a 文脈 — one frame of meaning

Within one context, one spelling means one referent.

Apple the fruit and Apple the company belong to different contexts. This one-spelling-one-referent convention keeps the graph a simple place to put knowledge rather than a disambiguation machine. Choosing which context to ask is language understanding — the LLM's job. For that, GET /contexts returns, on one row per context: the human-written description, machine-computed stats (association count, top concepts, label samples), and usage stats (reads, misses, writes, last access). "A context nobody ever picks" and "a context that gets picked but never answers" can both be spotted there.

Splitting is a precision lever, not only disambiguation. A search or query is scoped to the contexts it names, and a context is the finest unit that scoping can reach. Folding two unrelated frames into one context does not just mix noise into every read — it removes the ability to exclude the irrelevant frame at all, because nothing smaller than the context can be left out from within it. Split frames apart at write time; groups (below) bring them back together at read time only when a cue should legitimately reach across.

Groups bundle contexts — and searches cross them

A group is an organizational label over contexts, not context content.

/groups bundles contexts many-to-many (a context may sit in several groups) and may nest child groups — a shallow DAG, at most 3 groups tall, never cyclic. Membership updates are deltas (add_contexts / remove_contexts, add_groups / remove_groups), referential integrity is strict (members must exist; deleting a context or group drops it from every group), and each group persists as one {name}.group file beside the context files.

Groups are the addressing unit of cross-context search: POST /recall, POST /query, and POST /sources/search take contexts and/or groups lists and run one search across every context reached — nested children included, overlaps deduped, every match tagged with the context it came from. Graph matches merge on |weight| (weights share one scale — evidence mass); passage hits interleave by per-context rank, because passage scores are corpus-local. Deep dives (activate, explore, paths, resolve) stay per-context by design: search across, then pull the thread inside the context that answered.

Worked mappings — a paper split into section contexts and chapter groups, parts as sources, cross-paper search, and the pseudo-hierarchy trap — are in Modeling.

The graph is an index, not an archive

The skeleton goes on the graph; the flesh stays in the original text.

When ingesting a document, the skeleton of its facts is written as associations and, at the same time, the original text itself is registered with the sources API (source id → passage). The server splits the passage into paragraphs at blank-line boundaries, and every association's attribution can point at a paragraph via source and paragraph — which is what lets the citations API pull back the original sentence for verification. Give paragraphs a section (heading) and the resolved section name rides along on the attributions of every graph read. A paragraph can also carry a typed locator (a page/slide/sheet/table position, ADR 0007 §7) — independent of section, and unlike it, naming only that exact paragraph rather than extending to the next one; it resolves onto attributions and citations the same way.

The retrieval loop

The discipline a connected LLM receives from /protocol. Tracing it by hand with curl takes the same shape.

The normalized entrance — a cue is never matched raw

The entrance of resolve is a bigram inverted index built on top of NFKC normalization, case folding, and katakana→hiragana conversion. Full/half-width variation, kana spelling variation, and light typos are absorbed there. On a miss, rephrase, or temporarily lower dice_floor (default 0.3) to widen fuzzy matching.

Candidates carry a kind. exact/alias mean "the cue is a registered spelling itself"; containment/fuzzy mean "the strings merely overlap". A high score of the latter kind can be a look-alike — 京都 overlaps 東京都 at 0.67. Read the gloss that rides along with the top candidates (name + a summary of its heaviest facts) before adopting one. Never adopt on score alone.

The semantic tier backs up a weak entrance

With embeddings configured, tier:"semantic" candidates join when lexical candidates are absent or weak (top score < 0.5). Names are embedded as glosses, so paraphrases (醸造責任者 → 杜氏) and question-shaped cues land too. Scores are never compared across tiers — a lexical Dice coefficient and a cosine similarity are different rulers. The floor (semantic_floor) is a property of the embedding model; switch models and recalibrate.

When the expected name still doesn't appear — ask why, don't bisect

resolve/explain (and resolve_label/explain; MCP: explain_resolve) takes the cue plus the name you expected and returns the first verdict that applies: not in the vocabulary at all (nearest stored spellings attached — register an alias?), eclipsed by an exact match (the cue IS another stored spelling, so nothing else was ever scored), under the dice_floor in effect (with the score a lower floor would have admitted), cut by limit (with a verified limit that reaches it), or a semantic-tier reason — it never joined, or the gloss cosine sat under semantic_floor. One call replaces the re-run-with-varied-floors archaeology.

The text lane — a net for knowledge that won't become triples

Procedural detail, conditions, quantities, discourse. Knowledge that can't be skeletonized is searched as text from the start.

sources/search is paragraph-level search that fuses a lexical BM25 lane with (when TAGURU_EMBED_PASSAGES is on) a semantic lane over paragraph embeddings. Every hit carries lanes, always declaring which lane ranked it where — a vector-only hit means a paraphrase connected; a BM25-only hit means the surface forms matched. Either is evidence, not a verdict. Read the text. The vector lane admits only matches above a cosine floor — the context's semantic_floor or the server default, overridable per call with semantic_floor in the search request; it floors that lane alone, so BM25-only hits still return. And the response itself says what the search chose to do: its plan names every context searched and, per context, whether each lane ran — with the reason when one was skipped and the effective floor when the vector lane swept — so an empty result under a skipped semantic lane never masquerades as "nothing matched".

Every stored source also carries metadata: a server-stamped stored_at (when it was ingested) and, optionally, a document-supplied date and a set of tags. sources/search takes tags (any-of) and a half-open since/until window over each source's date ?? stored_at, and resolves them to an eligibility set before either lane runs — a filter narrows which sources may answer; it never re-weights BM25's corpus-global statistics or the vector lane's scoring, and a source with no matching metadata never matches a filter. The plan's per-context filter block reports how many of the context's sources were eligible, so a narrow filter and "nothing matched" stay distinguishable.

At scale, the vector lane turns approximate: at 10,000 paragraph vectors and up in one context, an IVF index trades the exact sweep for a probe over a handful of clusters, widening until it has gathered a comfortable oversample before ranking. The response shape, the floor, and every explain verdict stay exactly as they are below the threshold — but the probe can trade a little recall for that speed, so the result set itself is not guaranteed bit-for-bit identical to an exact sweep, only close to it in measured practice. The mechanics — cluster count, build timing, the oversample target, and the measured recall — are in Internal architecture.

Search with the shape of the answer, not the question. Most embedding models place a question ("Which plan includes SSO?") farther from its answer than the answering sentence does ("SSO is available on the Enterprise plan."). Guess a plausible declarative sentence and search with that — the guess doesn't have to be right, only the same shape as the sentence you're looking for. The complementary fix is doc2query: attach "questions this paragraph answers" to paragraphs at ingest — their terms join the paragraph's BM25 postings (and its vectors, where passage embedding is on), so question-shaped searches land on the paragraph through both lanes.

When a document you know is stored fails to surface, sources/search/explain (MCP: explain_search) answers why in one call: name the query and the source you expected, and the first applicable verdict comes back — never stored (or retracted), no shared term (both sides' terms shown as strings, which is how 酒蔵-vs-酒造 spelling forks become visible), or ranked past your limit (with the actual rank, the cutoff score, and a limit verified to reach it). Evidence includes each query term's tf/df and BM25 contribution, and the vector lane's cosine — or the reason that lane never ran.

Ingest discipline

Retrieval quality is decided here. The skeleton of the ingest loop in /protocol.

• Check before mint: resolve/resolve_label before creating a new spelling. Reuse what exists. The label vocabulary is listed by GET /contexts/{name}/labels.
• Make implicit membership explicit: "杜氏は高瀬" — whose master brewer? Run unreachable_from from the document's main entity to find islands you forgot to connect.
• Diff-sync when a document is updated: retract the old version's contribution (weights, attributions, passage) with sources/retract, then ingest the new version normally. Concepts and edges remain; only weights drop.
• Audit the vocabulary at milestones: vocabulary/audit lists candidate spelling splits — lexical twins (spelling variation) and semantic twins (a synonym split in two). Candidates, not verdicts. Same referent → pick a canonical and add an alias. A pair that keeps colliding but is genuinely different (東京都/京都, 前株/後株) is written down as a fact: (株式会社青嶺, 別物, 青嶺株式会社, 1.0) — from then on, both glosses warn right at the resolve site.
• Aliases are entry-only: results always return the canonical spelling. An alias cannot connect two existing concepts (that would be a merge — rebuild territory). A mistaken registration is retracted with DELETE on the same path.

Representation patterns

No new machinery — membership edges and negative weights express all of these.

Procedures (ordered knowledge)

[{"subject":"日本酒の醸造","label":"最初の工程","object":"洗米","weight":1.0,"source":"工程書"},
 {"subject":"洗米","label":"次の工程","object":"浸漬","weight":1.0,"source":"工程書"},
 {"subject":"日本酒の醸造","label":"工程","object":"洗米","weight":1.0,"source":"工程書"}]

• Order is a chain of next-step edges (次の工程 above; fix one canonical label — branches are multiple next-step edges, i.e. a DAG). Start is a first-step edge (最初の工程). Membership (工程) connects every step to the hub, which is what makes coverage audits work.
• Replay is "fetch the start → query {label:"次の工程"} for all pairs → line them up". Never use explore hop distance for order — membership edges create shortcuts through the hub that disagree with chain position.
• Quantities, conditions, and tricks of the trade are not forced onto the graph; they stay in the original text and are caught by sources/search.

Causality

[{"subject":"ストレス","label":"引き起こす","object":"不眠","weight":1.0,"source":"論文A"},
 {"subject":"カフェイン","label":"引き起こす","object":"不眠","weight":-0.8,"source":"論文C"},
 {"subject":"運動","label":"防ぐ","object":"不眠","weight":1.0,"source":"論文D"}]

• "Why X?" is query {label:["引き起こす","高める","要因"], object:"X"} — query by the causal labels with X as object. Chains show up in activate paths — the system shows A→B→C but never asserts A→C. Transitivity is the reader's judgment.
• "Does not cause" is a negative weight on the causal label; "prevents" (防ぐ) is an independent positive label. Don't mix them. Correlation stays on its own label (相関する) and is never promoted to causation.
• A disputed cause shows up as one edge with a small net weight and split attributions. Say so in the answer, too.

Source code

• Concepts are identifiers in their exact spelling, as short names: fetch_block, not CacheStore::fetch_block — qualification dilutes the entrance score. Namespacing is an edge (defined_in → src/store.rs). Files and paths are concepts too.
• Keep the label vocabulary small and reuse it: kind, defined_in, calls, field, variant, returns, invariant, purpose cover most facts about code.
• Case twins (type Frame and accessor frame) are separate concepts, and resolve returns both at 1.0 — break the tie with describe. Never merge them.
• The entrance absorbs camelCase/snake_case and typos (fetchBlock and evict_cld both arrive). Natural-language entry points are aliases onto identifiers (退避ループ → evict_cold).
• The highest-value facts are the ones grep cannot answer — invariants, purpose, design constraints, written as edges over the identifiers.