Local RAG over PDFs — fully local with the Python SDK, LangChain, and Ollama

Four responsibilities, four places they run

The offline half builds the corpus; the online half only ever reads it. Nothing here calls out past localhost.

This is the SDK-side counterpart to the MCP walkthrough: there, an LLM agent calls Taguru MCP tools directly and does its own decomposition and composition turn by turn. Here, your code owns both ends — TaguruIngester for the write path, TaguruRetriever and an LCEL chain for the read path — and the corpus underneath is modeled the same way either time.

Prerequisites: check, don't pull

Docker, Python 3.10+, and Ollama with the models this walkthrough names already present.

# the extract/answer model and the embedding model — both must already be pulled
ollama list | grep -E 'qwen2.5:7b-instruct|nomic-embed-text'

Nothing on this page pulls a model for you. Neither the shell snippets here nor the reference example runs ollama pull. A multi-gigabyte download happening as a side effect of running a walkthrough is a surprise nobody asked for — if ollama list comes back empty, pull the models yourself first: ollama pull qwen2.5:7b-instruct && ollama pull nomic-embed-text.

Start the server, wire it to Ollama's embeddings

deploy/docker-compose.yml as-is, plus the semantic-lane environment pointed at Ollama's OpenAI-compatible endpoint.

# docker-compose.override.yml — merges over deploy/docker-compose.yml
services:
  taguru:
    environment:
      # the container reaches the host's Ollama through this DNS name
      TAGURU_EMBED_URL: http://host.docker.internal:11434/v1/embeddings
      TAGURU_EMBED_MODEL: nomic-embed-text
      TAGURU_EMBED_PASSAGES: "1"   # opt in to embedding stored paragraphs, not just glosses

TAGURU_API_TOKENS='ops:CHANGE-ME' docker compose \
  -f deploy/docker-compose.yml -f docker-compose.override.yml up -d

Running the server binary directly instead of the container? Then Ollama is not needed for this step at all: a default build carries an in-process provider — TAGURU_EMBED_URL=local plus TAGURU_EMBED_MODEL naming one of taguru-code models (e.g. paraphrase-multilingual-minilm-l12-v2-q, or multilingual-e5-small for stronger recall) downloads the ONNX model once into HF_HOME when set, or ~/.taguru/models otherwise, and embeds on the CPU, offline afterwards. The Docker image ships without it (ONNX Runtime publishes no musl binaries), which is why the container path above points at Ollama instead.

Skipping this section is fine — search still works on BM25 and the graph lane alone, TAGURU_EMBED_PASSAGES is opt-in for a reason. What changes is paraphrase recall on the text lane, and TAGURU_SEMANTIC_FLOOR's default (0.35) is calibrated for text-embedding-3-large, not any particular local model — measure the right value for yours with taguru calibrate before trusting the floor, and see Troubleshooting — the semantic lane if a search response's plan says the vector lane didn't run.

nomic-embed-text is a poor pick for a Japanese corpus. Measured on a Japanese sake-brewery corpus: it ranked a manufacturing-process term above the brand name a cue actually named — an order inversion, not a marginal score. Prefer a model built for multilingual retrieval instead — embeddinggemma through the same Ollama endpoint, or the in-process local provider's multilingual-e5-small shown above — and see Troubleshooting — the semantic lane for the measured numbers and why a healthy multilingual model can still report calibrate OVERLAP on short concept names.

Pin every layer to the same minor version

Server, SDK, and langchain-taguru ship from one repository in lockstep — read all three before trusting a shape.

# the running server answers its own pin
curl -s localhost:8248/version   # {"server":"0.9.0","http_contract":{"current":1,"supported":[1]},"mcp_contract":{"current":1,"supported":[1]},...}
grep 'image:' deploy/docker-compose.yml   # ghcr.io/t0k0sh1/taguru:0.9.0 — the deployed pin

# pin the Python side to the matching minor
pip install 'taguru==0.9.*' 'langchain-taguru==0.9.*'

# fail loudly at startup rather than silently on the first odd response shape
python -c "import taguru, taguru_langchain as tl; \
assert taguru.__version__.startswith('0.9.') and tl.__version__.startswith('0.9.'); \
print(taguru.__version__, tl.__version__)"

GET /version also names the wire-contract versions (http_contract/mcp_contract) and the accepted batch/image formats, and it answers without auth — the first probe when versions are in doubt. The same build identity rides /metrics too: curl -s localhost:8248/metrics | grep taguru_build_info. Full triage for a mismatch (a 0.8.x client against a 0.9.x server, or the reverse) is Troubleshooting — version skew.

PDF into numbered sections

Extraction happens through the SDK here, not the CLI — taguru extract reads .md/.txt only.

taguru extract does not read PDFs. The CLI producer is documented against .md/.txt (the CLI shape); pointing it at a PDF directory extracts nothing. The standard PdfConnector is the shortest path today when one document per PDF is the granularity you want. This walkthrough instead splits each PDF into numbered sections by hand — plain pypdf, no LangChain document loader needed for this much — because it wants one source id per section (paper.pdf#3-shaped), finer than a connector's one-document-per-file default; see work units for when that finer split earns its keep. If you would rather stay on the CLI path after this step, write each section out as its own .md file and hand the directory to taguru extract + taguru import instead of the SDK path below — everything from context and group modeling onward applies either way.

from pypdf import PdfReader
import re

SECTION_RE = re.compile(r"^(\d+)\.\s+(.+)$", re.MULTILINE)

def pdf_to_sections(pdf_path, paper_id):
    pages = PdfReader(pdf_path).pages
    text = "\n".join(p.extract_text() for p in pages)
    marks = list(SECTION_RE.finditer(text))
    for i, m in enumerate(marks):
        start = m.end()
        end = marks[i + 1].start() if i + 1 < len(marks) else len(text)
        yield {
            "paper": paper_id,
            "n": int(m.group(1)),
            "title": m.group(2).strip(),
            "text": text[start:end].strip(),
        }

A numbered-heading regex is enough for a fixture; a real corpus's headings vary more and may need a smarter splitter. What matters for everything downstream is only the shape of the result: one dict per section, carrying which paper it belongs to, its section number, and its text.

One section, one context; one paper, one group

The same mapping the modeling guide's worked example uses for BERT, applied to these two papers.

group: paper/tanaka2024
├── context: section/tanaka2024/1
├── context: section/tanaka2024/2
└── context: section/tanaka2024/3

group: paper/sato2023
├── context: section/sato2023/1
└── context: section/sato2023/2

A section's vocabulary stays stable within it — exactly the one-spelling-one-referent contract a context exists to hold — so the section is the context, and the paper is the group that bundles its sections back together for cross-paper search (why a group, never a super-context). Group creation has to be idempotent by hand — groups.create 409s if the group already exists, so a rerun catches that and switches to a delta update instead of failing:

from taguru import Taguru, ConflictError

client = Taguru()

def ensure_group(name, description, contexts):
    try:
        client.groups.create(name, description=description, contexts=contexts)
    except ConflictError:
        client.groups.update(name, add_contexts=contexts)

Context creation gets the same idempotence for free from TaguruIngester itself — create_context=True stamps a create block that only takes effect when the context is still absent, so re-running the pipeline over an unchanged PDF neither fails nor duplicates anything. That call shows up next, where the ingester for each section actually runs.

One TaguruIngester per section, one local model doing the extracting

The extract responsibility from the table above, in code: a fresh context= for every section, the same llm= throughout.

from langchain_core.documents import Document
from langchain_ollama import ChatOllama
from taguru_langchain import TaguruIngester

extract_llm = ChatOllama(model="qwen2.5:7b-instruct", temperature=0)

section_contexts = []
for section in pdf_to_sections("papers/tanaka2024.pdf", "tanaka2024"):
    context = f"section/{section['paper']}/{section['n']}"
    ingester = TaguruIngester(
        context=context,                       # switches every iteration
        llm=extract_llm,
        client=client,
        create_context=True,
        context_description=f"{section['paper']} §{section['n']} — {section['title']}",
    )
    doc = Document(
        page_content=section["text"],
        metadata={"source": f"{section['paper']}/{section['n']}"},
    )
    outcome = ingester.ingest_documents([doc])[0]
    print(f"{context}: {outcome.associations} facts, {outcome.aliases} aliases")
    section_contexts.append(context)

ensure_group("paper/tanaka2024", "Tanaka et al. 2024, full paper", section_contexts)

metadata["source"] is the retract-then-apply idempotency unit — tanaka2024/3 here — chosen by this pipeline, not handed out by the server. There is no separate "internal source ID" API to look up: whatever string goes in at ingest is the same string every later read (search hits, cite_passage) comes back with. refresh_embeddings defaults on, so if the semantic lane is configured (above) each section's paragraphs are embedded right after its facts land — no separate step to remember.

One retriever, both papers, named by group

No super-context was ever created — the group names in groups= are the entire cross-paper search target.

from taguru_langchain import TaguruRetriever

retriever = TaguruRetriever(
    client=client,
    groups=["paper/tanaka2024", "paper/sato2023"],
    k=8,
)

for doc in retriever.invoke("How does low-temperature fermentation affect ginjo aroma yield?"):
    meta = doc.metadata
    print(f"  [{meta['lane']:>9}] {meta['context']} — {meta['source']} ¶{meta['paragraph']}")

Each returned Document's metadata carries context, source, paragraph, section, and lane — which paper section answered is never a guess, whether the hit came from the graph, the text lane, or both. Reaching for TaguruRetriever(context=...) per paper in a loop instead would undo the split this pipeline just built; naming both groups in one retriever is the point.

Search, then answer — as two separate, separately observed phases

The retrieval step and the generation step are shown here as distinct calls with distinct output, on purpose, and a different local model backs each.

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough

answer_llm = ChatOllama(model="qwen2.5:7b-instruct", temperature=0)

# the API has no notion of a "citation label" — this mapping is entirely this
# pipeline's own, built once from whatever is human-readable for your corpus
CITATION_LABELS = {
    "tanaka2024/3": "Tanaka et al. 2024, §3",
    "sato2023/2": "Sato et al. 2023, §2",
}

def format_docs(docs):
    return "\n".join(f"[{d.metadata['source']}] {d.page_content}" for d in docs)

PROMPT = ChatPromptTemplate.from_messages([
    ("system", "Answer using only the context below. Cite every claim "
     "with its bracketed source id, e.g. [tanaka2024/3].\n\n{context}"),
    ("human", "{question}"),
])

chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | PROMPT | answer_llm | StrOutputParser()
)

question = "How does low-temperature fermentation affect ginjo aroma yield?"

# phase 1 — observe what retrieval brought back, before any generation runs
for doc in retriever.invoke(question):
    label = CITATION_LABELS.get(doc.metadata["source"], doc.metadata["source"])
    print(f"  [{doc.metadata['lane']:>9}] {label}: {doc.page_content[:60]}…")

# phase 2 — observe the answer, generated from exactly those documents
print(chain.invoke(question))

A source id like tanaka2024/3 is a machine key chosen at ingest time; a citation label like "Tanaka et al. 2024, §3" is what a person reading the answer wants to see. Taguru's API only ever deals in the former — the mapping between the two is this pipeline's own CITATION_LABELS dict, built once from whatever front matter the PDFs carry.

The trace back to the original PDF is one call away, and it reaches the paragraph, not just the section:

citation = client.context("section/tanaka2024/3").cite_passage("tanaka2024/3", 0)
print(citation.text)   # the verbatim paragraph the answer's claim came from

Fixed means the pipeline is offline and the app never rebuilds it

Everything above this line runs once, out of band; everything below retrieve is all a deployed app ever needs.

Building the corpus is the job of the pipeline in PDF → sections through ingest — run it, walk away, and the serving code in retrieve and answer never touches TaguruIngester at all. There is deliberately no "re-index this corpus" button wired into an app UI: adding a paper is re-running the offline pipeline over one more PDF, not a feature the serving surface exposes. Re-running the pipeline over an already-ingested PDF is safe by construction — each section's source id is the same string every time, so TaguruIngester's retract-then-apply replaces that section's facts in place instead of accumulating duplicates, and ensure_group above is a no-op the second time a paper's sections are already members.

For a corpus too large for one sitting — dozens of papers instead of two — the same pipeline benefits from checkpointing so an interruption doesn't restart from PDF one; see Long-running ingestion for the shape (both the CLI and the SDK's own checkpoint_store) that this walkthrough's two-paper fixture didn't need.