Skip to content

Memory

Jump to bottom
Daniel Ellison edited this page Jun 11, 2026 · 4 revisions

Kai stores three kinds of information about each operator: rules, facts, and conversation history. Rules and conversation history work the same on every install. Facts are where the design has the most moving parts: a local vector store (semantic memory mode), an operator-curated MEMORY.md file (legacy mode), a toggle between the two, and a one-shot migration path from one to the other.

Mental model

Kai has three context layers per turn:

  • Rules. Always-on directives that govern Kai's behavior. Universal rules ship in CLAUDE.md (loaded as the inner Claude's system prompt at session start). Per-operator rules live in PREFERENCES.md (injected with every user message).
  • Facts. What Kai knows about you, the project, decisions, episodes. Either retrieved on similarity from a local vector store or read verbatim from a per-operator markdown file. The choice is operator-controlled via MEMORY_ENABLED.
  • Conversation history. Recent messages from the current chat, injected at session start. Stored as JSONL under DATA_DIR/history/<chat_id>/. Always on.

Kai's inner Claude never auto-edits any of these. It runs in stream-json mode, which bypasses Claude Code's interactive auto-memory. Every layer above is injected explicitly by the bot process.

The toggle: MEMORY_ENABLED

MEMORY_ENABLED=true (set in /etc/kai/env or via make config) routes facts through the semantic memory subsystem: MEMORY.md is not injected, the vector store is the only fact surface, and Kai's writes go through the /api/memory/add endpoint.

MEMORY_ENABLED=false runs the original design: MEMORY.md is read at session start and injected on every turn; the assistant edits it via Edit and Write tools when it has new information to save; the vector store is dormant.

The toggle is a full switch with no halfway states. On any given install, exactly one fact surface is active.

When toggling between modes, run the migration script first so the vector store contains the operator's existing curated knowledge:

python scripts/migrate-memory-md-to-qdrant.py --user-id <chat_id>

This walks the operator's MEMORY.md, splits it into chunks, and writes each chunk to the vector store with source="migration" metadata. Imported content remains identifiable so it can be rolled back as a unit:

python scripts/migrate-memory-md-to-qdrant.py --user-id <chat_id> --rollback

Semantic memory

When enabled, Kai writes structured facts to a local Qdrant vector store via Mem0. Storage is fully on-host: no external services, no Docker, no open ports.

Storage layout

  • Vector store. DATA_DIR/memory/qdrant/ (Qdrant embedded, single collection kai_memory).
  • Mem0 history database. DATA_DIR/memory/mem0_history.db (SQLite, used for dedup and history tracking).
  • Embedding model. all-MiniLM-L6-v2 from sentence-transformers, 384 dimensions. Cached under ~/.cache/huggingface/ after first run.

Every row carries user_id = str(chat_id). Cross-user retrieval is impossible: every read and write filters by user.

What gets stored

Source What it is How it gets there
extracted Structured fact extracted from a recent conversation A one-shot extractor runs after each exchange through the user's configured backend (claude, codex, opencode, or goose) and writes facts (with confidence, tags, prompt version, optional confirmation quote)
episode Per-conversation episode summary A two-stage extractor classifies whether the conversation was episode-worthy, then writes a Sophia-style record (goal, context, approach, outcome, outcome quality, lessons)
migration Operator-curated content imported from MEMORY.md One-shot via the migration script
"" (legacy) Rows from earlier code paths that predate the source-tagging system Hidden from the /memory UI; left in place rather than purged

The extracted and episode sources accumulate continuously. migration is a one-shot import. Legacy rows are filtered out of every user-facing surface.

Retrieval

Every user message triggers a semantic search. The pipeline:

  1. Embed the user's message with the local embedding model.
  2. Search Qdrant for the user's slice; fetch up to MEMORY_SEARCH_LIMIT results.
  3. Filter by relevance threshold (MEMORY_SEARCH_FLOOR). Loosely-relevant noise drops out.
  4. Re-rank with source weights. Extracted and episode rows weighted highest, migration slightly lower, legacy or unknown sources weighted down. Weighting affects ranking only; sub-threshold rows are never rescued.

Top hits get formatted with a short provenance tag (fact, episode, legacy) and date, then injected as a labeled context block at the top of the user's message. The block is explicitly marked as context, not instructions.

Tuning knobs (set in /etc/kai/env via make config):

Variable Default Meaning
MEMORY_ENABLED false Master toggle
MEMORY_SEARCH_LIMIT 10 Top-K returned to the prompt
MEMORY_SEARCH_FLOOR 0.3 Relevance threshold; clearly relevant scores 0.7+, noise below 0.3
MEMORY_TOKEN_BUDGET 2000 Maximum tokens of memory context per message
MEMORY_EXTRACTION_TIMEOUT_S 10 Per-turn cap on the extraction call
MEMORY_EMBEDDING_MODEL all-MiniLM-L6-v2 Override only if you also rebuild the Qdrant collection at the new dimension

Every retrieval emits a memory.recall log line with timing, hit counts, query and per-hit snippets, and the rejection reason for short-circuit returns.

Extraction

After each exchange, an asynchronous extraction pipeline runs. It makes a one-shot call through the user's configured backend (all four backends - claude, codex, opencode, and goose - are extraction-eligible; the model comes from the per-role registry) over the user message plus a capped prefix of Kai's response, and emits zero or more structured facts. Each fact carries:

  • text -- the fact in third-person past tense
  • confidence -- the extractor's self-reported confidence in [0.5, 1.0]
  • tags -- one to four entries from the closed enum: preference, decision, fact, constraint, confirmed_action, project, location, schedule, relationship
  • confirmation_quote -- verbatim text from the conversation when the fact records a confirmed action
  • prompt_version -- bumped whenever the extractor prompt changes; visible in /memory stats so prompt-version drift is observable

Extraction is time-capped and runs in the background. Failures don't surface to the user; the conversation continues.

Episodes

A second-stage classifier runs over the same exchange to decide whether the interaction was episode-worthy: clear goal, attempted approach, observable outcome. When it fires, a separate one-shot call produces a Sophia-style record (goal, context, approach, outcome, outcome quality, lessons, actors, tags). Episodes are written with source="episode" and surface in retrieval alongside extracted facts.

In production, episode rate runs around 0.4 per turn, the rate the design targeted: high enough to capture meaningful patterns, low enough not to flood retrieval.

MEMORY.md mode

When MEMORY_ENABLED=false, Kai uses the original design:

  • Storage. DATA_DIR/memory/<chat_id>/MEMORY.md per operator. Per-user isolation is enforced at the directory level.
  • Read path. Injected verbatim at session start.
  • Write path. The assistant uses Edit and Write tools when it has new information to save. There is no auto-edit.
  • Bootstrap. A fresh chat ID with no MEMORY.md yet seeds from MEMORY.md.example on first message.

In this mode, semantic memory is dormant. The Qdrant directory may exist on disk from a prior run or migration but isn't consulted on any read or write path.

/memory command

The /memory Telegram command surfaces the semantic store directly to operators. Disabled-mode installs return "Memory is not enabled in this install."

Subcommand Purpose
/memory Dashboard: one-line count summary plus a utility row of inline buttons (Facts (N), Episodes (N), Stats)
/memory stats Detailed view: per-source totals, tag distribution, confidence histogram, prompt-version breakdown
/memory search <query> Semantic search across the operator's slice, with relevance scores
/memory help Subcommand reference

Tapping Facts opens a paginated list of extracted and migration rows; tapping Episodes opens the episode list. Each row drills into a detail view where a per-row Forget button deletes a single record after a confirmation prompt. There is no bulk-by-tag forget; deletions are always per-row.

For full details and examples, see Slash Commands.

Privacy and isolation

  • Per-user. Every read and write filters on user_id = str(chat_id). Operators cannot see each other's memories.
  • Local. Qdrant runs embedded in the bot process. The only network calls in the memory path are the extractor's LLM call (to whichever provider the user's backend is configured for) and the HuggingFace embedding model download (one-shot on first install).
  • Operator-controlled. Each operator browses, searches, and deletes their own facts via /memory. The administrative python -m kai memory purge command exists for source-scoped cleanup but requires explicit --yes.

What hasn't shipped

  • Self-assessment. A calibrated capability model that aggregates task outcomes over time. Episodes capture per-interaction outcome quality, which is most of the raw signal; the longitudinal aggregation layer is on the roadmap, not built.

See also

Clone this wiki locally