Skip to content

feat: redesign prompt compaction with 4-tier memory management#47

Merged
chinkan merged 17 commits into
mainfrom
feat/redesign-prompt-compaction
Jul 7, 2026
Merged

feat: redesign prompt compaction with 4-tier memory management#47
chinkan merged 17 commits into
mainfrom
feat/redesign-prompt-compaction

Conversation

@chinkan

@chinkan chinkan commented Jul 7, 2026

Copy link
Copy Markdown
Owner

This branch redesigns prompt compaction with a 4-tier approach, RAG-aware context retrieval, and multi-provider support.

Key changes

  • 4-tier compaction: sync (truncation), reconstruct (few-shot struct), auto-compact (summarization), reactive (expensive) — each with model-specific context window awareness
  • RAG-aware compaction: retrieve_context_for_compaction fetches semantically relevant messages before summarization
  • vec0 metadata pre-filtering: is_summarized and role columns on the vec0 table for filtered similarity search with configurable RRF weights
  • Context window cache: ContextWindowCache with backoff/warmup, model-specific window sizing per provider
  • System prompt preservation: compaction-aware system message handling, EoT marker protocol
  • Background cache warmup: async warmup at startup
  • Configurable compaction: rrf_k, rrf_weight_fts, rrf_weight_vec, preserve_message_count, reactive_compact_pct

Files touched

config.example.toml |   1 +
src/agent.rs        | 214 +++-
src/agent_prompt.rs |  62 +-
src/config.rs       |  91 ++
src/main.rs         |  39 +-
src/memory/conversations.rs |  53 +-
src/memory/mod.rs   |  56 +-
src/memory/rag.rs   | 114 ++
src/provider.rs     | 294 +++-
9 files changed, 892 insertions(+), 118 deletions(-)
1 new design doc added

chinkan added 17 commits July 6, 2026 11:50
- Dynamic context window detection from OpenRouter API
- System prompt preservation across Tiers 1-4
- Structured YAML state block in compaction summaries
- RAG-aware Tier 3/4 using existing hybrid search
- sqlite-vec metadata columns + configurable RRF weights
- Clarify startup wiring in main.rs with code example
- Add refresh_context_window_cache method on Agent
- Tool-group-aware bookend truncation (fixes split tool groups)
- Explicit vec0 migration path via PRAGMA table_info
- Search query searches both to_summarize and preserved
- Replace %d with {} in boundary marker format string (Rust format! style)
- Add main.rs and mod.rs to Files Touched table
- Move metadata column description from conversations.rs to mod.rs
@chinkan chinkan self-assigned this Jul 7, 2026
@chinkan chinkan merged commit 6656e06 into main Jul 7, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant