Skip to content

feat: add semantic embedding text prefixes#1044

Open
lastguru-net wants to merge 1 commit into
basicmachines-co:mainfrom
lastguru-net:feat/semantic-embedding-prefixes
Open

feat: add semantic embedding text prefixes#1044
lastguru-net wants to merge 1 commit into
basicmachines-co:mainfrom
lastguru-net:feat/semantic-embedding-prefixes

Conversation

@lastguru-net

Copy link
Copy Markdown

Summary

Adds optional role-specific literal text prefixes for semantic embeddings:

  • semantic_embedding_document_prefix
  • semantic_embedding_query_prefix

These prefixes are prepended to the actual text sent to the embedding provider:

  • document prefix for indexed chunks during sync/reindex
  • query prefix for vector and hybrid search queries

Fixes #1008.

Why

Some embedding models require role-specific text prefixes as part of the input string, not as API parameters.

Examples include models where the recommended input format is something like:

  • documents: title: none | text: ...
  • queries: task: search result | query: ...

Basic Memory already supports LiteLLM input_type settings, but that only helps providers that understand an API-level role parameter. OpenAI-compatible endpoints such as TEI just receive the input strings, so prefix-sensitive models cannot currently be used in their recommended form without a proxy or local patch.
Without this, retrieval quality evaluation is misleading: a model can look worse than it really is because documents and queries are embedded without the format it was trained/evaluated with.

Implementation

  • Adds document/query prefix config fields.
  • Wraps embedding providers with a generic prefixing provider.
  • Works across fastembed, openai, and litellm.
  • Empty or unset prefixes preserve current behavior.
  • Includes prefixes in the provider cache key.
  • Includes prefixes in stored vector identity, so changing either prefix invalidates existing embeddings and requires reindexing.
  • Exposes prefix-enabled status in runtime/project status surfaces.
  • Documents the difference between LiteLLM input_type and literal text prefixes.

Validation

  • 201 passed focused tests
  • Ruff format/check on touched Python files
  • ty check src tests test-int
  • git diff --check

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7f7198d58a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/basic_memory/repository/prefixing_provider.py Outdated
Signed-off-by: Dmitry Golubev <lastguru@gmail.com>
@beru-ant-king beru-ant-king force-pushed the feat/semantic-embedding-prefixes branch from 7f7198d to 0818b8d Compare July 7, 2026 19:12
@lastguru-net

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 0818b8d7da

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add role-specific text prefixes for semantic embedding queries and documents

1 participant