feat(core): add Milvus semantic vector backend

[zc277584121]

Summary

• add an optional Milvus vector storage backend for semantic search
• keep full-text search and vector chunk metadata in SQLite/Postgres while storing embeddings in Milvus
• document Milvus Lite, Milvus server, and Zilliz Cloud configuration

Testing

• env UV_CACHE_DIR=/data2/zhangchen_workspace/auto_integrations/basic-memory/.uv-cache uv run --python 3.12 ruff check src/basic_memory/config.py src/basic_memory/repository/search_repository.py src/basic_memory/repository/search_repository_base.py src/basic_memory/repository/milvus_search_repository.py src/basic_memory/services/search_service.py src/basic_memory/services/project_service.py src/basic_memory/repository/project_repository.py tests/repository/test_milvus_search_repository.py tests/test_config.py tests/services/test_project_service_embedding_status.py
• env UV_CACHE_DIR=/data2/zhangchen_workspace/auto_integrations/basic-memory/.uv-cache uv run --python 3.12 ruff format --check src/basic_memory/config.py src/basic_memory/repository/search_repository.py src/basic_memory/repository/search_repository_base.py src/basic_memory/repository/milvus_search_repository.py src/basic_memory/services/search_service.py src/basic_memory/services/project_service.py src/basic_memory/repository/project_repository.py tests/repository/test_milvus_search_repository.py tests/test_config.py tests/services/test_project_service_embedding_status.py
• env UV_CACHE_DIR=/data2/zhangchen_workspace/auto_integrations/basic-memory/.uv-cache uv run --python 3.12 ty check src tests test-int
• env UV_CACHE_DIR=/data2/zhangchen_workspace/auto_integrations/basic-memory/.uv-cache uv run --python 3.12 pytest tests/repository/test_milvus_search_repository.py -q
• env UV_CACHE_DIR=/data2/zhangchen_workspace/auto_integrations/basic-memory/.uv-cache uv run --python 3.12 pytest tests/test_config.py::TestSemanticSearchConfig tests/services/test_project_service_embedding_status.py tests/services/test_semantic_search.py::test_reindex_all_uses_sqlite_vec_aware_drop tests/services/test_semantic_search.py::test_reindex_vectors_force_full_clears_project_vectors_before_resync tests/services/test_semantic_search.py::test_reindex_vectors_purges_sqlite_vectors_before_sync tests/repository/test_search_repository.py::test_init_search_index -q
• env UV_CACHE_DIR=/data2/zhangchen_workspace/auto_integrations/basic-memory/.uv-cache uv run --python 3.12 pytest tests/repository/test_sqlite_vector_search_repository.py tests/repository/test_postgres_search_repository_unit.py -q
• env UV_CACHE_DIR=/data2/zhangchen_workspace/auto_integrations/basic-memory/.uv-cache uv run --python 3.12 pytest tests/repository/test_postgres_search_repository.py -q (21 skipped)

[CLAassistant]

All committers have signed the CLA.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 389a50f42f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Drop existing pgvector table before Milvus markers

When a Postgres project already used the database vector backend, search_vector_embeddings has chunk_id, project_id, embedding_dims, and updated_at plus a NOT NULL embedding vector(...) column, so this subset check treats the pgvector table as a valid Milvus marker table. After users follow the documented backend switch and run bm reindex --embeddings --full, _replace_marker_rows inserts marker rows without embedding, causing the first Milvus sync batch to fail instead of migrating the derived table.

Useful? React with 👍 / 👎.

[zc277584121]

Hi @phernandez @groksrc @jope-bm, could you take a look when you have a chance?

This PR adds an optional Milvus semantic vector backend while keeping SQL metadata storage in the existing repository flow. CLA and DCO are passing now. The branch also handles existing pgvector-derived tables separately from Milvus marker tables, with focused coverage in the Milvus repository tests.