Skip to content

fix(rag): accept non-uuid string doc_id in the qdrant client - #9597

Open
anilkr09 wants to merge 1 commit into
crewAIInc:mainfrom
anilkr09:fix/qdrant-string-doc-id
Open

anilkr09 wants to merge 1 commit into
crewAIInc:mainfrom
anilkr09:fix/qdrant-string-doc-id

Conversation

@anilkr09

Copy link
Copy Markdown
Contributor

Related issue

Fixes #9596

Summary

With qdrant-client's in-memory client, QdrantClient.add_documents and aadd_documents raise ValueError: Point id doc-1 is not a valid UUID for a doc_id such as "doc-1", though BaseRecord.doc_id is a str. _create_point_from_document passes it unchanged as the point id (lib/crewai/src/crewai/rag/qdrant/utils.py:275-278 on main at 374c053920), and qdrant-client 1.14.3's in-memory client requires a string point id to parse as a UUID.

  • A string that is not a UUID is mapped to a uuid5 point id in its own namespace; the original is kept in the payload under crewai_doc_id.
  • _process_search_results returns the original as id and keeps the key out of metadata. The key is read only when it maps back to the point's id.
  • Integer ids, UUID strings and documents without a doc_id are stored as before.
Case main 374c053920 this PR
add_documents, "doc-1" (A) ValueError added; search returns 'doc-1'
aadd_documents, "doc-1" (D) ValueError added; asearch returns 'doc-1'
batch "7", "doc-2" (E) ValueError, 0 stored 2 stored
"doc-1" twice (K) ValueError 1 point, content new
UUID, no doc_id, int (F, G, H) added unchanged

Limit (M): an int 7 and a string "7" added in one call are two points, and both search back as '7'. Metadata that already holds crewai_doc_id together with a non-UUID doc_id raises a ValueError naming the key (T), as main already raised for that doc_id.

Verification

  • Tests added or updated for the changed behavior

  • Relevant tests and quality checks pass locally

  • New tests TestStringDocIds in lib/crewai/tests/rag/qdrant/test_client.py: 6 of 8 fail on main's source (the ValueError from the add), all 8 pass here, also in random order with xdist and on Python 3.10. In-memory Qdrant, no network. macOS.

  • pytest lib/crewai/tests/rag/qdrant lib/crewai/tests/knowledge lib/crewai/tests/rag/test_error_handling.py: 177 passed.

  • ruff check and ruff format --check on the three changed files, and mypy on lib/crewai/src/crewai/rag/qdrant/utils.py: clean.

  • Not run: the full suite, a Qdrant server, Linux and Windows.

Additional context

Three existing tests asserted the raw id with a non-UUID string. They now use a UUID string, which still stores as the point id, and TestStringDocIds covers the other strings.

Written with an AI coding agent (Claude Code). CONTRIBUTING asks for the llm-generated label on AI-generated PRs; I can't add labels, so could a maintainer add it?

@anilkr09
anilkr09 marked this pull request as ready for review November 20, 2026 07:32
@strix-security

strix-security Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Strix Security Review

✅ No open security findings.

PR summary: Fixes Qdrant document ingestion failing on non-UUID string `doc_id`s by mapping them to derived `uuid5` point ids, stashing the original id in a reserved payload key, and restoring it when search results are built.

Review summary

This PR makes the Qdrant RAG client accept a non-UUID string doc_id: it is stored at a derived uuid5 point id with the original kept in a reserved payload key, and restored as the result id on search. I traced the mapping from every place a Qdrant point id is built or read (the add/upsert helpers in lib/crewai/src/crewai/rag/qdrant/utils.py, the client's collection-level delete/reset, the knowledge storage and tools adapter ingestion paths), checked who can actually supply a doc_id or metadata, and checked every consumer of a search result's id and metadata. The remaining observations are id-hygiene behaviour in the mapping rather than security defects: the fallback mapping is not injective (a non-UUID doc_id and the UUID equal to its hash reach the same point), an integer 7 and a string "7" are stored as two points that both search back as "7", and a document with a UUID-shaped doc_id can carry a caller-supplied crewai_doc_id that is returned as its id. Each requires the party supplying documents to a collection it already has write access to, and neither crosses a privilege boundary nor reaches a consumer that acts on the identifier, so no vulnerability is reported.

Ruled out (3)
  • Can a document overwrite another document's Qdrant point through the new doc_id mapping, since `_point_id("doc-1")` equals a UUID another caller can pass verba…
    The only trigger is the party that supplies the record to `add_documents`, which can already name the victim's point id directly (the id was used verbatim before this change) and can call `delete_collection`/`reset`, so no access is gained that does not already permit the same harm.
  • Can a caller-supplied `crewai_doc_id` metadata value, or a UUID-shaped doc_id paired with it, make a search result report another document's id?
    The restore path only trusts the payload value when `_point_id(stored)` hashes back to the point's own id, and no code in the repository reads `SearchResult["id"]` or `metadata["crewai_doc_id"]`, so a relabelled id reaches no decision.
  • Does an untrusted end user reach the new mapping, i.e. can a user or tenant supply `doc_id`/`metadata` on any in-repo ingestion path?
    Both write paths generate their own ids - `knowledge_storage.py` passes `{"content": doc}` records with no `doc_id`, and the tools adapter uses `str(uuid.UUID(sha256(...)[:32]))` - and the client exposes no per-id retrieve, update or delete that a spoofed id could be aimed at.

Review 1 on this PR · d898f32 · requested Nov 20 07:32 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings · Try a deeper review with @strix-security /deep

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Qdrant in-memory RAG client add_documents raises "Point id doc-1 is not a valid UUID"

1 participant