Repository navigation
feat(embeddings): add DeepInfra provider with an opt-in catalog model - #7993
Georgi Atsev (ats3v) wants to merge 7 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
7dc1fcc to
066dbb7
Compare
066dbb7 to
b3260b8
Compare
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
Strix Security ReviewWarning This pull request has 79 commits after the last Strix review ( ✅ No open security findings. PR summary: Adds a `deepinfra` embedding provider for Crews, Agents, Memory and Knowledge, built on chromadb's OpenAI-compatible embedding function, and picks its default embedding model at runtime from DeepInfra's public model catalog when no model is configured, falling back to a static default. Also registers the provider in the embedding factory and in crewai-tools' `EmbeddingService`, with docs and tests. Review summaryNo security issues found in this PR. The new Ruled out (6)
Review 1 on this PR · Reviewed by Strix |
iris-clawd
left a comment
There was a problem hiding this comment.
I couldn’t independently verify DeepInfra’s catalog-ordering guarantee.
The notes inline on deepinfra_provider.py are what I'd want addressed before this merges. The other inline notes are small.
Someone else should sign off on this too: the change introduces public provider/configuration keys and changes the meaning of an omitted model from a fixed package value to a vendor-controlled runtime selection.
592bc1f to
af84028
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit af84028. Configure here.
89f1183 to
60122fb
Compare
Add a DeepInfraProvider built on the OpenAI embedding function with the DeepInfra base URL, register it in the factory, the provider spec union and the crewai-tools EmbeddingService, and document it next to the OpenRouter example in en, ar, ko and pt-BR. Defaults to BAAI/bge-m3 and reads DEEPINFRA_API_KEY. The `model` alias is popped before validation instead of copied, because the provider forwards its extra fields to the embedding function, which has no `model` parameter.
Use the more popular Qwen/Qwen3-Embedding-8B as the default embedding model in the provider, the crewai-tools EmbeddingService and the docs example in en, ar, ko and pt-BR.
chromadb's OpenAIEmbeddingFunction only forwards `dimensions` for OpenAI text-embedding-3 models, so a DeepInfra provider that accepted the field returned full-size vectors into a store sized for fewer. Refuse the key with a ValidationError that says why, and drop the field and its DEEPINFRA_DIMENSIONS alias from the provider and the config TypedDict.
When no model is configured, read DeepInfra's OpenAI-compatible model
list sorted for crewAI (GET {api_base}/models?filter=with_meta&sort_by=crewai)
and use the first entry tagged "embed", so the default follows what
DeepInfra recommends without a crewAI release. The lookup is public, has a
3 s timeout and falls back to Qwen/Qwen3-Embedding-8B with a warning when
the catalog cannot be read or lists no embedding model. Failures are not
cached; a successful pick is kept for the life of the process per base
URL, so every store a process opens shares one embedding space. A
configured model_name skips the lookup.
The crewai-tools EmbeddingService uses the same resolver with the
provider's base URL precedence (extra_config api_base, then
DEEPINFRA_API_BASE, then the default), so the model is chosen from the
host the embeddings go to.
The settings base matches every field name and alias against the environment, ignoring case. MODEL_NAME, which crewAI reads for the chat model, became the embedding model and skipped the catalog lookup, and API_BASE and API_KEY, which belong to other tools, redirected the requests and supplied the key sent to DeepInfra. The provider now takes its config, then DEEPINFRA_API_KEY and DEEPINFRA_API_BASE only.
A default read from DeepInfra's catalog can change between restarts, and a persisted store then queried with a different model of the same dimension loses retrieval quality without any error. Keep the default static and read the catalog only when asked for. - Without a model the provider uses Qwen/Qwen3-Embedding-8B again, with no network request. model_name="auto" (or model="auto") reads the catalog; crewai-tools' EmbeddingService passes "auto" through to the provider. - A failed lookup raises a ValidationError that names the catalog URL and says to set model_name, instead of substituting another model. - Picks are cached per base URL with any trailing slash stripped. Failures are not cached, so the next provider can succeed once the catalog is back. - effective_api_base is gone: the provider resolves its own base URL.
…ction Since crewAIInc#8934 the Azure and OpenRouter providers use crewAI's own OpenAIEmbeddingFunction, and the factory's DeepInfra overload already promised that class. The provider still built chromadb's function, so it returned a different type than declared and emitted chromadb's DeprecationWarning about the api_key not being persisted. The dimensions check stays: the native function also sends dimensions only for OpenAI text-embedding-3 models.
60122fb to
b1f5b3f
Compare
|
João Moura (@joaomdmoura) could you re-run the review gate here? Its notes are addressed: the default model is static again, and the catalog lookup is opt-in via |

Related issue
Relates to #7455 (closed by #7459, which listed the embeddings provider as a follow-up). Supersedes #7978.
Summary
Add
deepinfraas an embedding provider for Crews, Agents, Memory and Knowledge. Without a configured model it embeds withQwen/Qwen3-Embedding-8B;model_name="auto"opts into the model DeepInfra currently recommends.Provider
Same shape as the OpenRouter provider from #7127: a
DeepInfraProvideron crewAI's nativeOpenAIEmbeddingFunction(the one Azure and OpenRouter use since #8934) with the DeepInfra base URL, registered inPROVIDER_PATHS,ProviderSpec,AllowedEmbeddingProvidersand the crewai-toolsEmbeddingService(pluscreate_deepinfra_service()). No new module dependency.https://api.deepinfra.com/v1/openai,DEEPINFRA_API_BASEorapi_baseoverrides itmodel_namedefaults toQwen/Qwen3-Embedding-8B;modelis accepted as an alias;default_headerspasses throughdimensionsis rejected with aValidationErrorthat says why: the OpenAI embedding function only forwards it for OpenAItext-embedding-3models, so for a DeepInfra model the vectors would silently keep the model's full size (theopenaiandopenrouterproviders accept and drop it today)api_keyfrom config orDEEPINFRA_API_KEY; a missing key is aValidationErrorDEEPINFRA_API_KEYandDEEPINFRA_API_BASEare read from the environment. The settings base would otherwise match every field name and alias against it, ignoring case, soMODEL_NAME(crewAI's chat model),API_BASEor a genericAPI_KEYwould configure the embedder. Config values still win over the environment.One deliberate difference from the OpenAI/OpenRouter providers: the
modelalias is popped before validation instead of copied. The provider allows extra fields and forwards all of them to the embedding function, which has nomodelparameter, so copying leavesbuild_embedder({"provider": ..., "config": {"model": ...}})failing withTypeError: OpenAIEmbeddingFunction.__init__() got an unexpected keyword argument 'model'. Theopenaiandopenrouterproviders behave that way onmaintoday; not touched here, happy to send a separate fix.Opt-in:
model_name="auto"reads DeepInfra's catalog"auto"(asmodel_nameormodel) triggers the lookup. An omitted model is the static default and sends no request, so what an omitted model means never depends on the vendor.GET {api_base}/models?filter=with_meta&sort_by=crewaiis public (no key sent), 3 s timeout; the first entry whosemetadata.tagscontainsembedwins. The ordering is maintained server-side by DeepInfra for agent frameworks.embedentry) construction fails with aValidationErrorthat names the catalog URL and says to setmodel_nameto a fixed id. No other model is substituted, so an outage cannot switch a store's embedding space.api_base(config, thenDEEPINFRA_API_BASE, then the default), the same host the embeddings go to.EmbeddingService.create_deepinfra_service(model="auto")passes"auto"through to the provider."auto"uses the model DeepInfra currently recommends, which can change between runs, in en, ar, ko and pt-BR.The
"auto"pick can change between runs by design, so persisted Knowledge or Memory stores should keep a fixed id, as the default does.Verification
Tests added or updated for the changed behavior
Relevant tests and quality checks pass locally
uv run pytest lib/crewai/tests/rag/embeddings lib/crewai-tools/tests/rag -q- 538 passed, 2 skipped, afteruv sync --all-groups --all-extras. New intest_factory_deepinfra.py: for the provider, factory wiring, the native function's client pointed at DeepInfra's base URL,modelalias through a realOpenAIEmbeddingFunction,dimensionsrejected directly and through the factory, defaults, overrides, env vars and a missing key, and generic variables (MODEL_NAME,API_BASE,API_KEY,DEFAULT_HEADERS) leave the provider alone; for"auto", the default never reads the catalog, the firstembedentry wins in catalog order (viamodel_nameandmodel), a customapi_baseandDEEPINFRA_API_BASEare queried, an explicit model skips the lookup, aValidationErroron connection error / HTTP error / malformed payload / invalid JSON / no embedding model, successful picks cached per normalized base URL, failed lookups not cached, and an embedder built with"auto"through a realOpenAIEmbeddingFunction. Plus cases intest_embedding_factory.py,test_backward_compatibility.pyand the crewai-toolstest_embedding_service.py(static default,"auto"passed through to the provider). An autouse fixture keeps every lookup off the network.uv run ruff checkanduv run ruff format --checkon the changed files - cleanuv run mypyon the provider package,factory.py,types.pyand the crewai-toolsembedding_service.py- no issueslive against api.deepinfra.com: every embedder below is crewAI's native function and emits no
DeprecationWarning;build_embedder({"provider": "deepinfra", "config": {}})embeds withQwen/Qwen3-Embedding-8B(4096-dim vectors) without a catalog request;model_name="auto"resolves toQwen/Qwen3-Embedding-8Bfrom the catalog and embeds;model="auto"against an unreachable base fails with theValidationErrornaming the URL. Earlier on this branch:model="Qwen/Qwen3-Embedding-4B"through the alias embeds (2560-dim);EmbeddingService.create_deepinfra_service()withembed_text,embed_batch,get_embedding_dimension()= 4096 andvalidate_connection()= True; withMODEL_NAMEandAPI_BASEset for other tools, the embedder keeps its own settingsbranch rebased on
mainat d7dfa0eAdditional context
No behaviour change for existing providers.
dimensionswould be accepted by the OpenAI embedding function and then never sent for a DeepInfra model, so the provider refuses it up front rather than hand back full-size vectors into a store sized for fewer. A DeepInfra-specific embedding function that forwardsdimensionsis possible as a follow-up if there is interest.Runtime providers in this repo have not fetched model lists before. The closest precedent is the crew wizard's vendor catalog from #6462: a public endpoint and a short timeout, with a curated value as the default. Like #6462's non-interactive
--providerpath, an omitted model here stays on the hardcoded value; the lookup runs only when a user writes"auto", and then fails loudly rather than guessing. #7978 is the same provider without"auto"if you'd rather not have the lookup at all. Tests never touch the network.We (DeepInfra) maintain the API, the
sort_by=crewaiordering and this integration. The remaining follow-up from #7455, DeepInfra in the CLI provider picker, is #9316.The code was written with AI assistance (Claude Code), so per CONTRIBUTING the
llm-generatedlabel applies - I can't set labels on this repo, please add it.Note
Medium Risk
Introduces a new external embedding path (API keys and optional catalog HTTP) that can change vector dimension/model when
"auto"is used, though existing providers are unchanged.Overview
Adds
deepinfraas a first-class embedding provider for Crews, Knowledge, Memory, andEmbeddingService, wired through the RAG factory and typedProviderSpeclike OpenRouter.The new
DeepInfraProviderreuses Chroma’s OpenAI-compatible client against DeepInfra’s API, defaulting toQwen/Qwen3-Embedding-8Bwith credentials fromDEEPINFRA_API_KEY/DEEPINFRA_API_BASEonly (generic env vars likeMODEL_NAMEno longer bleed into config).model_name="auto"optionally resolves the first catalog model taggedembedvia a short HTTP lookup; failures raise with guidance to pin a fixed id, and successful picks are cached per process/base URL.dimensionsis rejected up front, andmodelis normalized tomodel_namebefore building the embedder.Knowledge docs in en/ar/ko/pt-BR gain an “Option 5” DeepInfra embedder example; tests cover factory wiring, env isolation, catalog
autobehavior, andcreate_deepinfra_service().Reviewed by Cursor Bugbot for commit af84028. Bugbot is set up for automated code reviews on this repo. Configure here.