Skip to content

feat(embeddings): add DeepInfra provider with an opt-in catalog model - #7993

Open
Georgi Atsev (ats3v) wants to merge 7 commits into
crewAIInc:mainfrom
ats3v:feat/deepinfra-embeddings-catalog-default
Open

Georgi Atsev (ats3v) wants to merge 7 commits into
crewAIInc:mainfrom
ats3v:feat/deepinfra-embeddings-catalog-default

Conversation

@ats3v

@ats3v Georgi Atsev (ats3v) commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Supersedes #7978, which is now a draft. This PR is the same DeepInfra embeddings provider plus model_name="auto", an opt-in that reads the model from DeepInfra's catalog, and a fix that keeps other tools' environment variables out of the provider. Without "auto" it behaves like #7978: a static default and no network request.

Changed after review: the catalog lookup used to run whenever no model was configured and fell back to the static default on failure. It now runs only for "auto" and fails with an actionable error instead of falling back.

Related issue

Relates to #7455 (closed by #7459, which listed the embeddings provider as a follow-up). Supersedes #7978.

Summary

Add deepinfra as an embedding provider for Crews, Agents, Memory and Knowledge. Without a configured model it embeds with Qwen/Qwen3-Embedding-8B; model_name="auto" opts into the model DeepInfra currently recommends.

Provider

Same shape as the OpenRouter provider from #7127: a DeepInfraProvider on crewAI's native OpenAIEmbeddingFunction (the one Azure and OpenRouter use since #8934) with the DeepInfra base URL, registered in PROVIDER_PATHS, ProviderSpec, AllowedEmbeddingProviders and the crewai-tools EmbeddingService (plus create_deepinfra_service()). No new module dependency.

crew = Crew(
    ...,
    embedder={
        "provider": "deepinfra",
        "config": {"model_name": "Qwen/Qwen3-Embedding-8B"},  # api_key defaults to DEEPINFRA_API_KEY
    },
)
  • base URL https://api.deepinfra.com/v1/openai, DEEPINFRA_API_BASE or api_base overrides it
  • model_name defaults to Qwen/Qwen3-Embedding-8B; model is accepted as an alias; default_headers passes through
  • dimensions is rejected with a ValidationError that says why: the OpenAI embedding function only forwards it for OpenAI text-embedding-3 models, so for a DeepInfra model the vectors would silently keep the model's full size (the openai and openrouter providers accept and drop it today)
  • api_key from config or DEEPINFRA_API_KEY; a missing key is a ValidationError
  • only DEEPINFRA_API_KEY and DEEPINFRA_API_BASE are read from the environment. The settings base would otherwise match every field name and alias against it, ignoring case, so MODEL_NAME (crewAI's chat model), API_BASE or a generic API_KEY would configure the embedder. Config values still win over the environment.
  • docs: "Option 5" in the knowledge embedder examples in en, ar, ko and pt-BR, code block identical in all four

One deliberate difference from the OpenAI/OpenRouter providers: the model alias is popped before validation instead of copied. The provider allows extra fields and forwards all of them to the embedding function, which has no model parameter, so copying leaves build_embedder({"provider": ..., "config": {"model": ...}}) failing with TypeError: OpenAIEmbeddingFunction.__init__() got an unexpected keyword argument 'model'. The openai and openrouter providers behave that way on main today; not touched here, happy to send a separate fix.

Opt-in: model_name="auto" reads DeepInfra's catalog

embedder={"provider": "deepinfra", "config": {"model_name": "auto"}}  # first `embed` entry of the catalog, today Qwen/Qwen3-Embedding-8B
  • Only "auto" (as model_name or model) triggers the lookup. An omitted model is the static default and sends no request, so what an omitted model means never depends on the vendor.
  • GET {api_base}/models?filter=with_meta&sort_by=crewai is public (no key sent), 3 s timeout; the first entry whose metadata.tags contains embed wins. The ordering is maintained server-side by DeepInfra for agent frameworks.
  • On any failure (connection error, HTTP error, malformed payload, no embed entry) construction fails with a ValidationError that names the catalog URL and says to set model_name to a fixed id. No other model is substituted, so an outage cannot switch a store's embedding space.
  • A successful pick is kept for the life of the process, per base URL (trailing slash stripped): every store a process opens shares one embedding space. Failures are not cached, so the next provider can succeed once the catalog is back.
  • The catalog is read from the provider's own api_base (config, then DEEPINFRA_API_BASE, then the default), the same host the embeddings go to. EmbeddingService.create_deepinfra_service(model="auto") passes "auto" through to the provider.
  • Docs: the knowledge example notes that "auto" uses the model DeepInfra currently recommends, which can change between runs, in en, ar, ko and pt-BR.

The "auto" pick can change between runs by design, so persisted Knowledge or Memory stores should keep a fixed id, as the default does.

Verification

  • Tests added or updated for the changed behavior

  • Relevant tests and quality checks pass locally

  • uv run pytest lib/crewai/tests/rag/embeddings lib/crewai-tools/tests/rag -q - 538 passed, 2 skipped, after uv sync --all-groups --all-extras. New in test_factory_deepinfra.py: for the provider, factory wiring, the native function's client pointed at DeepInfra's base URL, model alias through a real OpenAIEmbeddingFunction, dimensions rejected directly and through the factory, defaults, overrides, env vars and a missing key, and generic variables (MODEL_NAME, API_BASE, API_KEY, DEFAULT_HEADERS) leave the provider alone; for "auto", the default never reads the catalog, the first embed entry wins in catalog order (via model_name and model), a custom api_base and DEEPINFRA_API_BASE are queried, an explicit model skips the lookup, a ValidationError on connection error / HTTP error / malformed payload / invalid JSON / no embedding model, successful picks cached per normalized base URL, failed lookups not cached, and an embedder built with "auto" through a real OpenAIEmbeddingFunction. Plus cases in test_embedding_factory.py, test_backward_compatibility.py and the crewai-tools test_embedding_service.py (static default, "auto" passed through to the provider). An autouse fixture keeps every lookup off the network.

  • uv run ruff check and uv run ruff format --check on the changed files - clean

  • uv run mypy on the provider package, factory.py, types.py and the crewai-tools embedding_service.py - no issues

  • live against api.deepinfra.com: every embedder below is crewAI's native function and emits no DeprecationWarning; build_embedder({"provider": "deepinfra", "config": {}}) embeds with Qwen/Qwen3-Embedding-8B (4096-dim vectors) without a catalog request; model_name="auto" resolves to Qwen/Qwen3-Embedding-8B from the catalog and embeds; model="auto" against an unreachable base fails with the ValidationError naming the URL. Earlier on this branch: model="Qwen/Qwen3-Embedding-4B" through the alias embeds (2560-dim); EmbeddingService.create_deepinfra_service() with embed_text, embed_batch, get_embedding_dimension() = 4096 and validate_connection() = True; with MODEL_NAME and API_BASE set for other tools, the embedder keeps its own settings

  • branch rebased on main at d7dfa0e

Additional context

No behaviour change for existing providers.

dimensions would be accepted by the OpenAI embedding function and then never sent for a DeepInfra model, so the provider refuses it up front rather than hand back full-size vectors into a store sized for fewer. A DeepInfra-specific embedding function that forwards dimensions is possible as a follow-up if there is interest.

Runtime providers in this repo have not fetched model lists before. The closest precedent is the crew wizard's vendor catalog from #6462: a public endpoint and a short timeout, with a curated value as the default. Like #6462's non-interactive --provider path, an omitted model here stays on the hardcoded value; the lookup runs only when a user writes "auto", and then fails loudly rather than guessing. #7978 is the same provider without "auto" if you'd rather not have the lookup at all. Tests never touch the network.

We (DeepInfra) maintain the API, the sort_by=crewai ordering and this integration. The remaining follow-up from #7455, DeepInfra in the CLI provider picker, is #9316.

The code was written with AI assistance (Claude Code), so per CONTRIBUTING the llm-generated label applies - I can't set labels on this repo, please add it.


Note

Medium Risk
Introduces a new external embedding path (API keys and optional catalog HTTP) that can change vector dimension/model when "auto" is used, though existing providers are unchanged.

Overview
Adds deepinfra as a first-class embedding provider for Crews, Knowledge, Memory, and EmbeddingService, wired through the RAG factory and typed ProviderSpec like OpenRouter.

The new DeepInfraProvider reuses Chroma’s OpenAI-compatible client against DeepInfra’s API, defaulting to Qwen/Qwen3-Embedding-8B with credentials from DEEPINFRA_API_KEY / DEEPINFRA_API_BASE only (generic env vars like MODEL_NAME no longer bleed into config). model_name="auto" optionally resolves the first catalog model tagged embed via a short HTTP lookup; failures raise with guidance to pin a fixed id, and successful picks are cached per process/base URL. dimensions is rejected up front, and model is normalized to model_name before building the embedder.

Knowledge docs in en/ar/ko/pt-BR gain an “Option 5” DeepInfra embedder example; tests cover factory wiring, env isolation, catalog auto behavior, and create_deepinfra_service().

Reviewed by Cursor Bugbot for commit af84028. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitai Bot commented Oct 9, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ats3v
Georgi Atsev (ats3v) force-pushed the feat/deepinfra-embeddings-catalog-default branch from 7dc1fcc to 066dbb7 Compare October 9, 2026 12:43
@ats3v
Georgi Atsev (ats3v) force-pushed the feat/deepinfra-embeddings-catalog-default branch from 066dbb7 to b3260b8 Compare November 10, 2026 06:08
@mintlify

mintlify Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit b1f5b3f.

Approve preview

@strix-security

strix-security Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 79 commits after the last Strix review (b3260b8). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Adds a `deepinfra` embedding provider for Crews, Agents, Memory and Knowledge, built on chromadb's OpenAI-compatible embedding function, and picks its default embedding model at runtime from DeepInfra's public model catalog when no model is configured, falling back to a static default. Also registers the provider in the embedding factory and in crewai-tools' `EmbeddingService`, with docs and tests.

Review summary

No security issues found in this PR. The new deepinfra embedding provider follows the existing OpenAI/OpenRouter provider shape, and the one genuinely new behaviour — the runtime catalog lookup used to pick a default embedding model — was traced end to end: the request carries no credentials, does not follow redirects, is capped at a short timeout, and the model id it returns only ever reaches the JSON body of the embeddings request at the same operator-configured base URL. Parsing of the catalog payload fails closed to the static default on every malformed shape tested, and the process-wide pick cache is isolated per base URL. The dimensions guard holds both directly and through the factory, and the model alias does not let extra config fields bypass it or substitute the embedding callable. Two residual secret-handling concerns (pydantic validation-error text carrying the key into a pre-existing error log, and the config-derived base URL appearing in a catalog-failure warning) were independently validated and rejected: both are operator-supplied values read only by the operator, and the logging paths predate this PR and apply to every provider. Coverage spanned the provider package, factory registration, embedding types, the crewai-tools service changes, and the docs.

Ruled out (6)
  • Can an untrusted party drive the new catalog lookup into an SSRF by controlling api_base or DEEPINFRA_API_BASE?
    Every producer of the base URL is developer or operator configuration (Crew/Memory/Knowledge embedder config, RagTool config, crew.jsonc, the memory CLI flag, the env var), no HTTP route or tool argument reaches it, httpx does not follow redirects for the catalog GET, and the request carries no API…
  • Can a forged or intercepted catalog response do more than choose a model name?
    The catalog id is only ever placed in the JSON body of the embeddings request (never the URL path, a header, or a filesystem/exec sink), so a hostile catalog grants nothing beyond what controlling the configured embedding host already grants.
  • Can the process-wide catalog pick cache poison another caller or split a persisted store across models?
    Picks are isolated per base URL string, growth is bounded by the number of distinct configured bases, and the cross-process dimension caveat is explicitly documented with the instruction to pin model_name.
  • Does the new dimensions guard leak the DeepInfra API key into logs and exception text?
    Pydantic does render part of the key in the ValidationError input dict, but the logging and re-raise it flows into are unchanged by this PR and already reachable via the existing watsonx validator, the key is supplied and the log read by the same operator, and telemetry records only exception class…
  • Does the catalog-failure warning expose credentials embedded in the configured base URL?
    Only an operator who deliberately puts userinfo in their own base URL can put a secret there, the credential channel api_key never reaches that log statement, and the destination is the operator's own process log.
  • Can an embedder config dict inject an arbitrary embedding_callable or bypass the dimensions rejection?
    The callable field is type-enforced to the bound embedding-function class (dotted strings, plain functions and unrelated classes are all rejected), and the dimensions guard fires on any non-None value both directly and through the factory, including via environment variables.

Review 1 on this PR · b3260b8 · requested Nov 10 06:41 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@iris-clawd iris-clawd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn’t independently verify DeepInfra’s catalog-ordering guarantee.

The notes inline on deepinfra_provider.py are what I'd want addressed before this merges. The other inline notes are small.

Someone else should sign off on this too: the change introduces public provider/configuration keys and changes the meaning of an omitted model from a fixed package value to a vendor-controlled runtime selection.

@ats3v Georgi Atsev (ats3v) changed the title feat(embeddings): pick the DeepInfra default model from its catalog feat(embeddings): add DeepInfra provider with an opt-in catalog model Nov 13, 2026
@ats3v
Georgi Atsev (ats3v) force-pushed the feat/deepinfra-embeddings-catalog-default branch from 592bc1f to af84028 Compare November 14, 2026 05:25

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit af84028. Configure here.

@ats3v
Georgi Atsev (ats3v) force-pushed the feat/deepinfra-embeddings-catalog-default branch from 89f1183 to 60122fb Compare November 16, 2026 05:34
Add a DeepInfraProvider built on the OpenAI embedding function with the
DeepInfra base URL, register it in the factory, the provider spec union and
the crewai-tools EmbeddingService, and document it next to the OpenRouter
example in en, ar, ko and pt-BR. Defaults to BAAI/bge-m3 and reads
DEEPINFRA_API_KEY.

The `model` alias is popped before validation instead of copied, because the
provider forwards its extra fields to the embedding function, which has no
`model` parameter.
Use the more popular Qwen/Qwen3-Embedding-8B as the default embedding
model in the provider, the crewai-tools EmbeddingService and the docs
example in en, ar, ko and pt-BR.
chromadb's OpenAIEmbeddingFunction only forwards `dimensions` for OpenAI
text-embedding-3 models, so a DeepInfra provider that accepted the field
returned full-size vectors into a store sized for fewer. Refuse the key
with a ValidationError that says why, and drop the field and its
DEEPINFRA_DIMENSIONS alias from the provider and the config TypedDict.
When no model is configured, read DeepInfra's OpenAI-compatible model
list sorted for crewAI (GET {api_base}/models?filter=with_meta&sort_by=crewai)
and use the first entry tagged "embed", so the default follows what
DeepInfra recommends without a crewAI release. The lookup is public, has a
3 s timeout and falls back to Qwen/Qwen3-Embedding-8B with a warning when
the catalog cannot be read or lists no embedding model. Failures are not
cached; a successful pick is kept for the life of the process per base
URL, so every store a process opens shares one embedding space. A
configured model_name skips the lookup.

The crewai-tools EmbeddingService uses the same resolver with the
provider's base URL precedence (extra_config api_base, then
DEEPINFRA_API_BASE, then the default), so the model is chosen from the
host the embeddings go to.
The settings base matches every field name and alias against the
environment, ignoring case. MODEL_NAME, which crewAI reads for the chat
model, became the embedding model and skipped the catalog lookup, and
API_BASE and API_KEY, which belong to other tools, redirected the
requests and supplied the key sent to DeepInfra. The provider now takes
its config, then DEEPINFRA_API_KEY and DEEPINFRA_API_BASE only.
A default read from DeepInfra's catalog can change between restarts, and a
persisted store then queried with a different model of the same dimension
loses retrieval quality without any error. Keep the default static and read
the catalog only when asked for.

- Without a model the provider uses Qwen/Qwen3-Embedding-8B again, with no
  network request. model_name="auto" (or model="auto") reads the catalog;
  crewai-tools' EmbeddingService passes "auto" through to the provider.
- A failed lookup raises a ValidationError that names the catalog URL and
  says to set model_name, instead of substituting another model.
- Picks are cached per base URL with any trailing slash stripped. Failures
  are not cached, so the next provider can succeed once the catalog is back.
- effective_api_base is gone: the provider resolves its own base URL.
…ction

Since crewAIInc#8934 the Azure and OpenRouter providers use crewAI's own
OpenAIEmbeddingFunction, and the factory's DeepInfra overload already
promised that class. The provider still built chromadb's function, so it
returned a different type than declared and emitted chromadb's
DeprecationWarning about the api_key not being persisted.

The dimensions check stays: the native function also sends dimensions only
for OpenAI text-embedding-3 models.
@ats3v
Georgi Atsev (ats3v) force-pushed the feat/deepinfra-embeddings-catalog-default branch from 60122fb to b1f5b3f Compare November 20, 2026 08:07
@ats3v

Copy link
Copy Markdown
Contributor Author

João Moura (@joaomdmoura) could you re-run the review gate here? Its notes are addressed: the default model is static again, and the catalog lookup is opt-in via model_name="auto". CI is green. The gate also asked for a second sign-off. Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants