[paoValle]

Motivation

A semantic cache saves latency and money, but RedisVL gives no built-in way to see whether it is working: the index statistics it exposes say how many entries exist, not how many checks hit, how many missed, or how long a lookup took. Today every deployment writes its own counters around check()/store(). Closes #684.

Changes

A CacheStats snapshot on every cache

BaseLLMCache.stats returns a frozen CacheStats with checks, hits, misses, errors, stores, lookup_seconds, and the derived hit_rate and avg_lookup_seconds. Absent from hits + misses + errors is impossible by construction: a check that raises increments checks and errors and is never counted as a miss.

cache.check(prompt="What is the capital of France?")
cache.store(prompt="What is the capital of Italy?", response="Rome")
cache.stats.hit_rate        # 0.0
cache.stats.avg_lookup_seconds

Counting in decorators, applied once per method

The counters live in redisvl/extensions/cache/llm/stats.py as observe_check/observe_acheck/count_store/count_astore, applied to check, acheck, store and astore on SemanticCache and LangCacheSemanticCache. functools.wraps keeps the public signatures, docstrings and iscoroutinefunction intact, and there is a test that says so.

Per process, per object, no content

Nothing is written to Redis, so two workers sharing a cache name do not aggregate, and no prompt, response or metadata is recorded. The increments are not locked, so a cache object shared across threads counts best-effort.

Release Notes

Every LLM cache now exposes cache.stats, with hits, misses, errors, completed stores, hit rate and average lookup time. The counters are per cache object and per process; nothing changes on the wire, and sharing a cache object across threads makes the totals approximate.

Reviewer Brief

Start at stats.py — the four decorators and the CacheStats dataclass — then base.py for _record_check/_record_error and the stats property. The risk is in the wrappers rather than the arithmetic: an async wrapper that swallowed an exception would look fine in the happy-path tests, so tests/unit/test_cache_stats.py pins the async error path and the preserved signature separately.

Not verified: the integration test in tests/integration/test_llmcache.py needs a live Redis and was not run here; the unit tests mock the LangCache client and ran green (pytest tests/unit — 1360 passed, with the mcp and google-genai modules failing to collect for missing optional dependencies, as they do on main).

---

Note

Low Risk
Additive observability around existing check/store methods; cache semantics and Redis behavior are unchanged aside from lightweight in-process counting.

Overview
Adds built-in, in-process usage metrics for LLM caches so callers can read hit/miss behavior without wrapping every check/store call.

BaseLLMCache.stats returns a frozen CacheStats snapshot: checks, hits, misses, errors, stores, total lookup time, plus derived hit_rate (errors excluded from the denominator) and avg_lookup_seconds. Counters live on each cache instance only—nothing is persisted to Redis and no prompt/response content is recorded.

Instrumentation is applied via observe_check / observe_acheck / count_store / count_astore on SemanticCache and LangCacheSemanticCache. Failed checks increment errors (not misses); stores count only on successful completion. CacheStats is exported from the public LLM cache module and documented in docs/api/cache.rst.

Integration tests cover SemanticCache; unit tests (mocked LangCache client) cover error paths, snapshot immutability, per-instance isolation, and preserved method signatures.

^{Reviewed by Cursor Bugbot for commit 27393bf. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vishal-bala]

Thanks for picking up #684! There's an open OpenTelemetry stack for the same issue (#755, #776, #777). It already records checks, hits, misses, errors, stores and lookup latency for SemanticCache, and it decorates the same four methods, so this PR would duplicate it and conflict with it.

What's still missing is LangCacheSemanticCache, which the stack doesn't instrument yet. Could you rework this PR to cover just that, on top of #777?

• Drop stats.py, BaseLLMCache.stats and the SemanticCache decorators.
• Decorate the LangCache check/acheck with _telemetry.operation("check", lookup=True), store/astore with operation("store"), and the delete_by_* methods with operation("drop").
• Skip the retrieval span and the distance histogram, since LangCache's search is an HTTP call and its misses return no candidates.
• Test against an in-memory metric reader, as tests/unit/test_cache_telemetry.py does. Your mocked LangCache fixture should still work there.
• Add LangCache to the observability guide.

Keep targeting main, and rebase once the stack merges.

[paoValle]

The rework is written and passes locally: LangCache check/acheck as lookups, store/astore, and the four delete_by_* as drops, with no retrieval span and no distance point, a unit test that reads the spans and duration points back through an in-memory reader, and a LangCache section in the observability guide.

I am holding the push until #777 merges. Building on a module that is not on main yet would put the whole OTel stack in this PR's diff, which is what your review asked me to avoid. The moment it lands I will rebase this branch onto main and push.