> ## Documentation Index
> Fetch the complete documentation index at: https://gateway.forceaisecurity.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Response caching

> Serve a repeated request from Redis instead of calling the provider, cutting cost and latency. How it is wired in ForceAI, how to tune it, and how to see cache hits.

Response caching stores the answer to a request keyed by its content. When an identical request comes in again, the gateway returns the stored answer instead of calling the provider, so a cache hit costs nothing and returns immediately. This is distinct from Token Optimizer: caching skips the call on a repeat, Token Optimizer compresses the prompt on every call. The two stack

<Info>
  Caching is a gateway (data-plane) feature. The control plane shares the same Redis so the Caching page's health check reports correctly, but the actual response caching happens where the LLM calls run
</Info>

## How it is wired in ForceAI

The stack ships a Redis service and a small config that turns caching on:

* A pinned `redis` service in the compose (cache only, no persistence)
* `deploy/gateway/cache_config.yaml` with `litellm_settings.cache: true` pointing at Redis, mounted into the gateway and backend via `CONFIG_FILE_PATH`
* Database-stored models still load on top of the config, so nothing about routing changes

The config caches completion and embedding call types with a one-hour TTL:

```yaml theme={null}
litellm_settings:
  cache: true
  cache_params:
    type: redis
    host: redis
    port: 6379
    ttl: 3600
    supported_call_types:
      - completion
      - acompletion
      - embedding
      - aembedding
```

## Tune it

Edit `deploy/gateway/cache_config.yaml` and recreate the gateway (and backend) so the mounted config is re-read:

* `ttl` sets how long a cached entry lives, in seconds
* `supported_call_types` limits what gets cached; drop `embedding`/`aembedding` if you only want chat responses
* Point `host`/`port` at an external Redis instead of the bundled one for a shared or managed cache

## See cache hits

Open the dashboard **Experimental -> Caching** page. The health tile shows a healthy Redis cache, and the analytics show the hit ratio, cached responses, and cached tokens over a date range. Send the same prompt twice from the Playground and watch the cached-rows count go up

## Verify by API

Two identical completions; the second is served from cache (same response id, no new provider call):

```bash theme={null}
BODY='{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"Say the single word: pumpernickel"}],"temperature":0}'

curl -s http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $FORCEAI_KEY" -H "Content-Type: application/json" -d "$BODY" | jq -r '.id'
curl -s http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $FORCEAI_KEY" -H "Content-Type: application/json" -d "$BODY" | jq -r '.id'
```

The two `id` values match on a hit. The cached call also records `cache_hit=true` in the spend logs, which is what the Caching page's analytics read

Check the cache backend directly:

```bash theme={null}
curl -s http://localhost:3000/cache/ping -H "Authorization: Bearer $FORCEAI_KEY" | jq '{status, cache_type}'
```

A healthy cache returns `{"status": "healthy", "cache_type": "redis"}`

<Warning>
  Caching runs after the pre-call hooks, so a cache hit still respects guardrails and access-group gating; it only avoids the provider round-trip. Two requests are the same entry only when their model, messages, and parameters (including temperature) match exactly
</Warning>