Response cache
The Advanced step of the policy wizard has one control, and it exists mainly so you can turn something off.
qRaptor reuses a stored answer when the same request repeats — it returns immediately, and it costs no credits. That’s a straightforward win for most work, so it’s on by default. But if your agent handles data that must not be retained, storing a response is exactly what you don’t want, and this is where you say so.
Turning it off
Switch Enable Response Cache off when:
- Responses carry regulated data — health records, financial detail, anything under a retention rule. The HIPAA Strict template ships with this off for precisely this reason.
- An audit requirement says every answer must come from a genuine provider call.
That’s the whole decision. Everything below is detail for when you want to know what you’re switching.
Turning it off has a real cost: identical requests are re-run and re-billed every time. Do it because you have a reason, not as a precaution.
When caching actually applies
The toggle is a permission, not an override. Switching it on doesn’t make everything cacheable — it permits caching for requests that already qualify. A request qualifies when both are true:
- temperature is 0, and
- no tools are attached to the request.
Anything creative or tool-using is never served from cache, whatever the policy says. Serving a stored answer for a request whose whole point is variation would be a footgun.
In practice:
| What’s running | Cacheable |
|---|---|
A Quick Prompt node with its temperature set to 0 | Yes |
| A code node calling the LLM SDK at temperature 0 with no tools | Yes |
| A Quick Prompt node left at its default temperature | No |
| An AI Agent conversation | No — agents call tools |
The sweet spot is a deterministic step inside a Task Agent — classification, extraction, or generating a fixed format. Set that node’s temperature to 0 and repeated runs over the same input stop costing anything.
Caching works on both streaming and non-streaming calls. A cached streamed reply is replayed as a stream, so it reads identically to a live one.
The three states
| State | Meaning |
|---|---|
| On | Caching is explicitly permitted |
| Off | Caching is explicitly disabled for agents using this policy |
| Untouched | The policy expresses no opinion; the platform default applies, which is on for qualifying requests |
That’s why an attached policy’s summary shows Response Cache: Disabled only when it was switched off deliberately. No entry means no decision was made, not that caching is off.
What this is not
- Not prompt caching at the provider. That’s a separate provider-side optimisation, not controlled here.
- Not a per-conversation cache. Two people asking the same thing in different conversations can share a cached answer — the key is the request, not the session.
- Not tunable. On and off; there’s no cache-key strategy to choose.
Common issues
- Nothing is being cached — check the request qualifies. Temperature above 0, or any tool attached, and it never will. See the table above.
- A cached answer looks stale — that’s the trade. Switch caching off in the policy if that answer must be recomputed every time.
- You turned it on and nothing changed — expected if the requests weren’t eligible to begin with. The toggle permits caching; it doesn’t cause it.
Next
- Apply a policy to an agent — making any of this take effect.