mirror of
https://github.com/trycua/computer.git
synced 2026-02-18 04:19:38 -06:00
55 lines
1.8 KiB
Plaintext
55 lines
1.8 KiB
Plaintext
---
|
|
title: Prompt Caching
|
|
sidebar_position: 8
|
|
description: How to use prompt caching in ComputerAgent and agent loops.
|
|
---
|
|
|
|
Prompt caching is a cost-saving feature offered by some LLM API providers that helps avoid reprocessing the same prompt, improving efficiency and reducing costs for repeated or long-running tasks.
|
|
|
|
## Usage
|
|
|
|
The `use_prompt_caching` argument is available for `ComputerAgent` and agent loops:
|
|
|
|
```python
|
|
agent = ComputerAgent(
|
|
...,
|
|
use_prompt_caching=True,
|
|
)
|
|
```
|
|
|
|
- **Type:** `bool`
|
|
- **Default:** `False`
|
|
- **Purpose:** Use prompt caching to avoid reprocessing the same prompt.
|
|
|
|
## Anthropic CUAs
|
|
|
|
When using Anthropic-based CUAs (Claude models), setting `use_prompt_caching=True` will automatically add `{ "cache_control": "ephemeral" }` to your messages. This enables prompt caching for the session and can speed up repeated runs with the same prompt.
|
|
|
|
> **Note:** This argument is only required for Anthropic CUAs. For other providers, it is ignored.
|
|
|
|
## OpenAI Provider
|
|
|
|
With the OpenAI provider, prompt caching is handled automatically for prompts of 1000+ tokens. You do **not** need to set `use_prompt_caching`—caching will occur for long prompts without any extra configuration.
|
|
|
|
## Example
|
|
|
|
```python
|
|
from agent import ComputerAgent
|
|
agent = ComputerAgent(
|
|
model="anthropic/claude-3-5-sonnet-20241022",
|
|
use_prompt_caching=True,
|
|
)
|
|
```
|
|
|
|
## Implementation Details
|
|
- For Anthropic: Adds `{ "cache_control": "ephemeral" }` to messages when enabled.
|
|
- For OpenAI: Caching is automatic for long prompts; the argument is ignored.
|
|
|
|
## When to Use
|
|
- Enable for Anthropic CUAs if you want to avoid reprocessing the same prompt in repeated or iterative tasks.
|
|
- Not needed for OpenAI models unless you want explicit ephemeral cache control (not required for most users).
|
|
|
|
## See Also
|
|
- [Agent Loops](./agent-loops)
|
|
- [Migration Guide](./migration-guide)
|