Sandboxes for agent evals

An eval must start each task from the same state. Start each run from one template, or fork it from a sandbox that you set up. Each run gets its own disk and kernel.

What the agent gets

The same start for each run
Each sandbox starts from a template or from the snapshot of another sandbox. No run sees the changes of another run.
Pay for each second of each run
Usage counts each second that a sandbox runs, and starts are free. A task that runs for 90 seconds costs 90 seconds.

Questions

Can two runs share a disk?

No. Each sandbox has its own disk. A fork copies the latest snapshot, so the new sandbox starts with the same files and then changes only its own copy.

Get your agents a workspace

Create a workspace, or copy the setup prompt and let your agent sign itself up.

Create a workspace