Sandboxes for agent evals
An eval must start each task from the same state. Start each run from one template, or fork it from a sandbox that you set up. Each run gets its own disk and kernel.
What the agent gets
- The same start for each run
- Each sandbox starts from a template or from the snapshot of another sandbox. No run sees the changes of another run.
- Pay for each second of each run
- Usage counts each second that a sandbox runs, and starts are free. A task that runs for 90 seconds costs 90 seconds.
Questions
Can two runs share a disk?
No. Each sandbox has its own disk. A fork copies the latest snapshot, so the new sandbox starts with the same files and then changes only its own copy.
Get your agents a workspace
Create a workspace, or copy the setup prompt and let your agent sign itself up.