Skip to main content
Each sandbox has a desktop. It starts when the sandbox starts. Open it in a browser tab to watch your agent or to help it. The desktop shows at 1080p and 60 frames each second, with sound. With no viewer, it uses about 350 MiB of memory in the sandbox. With the CLI, vms desktop <name> opens the desktop in your default browser. Add --mode browser to show only Chrome. See CLI reference.

Write the code that opens the desktop of a sandbox.

Before you start

  • You need an API key with the sandboxes:desktop scope. See API keys.
  • The sandbox must be running.

Open the desktop

In the app, open a running sandbox on the Sandboxes page and choose Open desktop. See Open the desktop. With the API, send POST /v1/sandboxes/{id}/desktop-sessions.
The answer is 201 with the link:
Open url in a browser. The link contains a token, so do not share it. To see only the Chrome window of the sandbox, send "mode": "browser". See Show only Chrome.
  • The link stops working at expiresAt, 10 minutes after the create, if nobody opens it.
  • The link opens one time. A second open answers 401.
  • A browser that opens the link gets a cookie and shows the desktop. The cookie works for 1 hour. Each load of the desktop page checks your access again and gives a new cookie for 1 hour.
  • If the member who made the link leaves the workspace, or you delete the API key that made it, the desktop page answers 403.
  • Each create gives a new link. Make a new link for each person or each browser.

Use the desktop

  • The mouse and the keyboard of your browser control the desktop.
  • A program in the sandbox shows on the desktop when it uses the display :0. Commands and SSH sessions have DISPLAY=:0. For example, run xterm &.
  • A computer-use agent in the sandbox can use the same display.

Show only Chrome

Send "mode": "browser" to get a link that shows only the Chrome window of the sandbox. You and a computer-use agent see the same page.
  • Chrome fills the screen and stays above the other windows.
  • If Chrome is not open, it opens when the page loads.
  • The sandbox has one screen. The last page that loads sets what it shows. A desktop link shows the other windows again.
  • This mode changes the view only. A person with the link can use the keyboard and the mouse as in desktop mode.
Chrome in the sandbox uses the profile /home/user/.config/vms-browser. It accepts the Chrome DevTools Protocol on 127.0.0.1:9222, so an agent in the sandbox can control the page that you see. To open Chrome from a command, run sudo systemctl start vms-desktop-browser.

Sound

The stream carries the sound of the sandbox. A program that plays sound in the sandbox plays in your browser tab. For example, play a video in Chrome, or run paplay sound.wav with a WAV file of your own.
  • Your browser starts the sound after you click or press a key in the tab.
  • Programs in the sandbox use PulseAudio. Commands and SSH sessions connect to it with no setup.

Networks and firewalls

The desktop streams the screen with WebRTC. It connects directly over IPv6 when it can. On a network that blocks a direct connection, such as a strict NAT or a company firewall, the desktop goes through a relay (TURN) on the host of the sandbox. You do not set anything.