> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sailresearch.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run Harbor tasks on Sailboxes

> Run every Harbor trial in its own Sailbox with one flag

[Harbor](https://harborframework.com) is a framework for evaluating and
training agents on sandboxed tasks. Each trial runs an agent against one
task inside an isolated environment, then a verifier scores the result.
Harbor includes environments for Docker on your own machine and for several
cloud sandboxes, and loads any other environment by import path. Sail
provides one:

```text theme={null}
sail.harbor:SailboxEnvironment
```

Pass that import path to Harbor and every trial's environment becomes a
Sailbox. Tasks, agents, verifiers, and datasets are unchanged.

## Quickstart

Install `sail` and `harbor` in the same Python environment (Harbor needs
Python 3.12 or newer) and set your Sail API key. Then run Harbor's
`hello-world` dataset with the `oracle` agent, which runs each task's
reference solution and doesn't need a model key:

```bash theme={null}
pip install sail harbor
export SAIL_API_KEY=sk_your_key_here

harbor run -d hello-world@1.0 --agent oracle \
  --env sail.harbor:SailboxEnvironment -o ./jobs
```

Harbor creates a Sailbox, runs the task's solution in it, runs the verifier,
writes the trial's results under `./jobs`, and terminates the Sailbox when
the trial ends.

A real evaluation is the same command with a real agent, a model, and more
trials in flight. Each trial gets its own Sailbox, so the machine running
Harbor is no longer the limit on concurrency:

```bash theme={null}
harbor run -d terminal-bench@2.0 --agent <agent> --model <model> \
  --env sail.harbor:SailboxEnvironment -n 32
```

A job config file (`harbor run -c job.yaml`) selects the environment the
same way:

```yaml theme={null}
environment:
  import_path: sail.harbor:SailboxEnvironment
```

## Credentials and app

The process that runs Harbor needs a Sail credential: set `SAIL_API_KEY`, or
run `sail auth login` once on that machine. Sailboxes are created in the
`harbor` [app](/sailbox-sdk-apps) in your organization, which is created on
first use. Set `SAIL_APP` to use a different app. Each Sailbox is named
`harbor-` followed by the trial's session id, and `sail box list --app harbor`
(or the app you chose) lists the ones a run created.

## Task images

Harbor decides where an environment's image comes from, and Sail follows the
same rules as Harbor's other cloud providers.

* A task that includes an `environment/Dockerfile` has it built into a Sailbox
  image. The build happens once per organization: the first trial of that
  task waits for it, and later trials start from the cached image.
  `--force-build` rebuilds it. See
  [Build an image from a Dockerfile](/sailboxes-images#build-an-image-from-a-dockerfile)
  for what a Dockerfile can contain.
* A task that declares a `docker_image` in its `task.toml` runs on that
  image, pulled from its registry. Docker-style short references work:
  `python:3.11` means `docker.io/library/python:3.11`. The registry must be
  one Sail supports (`docker.io`, `ghcr.io`, `public.ecr.aws`, or `quay.io`),
  and the image must be Debian or Ubuntu based. See
  [Bring your own base image](/sailboxes-images#bring-your-own-base-image)
  for the full requirements.
* A task can have both. Harbor treats `docker_image` as the published build
  of the Dockerfile and runs on that image unless you pass `--force-build`.
* A task that includes an `environment/docker-compose.yaml` runs as a
  [Docker Compose project](#docker-compose-projects).

### Docker Compose projects

The project runs inside the Sailbox. Commands run in the task's `main`
service, and Harbor's per-service operations (exec, download, stop) reach the
other services. If the task also has a `docker_image` or a Dockerfile, `main`
runs on that image, and the compose file's `main` entry adds settings such as
`environment` or `ports`.

Sail builds the service images into the Sailbox image, cached and rebuilt the
same way as a Dockerfile build. The compose files follow the rules in
[Build an image from a Compose project](/sailboxes-images#build-an-image-from-a-compose-project).
A project that uses an unsupported feature fails before a Sailbox is created,
and the error identifies the feature.

<Accordion title="Compose variables">
  * `${VAR}` takes its value from the environment of the machine running
    Harbor, then from the project's `.env`, then from the default in the
    expression.
  * `DOCKER_*` and `COMPOSE_*` names are never read from the machine, and an
    `env_file` path written with `${VAR}` is rejected.
  * The build sees `.env` and the machine variables that pick a service's
    image: the ones under `image`, `build`, `platform`, and `extends`, and any
    `.env` itself uses. Other machine variables are passed to the project only
    when it starts in the Sailbox, so a secret a service's `environment` reads
    from the machine never enters the build.
</Accordion>

## Sizing

A task's CPU, memory, and storage requests pick the smallest
[Sailbox size](/sailboxes-pricing) that covers them: `s` (1 vCPU), `m`
(4 vCPU), or `l` (8 vCPU). A request above the size's default memory or
disk ceiling raises that ceiling, up to the size's maximum. A task that
doesn't declare resources gets an `s` Sailbox. A task that requests more
than the largest size provides fails up front, before a Sailbox is created,
and the error names the request and that size's limits.

The size is a ceiling, not a reservation: a Sailbox is billed for the CPU,
memory, and disk it actually uses, so a task that requests 8 CPUs and uses
one pays for one.

## Cleanup

By default, `harbor run` deletes each environment when its trial ends, which
terminates the Sailbox. A termination that fails is reported in that trial's
result, like any other trial error.

With `--no-delete`, Harbor stops each environment without deleting it. The
Sailbox goes to sleep with its filesystem and processes intact, and billing
stops until it is resumed. Terminate those Sailboxes yourself when you are
done with them, with `sail box terminate <id>` or from the dashboard.

A trial whose environment fails to start terminates its Sailbox before the
error is reported, even with `--no-delete`. If that termination fails, the
trial's normal cleanup stops the Sailbox the same way it would after a
completed trial.

## Unsupported task features

Harbor checks a task's needs against the environment before starting it, so a
task that needs something unsupported fails up front with a clear error
instead of running without the feature. Sailboxes do not offer:

* GPUs, TPUs, or Windows environments.
* IPv6 allowlist entries. A Sailbox does not reach the internet over IPv6.

The [egress policy reference](/reference/egress-policy#entries) has the
allowlist entry rules.

Mounts declared for a task are ignored, except in a Compose task, where they
are bound into its `main` service from the Sailbox's filesystem.
