# RL environments: thousands of isolated rollouts

URL: https://munito.ai/solutions/rl-environments

Give every reinforcement-learning rollout a machine of its own, started in about a second and billed per second. One run started 32,752 of them at once.

## Thousands of isolated rollouts, started in about a second each

Reinforcement learning on real tasks needs a real environment: a machine the policy can break, reset, and break again. The hard part is never one environment. It is thirty thousand of them, cheaply, now.

Munito AI starts a sandbox in about a second and bills it by the second. One run started **32,752 of them at once**, through the public API, from Paris.

## Why a sandbox fits a rollout

| A rollout needs      | A sandbox gives it                                            |
| -------------------- | -------------------------------------------------------------- |
| Isolation            | Its own machine; one rollout cannot reach another               |
| A fast start         | About a second, so the environment is not the bottleneck        |
| A clean reset        | Throw it away and fork a new one from the same snapshot         |
| A real shell         | Commands, files, packages, ports, a browser, a desktop          |
| Honest cost          | Per second, from $0.0073 an hour for `tiny`                     |

A `tiny` sandbox is 1 vCPU and 128 MiB at $0.0073125 an hour. A `default` is 2 vCPUs and 4 GiB at $0.018. Saved state costs $0.0002 per GiB-hour, about $0.15 per GiB-month.

## Fork from a snapshot, do not rebuild

Build the environment once: the repository, the dependencies, the fixtures, the service it talks to. Pause it. Then fork it for every rollout.

A fork starts with the whole tree already in place, so the policy begins at step one instead of waiting on an install. That is the difference between a batch of rollouts that takes minutes and one that takes hours. See [Persistence](https://docs.munito.ai/sandboxes/persistence).

## Scale is the product

Starting one machine is easy. Starting thirty thousand, keeping them isolated, and tearing them down without leaking spend is the part that is hard to build and boring to maintain.

The board on the [Sandboxes page](https://munito.ai/sandboxes) is a real run of exactly that: one sandbox for each pixel of the wordmark, 32,752 at once, counted live.

Use [networks](https://docs.munito.ai/sandboxes/networks) to decide what a rollout may reach, which for an untrusted policy is usually nothing.

## Grade the rollouts with the same account

A reward model is an inference call. Run the policy on sandboxes, grade the transcripts with an open model through the same key, and keep both in the region you chose. Structured outputs make a grade a field rather than a sentence to parse.

Open models matter here for a second reason: you can serve the model you are training against, not only the one a vendor sells. A [dedicated endpoint](https://docs.munito.ai/dedicated-endpoints) runs an open model that the public catalogue does not list.

## What we do not do

We do not train. There is no fine-tuning API, no trainer, no checkpoint store. We run the environments and the inference; the learning loop is yours, and so are the weights.

If you need GPUs by the hour to train on, that is a different product and we will say so.

## The parts a security review asks about

Rollouts run untrusted code by design. Each sandbox is isolated, the network policy is yours to set, and nothing persists unless you ask it to. Inference keeps no prompts and no outputs by default, and trains on nothing you send.

The whole platform can run inside your own cloud account or on-premises. See [Enterprise](https://munito.ai/enterprise).

## Start

- [Read the Sandboxes guide](https://docs.munito.ai/sandboxes)
- [Fork from a snapshot](https://docs.munito.ai/sandboxes/persistence)
- [Control what a rollout reaches](https://docs.munito.ai/sandboxes/networks)
- [Talk to an engineer](https://munito.ai/contact) about the scale and the shape of your loop