docs

Get started

GPU Zero has two halves. Ravex and Moonclip are open-source libraries that checkpoint a PyTorch training run and resume it where it stopped; they need no account. The console runs your repository on GPUs in your own cloud account, shows each run's metrics, and lets you resume or fork a run from a real checkpoint.

1. Checkpoint and resume with Ravex

Ravex needs Python 3.11 or later and the PyTorch you already have.

pip install "ravex[moonclip]"

This runs as it is, on a CPU, in a few seconds. Save it as train.py:

import torch
import ravex

@ravex.train_loop(max_steps=2000)
def train():
    torch.manual_seed(0)
    model = torch.nn.Sequential(torch.nn.Linear(16, 64), torch.nn.ReLU(), torch.nn.Linear(64, 1))
    optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3)
    x = torch.randn(4096, 16)
    data = torch.utils.data.TensorDataset(x, x.sum(dim=1, keepdim=True))
    loader = torch.utils.data.DataLoader(data, batch_size=32, shuffle=True)
    for epoch in range(100):
        for x, y in loader:
            loss = torch.nn.functional.mse_loss(model(x), y)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            if ravex.step() % 200 == 0:
                print(f"step {ravex.step()}  loss {loss.item():.4f}")

train()
python train.py      # press Ctrl-C after a few lines
python train.py      # carries on past where it stopped

The decorator is the only change to a training script. Ravex finds the model, the optimizer, the LR scheduler and the dataloader as they are built, and restores all of them with the random number generators, so a resumed run continues with the same losses it would have had. max_steps is the length of the whole training across restarts. A SIGTERM, which is what a spot preemption, scancel or docker stop send, stops the run at the next step with a final checkpoint. The Ravex README has the rest.

2. Run a repository on your GPUs

The console runs a commit of a GitHub repository on a node it rents in your own provider account, and returns the node when the run ends or the budget is spent. The provider bills you directly.

What the repository needs

The quickest start is to fork gpuzero-dev/quickstart, a small GPT that learns Shakespeare and has all three. Or, with your own code, a minimal repository with a parameter the Launch page can change:

import argparse
import torch
import ravex

parser = argparse.ArgumentParser()
parser.add_argument("--lr", type=float, default=1e-3)
parser.add_argument("--width", type=int, default=64)
parser.add_argument("--steps", type=int, default=2000)
args = parser.parse_args()

@ravex.train_loop(max_steps=args.steps)
def train():
    torch.manual_seed(0)
    model = torch.nn.Sequential(torch.nn.Linear(16, args.width), torch.nn.ReLU(), torch.nn.Linear(args.width, 1))
    optimizer = torch.optim.AdamW(model.parameters(), lr=args.lr)
    x = torch.randn(4096, 16)
    data = torch.utils.data.TensorDataset(x, x.sum(dim=1, keepdim=True))
    loader = torch.utils.data.DataLoader(data, batch_size=32, shuffle=True)
    for epoch in range(1000):
        for x, y in loader:
            loss = torch.nn.functional.mse_loss(model(x), y)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            ravex.log_metrics({"train/loss": loss.item()})

train()
version = 1

[scripts.train]
path = "train.py"
model = ["width"]
params = { lr = 0.001, width = 64, steps = 2000 }

Leave the checkpoint location out of the decorator: an argument there wins over the environment, and the node names the run's store through it.

In the console

3. gpuzero.toml reference

One file at the root of the repository. The node reads it at the commit being run, so each commit says what it may run. The shortest one names a script by its path:

[scripts]
train = "train.py"

version

1, which is also what a file without it is. A file with another version is refused rather than read wrongly.

[scripts]

Each key is a script's name, the one Launch shows. Its value is the path of a Python file in the repository, relative to the root, or a table with:

Where it runs

Not in the repository. A run on a cloud starts from your workspace's Infrastructure sheet, the Workflow page you reach from Providers, once it is locked. Its clusters say which provider, region, cards and how many nodes; Launch asks which cluster when there is more than one. The most a node may cost an hour, the amount at which a run is stopped and the monthly budget are set on Providers. The first two apply to a run on one node; a run on several nodes is held by the monthly budget only, and is never spot. A [compute] table left in gpuzero.toml is no longer read: the run goes ahead and its first log line says so.

4. From your own machine

A run on a machine you already have, a workstation or a university cluster, can send its metrics, checkpoints list and resumes to the console too. Make a token in the console under Settings → Machine tokens (it asks you to have signed in within the last ten minutes, and shows the token once), then give Ravex the address and the token:

export RAVEX_METRICS_ENDPOINT=https://api.gpuzero.dev
export RAVEX_METRICS_TOKEN=gzat_...
python train.py

The run appears on the console's Metrics page like one launched from there. A token acts for the workspace until you revoke it on the same page, even after you sign out.

5. Coming from W&B or Neptune

It will feel familiar: live charts, logs, runs side by side, one line to log a metric. What changes is that a resume or a fork brings the training back, not just the chart, and every run downloads as one file. The full guide, with importing old runs: from W&B or Neptune to GPU Zero.

6. Limits today