docs
Get started
GPU Zero has two halves. Ravex and Moonclip are open-source libraries that checkpoint a PyTorch training run and resume it where it stopped; they need no account. The console runs your repository on GPUs in your own cloud account, shows each run's metrics, and lets you resume or fork a run from a real checkpoint.
- 1. Checkpoint and resume with Ravex
- 2. Run a repository on your GPUs
- 3. gpuzero.toml reference
- 4. From your own machine
- 5. Coming from W&B or Neptune
- 6. Limits today
1. Checkpoint and resume with Ravex
Ravex needs Python 3.11 or later and the PyTorch you already have.
pip install "ravex[moonclip]"This runs as it is, on a CPU, in a few seconds. Save it as train.py:
import torch
import ravex
@ravex.train_loop(max_steps=2000)
def train():
torch.manual_seed(0)
model = torch.nn.Sequential(torch.nn.Linear(16, 64), torch.nn.ReLU(), torch.nn.Linear(64, 1))
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3)
x = torch.randn(4096, 16)
data = torch.utils.data.TensorDataset(x, x.sum(dim=1, keepdim=True))
loader = torch.utils.data.DataLoader(data, batch_size=32, shuffle=True)
for epoch in range(100):
for x, y in loader:
loss = torch.nn.functional.mse_loss(model(x), y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
if ravex.step() % 200 == 0:
print(f"step {ravex.step()} loss {loss.item():.4f}")
train()python train.py # press Ctrl-C after a few lines
python train.py # carries on past where it stoppedThe decorator is the only change to a training script. Ravex finds the model, the optimizer, the LR scheduler and the dataloader as they are built, and restores all of them with the random number generators, so a resumed run continues with the same losses it would have had. max_steps is the length of the whole training across restarts. A SIGTERM, which is what a spot preemption, scancel or docker stop send, stops the run at the next step with a final checkpoint. The Ravex README has the rest.
2. Run a repository on your GPUs
The console runs a commit of a GitHub repository on a node it rents in your own provider account, and returns the node when the run ends or the budget is spent. The provider bills you directly.
What the repository needs
- A training script that uses
@ravex.train_loop. That is how its metrics, checkpoints and resumes reach the console.ravex.log_metricsadds your own values to the run's charts. - A
gpuzero.tomlat the root, naming the scripts that may be run (reference below). - Its dependencies in
requirements.txtorpyproject.tomlat the root, installed on the node before the run. PyTorch is already on the node; list it only to require a version.
The quickest start is to fork gpuzero-dev/quickstart, a small GPT that learns Shakespeare and has all three. Or, with your own code, a minimal repository with a parameter the Launch page can change:
import argparse
import torch
import ravex
parser = argparse.ArgumentParser()
parser.add_argument("--lr", type=float, default=1e-3)
parser.add_argument("--width", type=int, default=64)
parser.add_argument("--steps", type=int, default=2000)
args = parser.parse_args()
@ravex.train_loop(max_steps=args.steps)
def train():
torch.manual_seed(0)
model = torch.nn.Sequential(torch.nn.Linear(16, args.width), torch.nn.ReLU(), torch.nn.Linear(args.width, 1))
optimizer = torch.optim.AdamW(model.parameters(), lr=args.lr)
x = torch.randn(4096, 16)
data = torch.utils.data.TensorDataset(x, x.sum(dim=1, keepdim=True))
loader = torch.utils.data.DataLoader(data, batch_size=32, shuffle=True)
for epoch in range(1000):
for x, y in loader:
loss = torch.nn.functional.mse_loss(model(x), y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
ravex.log_metrics({"train/loss": loss.item()})
train()version = 1
[scripts.train]
path = "train.py"
model = ["width"]
params = { lr = 0.001, width = 64, steps = 2000 }Leave the checkpoint location out of the decorator: an argument there wins over the environment, and the node names the run's store through it.
In the console
- Projects: add the repository and its branch. A private one is read through the GPU Zero GitHub App, installed on the repositories you choose. A new workspace already has the quickstart, read-only, to try a launch before adding yours.
- Providers: connect a GPU provider with an API key from that account; the page lists the ones you can connect.
- Providers again: set a monthly budget, and if you like the most a node may cost an hour and the amount at which a run is stopped. Without a budget no node is made.
- Workflow, reached from Providers: draw the clusters a run may use, with their provider, region and cards, and lock the sheet. A cloud run starts only from a locked sheet.
- Secrets, if the script needs them: an API token, a dataset key. They reach the run as environment variables and are never shown again.
- Launch: pick a commit and a script, change its parameters, queue it. The run's page shows its metrics, logs and checkpoints as it goes.
3. gpuzero.toml reference
One file at the root of the repository. The node reads it at the commit being run, so each commit says what it may run. The shortest one names a script by its path:
[scripts] train = "train.py"
version
1, which is also what a file without it is. A file with another version is refused rather than read wrongly.
[scripts]
Each key is a script's name, the one Launch shows. Its value is the path of a Python file in the repository, relative to the root, or a table with:
path: the file, as above. It has to be inside the repository.params: the parameters a run starts from, with their values. Launch fills its form with them, and each reaches the script as a flag:lr = 0.001becomes--lr 0.001,truebecomes the bare flag,falseleaves it out. Names are letters, digits,_and-, up to 64 characters.model: the parameters that change the model itself, such as its width. A run cannot continue from a checkpoint with one of these changed, so the console offers a fresh run instead of a resume or a fork when you change one.
Where it runs
Not in the repository. A run on a cloud starts from your workspace's Infrastructure sheet, the Workflow page you reach from Providers, once it is locked. Its clusters say which provider, region, cards and how many nodes; Launch asks which cluster when there is more than one. The most a node may cost an hour, the amount at which a run is stopped and the monthly budget are set on Providers. The first two apply to a run on one node; a run on several nodes is held by the monthly budget only, and is never spot. A [compute] table left in gpuzero.toml is no longer read: the run goes ahead and its first log line says so.
4. From your own machine
A run on a machine you already have, a workstation or a university cluster, can send its metrics, checkpoints list and resumes to the console too. Make a token in the console under Settings → Machine tokens (it asks you to have signed in within the last ten minutes, and shows the token once), then give Ravex the address and the token:
export RAVEX_METRICS_ENDPOINT=https://api.gpuzero.dev
export RAVEX_METRICS_TOKEN=gzat_...
python train.pyThe run appears on the console's Metrics page like one launched from there. A token acts for the workspace until you revoke it on the same page, even after you sign out.
5. Coming from W&B or Neptune
It will feel familiar: live charts, logs, runs side by side, one line to log a metric. What changes is that a resume or a fork brings the training back, not just the chart, and every run downloads as one file. The full guide, with importing old runs: from W&B or Neptune to GPU Zero.
6. Limits today
- Providers: the ones on the console's Providers page. More are being added, the big clouds among them.
- PyTorch only. Ravex follows PyTorch's objects, and does not work with JAX.
- A cap on nodes at once and on dollars an hour of nodes per workspace, shown on the Compute page.
- Without a bucket of your own, checkpoints are kept in a bucket GPU Zero operates, up to 50 GB per workspace. Connect your own S3 or R2 bucket on the Compute page to keep them in your account.
- A run and its metrics are kept for six months, then deleted; the run's page exports it as a file before that.