Lium GPU Cloud

Lium — Python SDK & CLI

lium.io is a Python package that provides both a command-line interface and a Python SDK for managing GPU pods on the Lium platform. Install it once — use whichever interface fits the job.

Lium

Quickstart • Website • CLI Docs • SDK Docs • Discord

Installation

Python package

pip install lium.io

Binary install (macOS amd64/arm64 / Linux amd64/arm64)

curl -fsSL https://lium.io/install.sh | bash

Fresh binary installs place a managed symlink at ~/.lium/bin/lium that points to a versioned binary under ~/.lium/versions/ /lium.

Quick Start

CLI

# First-time setup: create an account (mints and stores an API key) …

lium signup --email you@example.com

… or link an existing account (opens a browser). Headless (agents, CI, containers): pass the key

instead — lium init --api-key sk_... (keys: https://lium.io/api-keys), or export LIUM_API_KEY and skip init.

lium init lium balance

List available nodes (GPU machines)

lium ls

Price a job on node #1 before renting it (no API key needed)

lium quote 1 --hours 2

Create a pod using node index

lium up 1 # Use node #1 from previous ls

Or create a pod using filters

lium up --gpu A100 # Auto-select best A100 node

List your pods

lium ps lium ps --filter status=RUNNING --sort spent # running pods, most expensive so far first lium spend # burn per hour, spend per pod, runway

Copy files to pod

lium scp 1 ./my_script.py

SSH into a pod

lium ssh

Stop a pod — billing is per second and runs until you do this

lium rm

First hour on a pod

A few things that save time on a freshly rented pod (full version in docs/getting-started.rst):

• Always pass --ttl (or --until) to lium up; a pod bills until it is removed.
• The pod's local volume (/root on the standard templates; pod.volume_path in the SDK) is the only path lium bk can back up and the one encryption covers; an attached Volume is under /mnt; everything else (/workspace, /tmp) is plain container filesystem, neither encrypted nor backup-able. Keep weights, datasets and the Hugging Face cache on the volume: mkdir -p /root/hf /root/logs; export HF_HOME=/root/hf HF_HUB_ENABLE_HF_TRANSFER=1.
• Ubuntu 24.04 images: use a venv (python -m venv /root/venv) or export PIP_BREAK_SYSTEM_PACKAGES=1 before pip install.
• Blackwell GPUs (B200, B300, RTX PRO 6000, RTX 5090) need a cu128+ PyTorch build: pip install torch --index-url https://download.pytorch.org/whl/cu130. FlashAttention-3 is Hopper-only; use FlashAttention-4 or cuDNN attention on Blackwell.
• Missing tools: apt-get update && apt-get install -y ffmpeg rsync.
• Background jobs: nohup setsid cmd > /root/logs/x.log 2>&1 < /dev/null &.
• Check utilisation: nvidia-smi --query-gpu=timestamp,index,utilization.gpu,memory.used --format=csv -l 5 > /root/logs/gpu.csv &.

SDK

The SDK mirrors the CLI's capabilities for programmatic use. Two entry points: the @lium.machine decorator for quickly offloading isolated functions, and the Lium() client for long-lived orchestration code.

High-level decorator — annotate a function and offload work to a GPU pod. machine is " x " or " " ("1xH200", "A100", "2xRTX4090"; count defaults to 1; the GPU is named as lium ls --gpu takes it and matched whole, so "A100" never rents an RTX A1000) and the cheapest matching node is rented; timeout= (default 1 h) bounds the run and the pod's lifetime. Arguments travel as a pickle (your own bytes, loaded on your own pod); the result comes back as a JSON envelope plus an .npz sidecar for numpy arrays read with allow_pickle=False, so nothing the pod writes is unpickled on your machine. What round-trips: None/bool/int/float/str/bytes, list/tuple/set/frozenset/dict of those, datetime/date/time/timedelta, Decimal, pathlib.Path, uuid.UUID, numpy.ndarray (any dtype without Python objects) and numpy scalars — anything else is a lium.ResultEncodingError on the pod naming the type (return .tolist(), dict(x), x.value instead). Only the function's own def is sent, so import inside it and pass everything else as arguments. A remote exception is re-raised with its type when that type is a builtin (except ValueError works; other types arrive as lium.RemoteExecutionError with the name), with lium.RemoteExecutionError (remote traceback, exit code, output) as its cause:

import lium

@lium.machine(machine="A100", requirements=["torch", "transformers", "accelerate"]) def infer(prompt: str) -> str: from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("sshleifer/tiny-gpt2") model = AutoModelForCausalLM.from_pretrained("sshleifer/tiny-gpt2", device_map="cuda") tokens = tokenizer(prompt, return_tensors="pt").to("cuda") out = model.generate(**tokens, max_new_tokens=50) return tokenizer.decode(out[0], skip_special_tokens=True)

print(infer("Who discovered penicillin?"))

keep_warm=300 keeps the pod five minutes for the next call or the next run of the script; infer.map(prompts) runs every item on one pod; infer.local(...) runs the function here (local=True / LIUM_MACHINE_LOCAL=1 does so for every call); infer.close() removes a warm pod. Progress goes to stderr (quiet=True to silence):

[lium] infer: renting 1xA100 $1.20/h (swift-fox-c8, US), removal in 1.2h

[lium] infer: pod ready in 48s [lium] infer: preparing environment (3 package(s): torch, transformers, accelerate) [lium] infer: environment ready in 21s [lium] infer: running [lium] infer: done in 96s (~$0.0320) [lium] infer: pod removed

Direct SDK usage follows the same pattern:

from lium.sdk import Lium

lium = Lium()

the cheapest available 1×A100 with at least 32 CPUs, chosen and rented in one call

rented = lium.rent(gpu_type="A100", min_cpus=32, name="demo") print(f"{rented.executor.huid} at ${rented.price_per_hour:.2f}/h") ready = lium.wait_ready(rented.pod, timeout=600) # None only if still starting after 600 s print(lium.exec(ready, command="nvidia-smi", timeout=60)["stdout"]) lium.down(ready)

wait_ready() raises PodStartError — with .pod, .status, .history and .cause (what the backend recorded, e.g. Container creation failed due to ... (failure_step: ssh_connect)) — when the pod reaches FAILED/CREATION_FAILED/STOPPED/BROKEN or disappears from the pod list, so a dead pod is not mistaken for a slow one. Pass on_poll=lambda pod, status, elapsed: ... to be told about every poll. lium up is bounded by --timeout SECONDS (default 900) for the whole rent, prints waiting for … ( s) while it waits, and exits 1 naming the pod when the budget runs out; --ready-timeout caps only the wait.

A long job goes on a pod the caller keeps: detach=True starts it in the background and returns at once, and the pod stays up until you remove it. lium.ls() lists the nodes when you want to name one; up(wait=True) rents it and returns the ready pod.

node = lium.ls(gpu_type="A100")[0]

pod = lium.up(executor_id=node.id, name="train", wait=True) job = lium.exec(pod, command="python train.py", detach=True) # {"pid", "log_path", "command"} for gpu in lium.gpu_stats(pod): # parsed nvidia-smi print(gpu.index, gpu.utilization_pct, gpu.memory_pct) print(pod.to_dict()) # JSON-ready

later: lium.down(pod)

For work that must not outlive the code using it, rental() rents a named node for a with block and removes the pod on the way out, whatever happened inside — so run the work to completion inside the block (a detached job started here would be killed with the pod). rent() above is the other way in: it picks the node by spec and hands you a pod you own.

with lium.rental(executor_id=node.id, name="eval") as pod:

result = lium.exec(pod, command="python eval.py", timeout=1800) print(result["stdout"])

lium.pod_by_name("job") finds a pod by name, huid or id.

A server or a training run should outlive the call that starts it. run_background() starts it detached with a PID file, an exit-code file and a log on the pod (plus the process's boot id and start time in a .id file, so status() and kill() never take a PID reused after a pod restart for the job; a job without that file reads gone and is not signalled), and the returned Job knows how to wait for it:

job = lium.run_background(pod, "vllm serve Qwen/Qwen3-8B --port 8000", name="vllm")

job.wait_for_port(8000, timeout=900) # raises at once, with the log tail, if vllm dies first print(job.logs(tail=20))

later, from another process or agent turn:

job = lium.job(pod, "vllm") # re-attach by name; job.status(), job.kill()

lium.wait_for_port(pod, 8000) probes a port without a job (a template that serves on start), and lium.wait_ready(pod, ready_port=8000) waits for RUNNING and the port together. Multi-node clusters — N whole nodes on one InfiniBand/RoCE fabric, rented as one order, each with a private overlay address:

offer = lium.clusters()[0]                                   # fabrics with free nodes

cluster = lium.up_cluster([n.id for n in offer.cheapest(2)], name="train", wait=True) print(cluster.master_addr) # 10.42.0.1 — MASTER_ADDR for torchrun for pod in cluster.pods: lium.exec(pod, command=f"torchrun {cluster.torchrun_args(pod)} --nproc_per_node 8 train.py") open("hostfile", "w").write(cluster.hostfile()) # mpirun / DeepSpeed lium.rm_cluster(cluster) # one DELETE /clusters/{id}; one result per member

up_cluster() raises ClusterNotListedError when the nodes are, or may be, rented but the pod list did not show a whole cluster (.confirmed says whether the API confirmed the order, .pod_ids and .listed what it named and what was listed): do not rent again, list the pods to find the cluster. wait_cluster_ready() raises PodStartError at once when a member fails or is missing from the pod list. rm_cluster() sends one DELETE /clusters/{cluster_id}: the API removes every member and answers one row per member (pod, huid, name, node_rank, success, message, error); a member that failed is reported and the rest are still removed, so call again to retry it. A 404 (the cluster is gone, or the API has no such route) raises LiumNotFoundError and nothing is removed; there is no per-pod fallback.

Full API reference: https://docs.lium.io/developers/sdk/reference

lium.ssh(pod) returns the pod's ssh command with -i and the pinned host-key options described under Configuration; pass refresh=True to rebuild it from the pod's current host and port after a restart (lium.refresh_pod(pod) re-reads one pod by id or huid; LiumNotFoundError when it is gone). lium ps --format json and lium describe (table and --json) show the same command without -i as ssh_command (the key path lives in the SDK config, not in the pod record); the JSON keeps the API's raw v