Onde Inference CLI: Fine-tune & Deploy Local LLMs

Onde Inference CLI

Command-line interface for Onde Inference .

Swift · Flutter · React Native · Rust · Website

---

Manage your Onde Inference account, fine-tune local models, and export them to GGUF, all from the terminal.

Install

Install onde-cli with your favorite tool. For package docs and the full install matrix, see https://ondeinference.com/cli.

npm

npm install -g @ondeinference/cli

Homebrew

brew tap ondeinference/homebrew-tap && brew trust --tap ondeinference/homebrew-tap

brew install onde

pip / uv / uvx

pip install onde-cli

or

uv tool install onde-cli uv run onde --tui

or with

uvx --from onde-cli onde --tui

.NET tool

dotnet tool install --global Onde.Cli

Dart pub global

dart pub global activate onde_cli

The Dart package is a thin launcher. On first run it downloads the right native binary into ~/.onde/cli, then reuses the local copy.

Pre-built binary

Download a release from GitHub Releases:

# macOS Apple Silicon

curl -Lo onde https://github.com/ondeinference/onde-cli/releases/latest/download/onde-macos-arm64 chmod +x onde && mv onde /usr/local/bin/onde

---

Usage

onde --tui

This opens the TUI. Plain onde prints usage and exits. You can sign up or sign in right there.

MCP server

Run onde as a Model Context Protocol server over stdio instead of the TUI:

onde --mcp

This exposes Onde account and model-catalog operations as MCP tools — login, me, apps_list, app_create, app_rename, models_list, model_register, model_assign, hf_search — returning structured JSON. stdout is the JSON-RPC channel; tools run non-interactively and reuse the token from a TUI sign-in (or the login tool). Point any MCP client at the command onde --mcp.

---

Fine-tuning

onde includes a LoRA fine-tuning pipeline for Qwen2, Qwen2.5, and Qwen3 models. It runs locally: Metal on Apple Silicon, CPU elsewhere. No cloud setup. No Python environment.

The flow is straightforward: download a safetensors base model, fine-tune it with LoRA, merge the adapter back into the base weights, then export to GGUF for use in the Onde SDK.

If you want a quick refresher on what the model is actually doing at inference time, Onde has a short note on the forward pass.

Training data format

Each line should be one complete conversation in Qwen's chat template:

{"text": "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWhat is LoRA?<|im_end|>\n<|im_start|>assistant\nLoRA adds small trainable matrices to frozen layers, letting you fine-tune large models without updating all the weights.<|im_end|>"}

Save the file wherever you want. The TUI lets you point to it directly.

Running it

onde --tui

→ Models tab (Tab from Apps) → Select a safetensors model (↑↓, Enter) → Press f

Only safetensors models can be fine-tuned. GGUF models are already quantized, so their weights are not differentiable.

Configure the run:

Press Enter to start. In a healthy run, loss usually starts dropping by epoch 2. If it stays flat, try 0.0003.

After training

For rank 8 on a 0.6B model, the adapter is about 1.5 MB. From the fine-tune complete screen:

• m to merge the adapter into the base model
• g to export the merged model to GGUF

The resulting GGUF loads directly in the Onde SDK for on-device AI inference.

Supported base models

You can search for any of these from the Models tab with /.

---

Debug

Logs are written to ~/.cache/onde/debug.log.

If you installed through pub.dev, the launcher cache lives under ~/.onde/cli.

---

License

Dual-licensed under MIT and Apache 2.0.

Copyright

© 2026 Splitfire AB (Onde Inference).