FitLLM

FitLLM Engine

FitLLM is an open-source, zero-dependency engine that checks whether a local LLM fits on a GPU or Apple Silicon Mac using architecture-aware memory math.

Live: https://fitllm.run · Bilingual · Free · No ads · No login

Open engine: fitllm-engine (MIT · npm fitllm-engine · npx fitllm)

Zero dependencies. One readable file: engine.js. Conformance-vector tested. MIT.

Install

# self-contained binary — no Node required

curl -fsSL https://fitllm.run/install.sh | sh

Homebrew (tap) — the trust line is required for any third-party tap

brew tap click6067-ship-it/fitllm brew trust click6067-ship-it/fitllm brew install fitllm

Windows (Scoop)

scoop install https://raw.githubusercontent.com/click6067-ship-it/fitllm-engine/master/packaging/fitllm.json

already have Node? nothing to install

npx fitllm "Gemma 4 12b" --gpu "RTX 4090"

Binaries for macOS (arm64/x64), Linux (x64/arm64, glibc and musl) and Windows x64 are attached to every release with a .sha256 next to each one; install.sh verifies the checksum and installs nothing if it does not match.

Two things stated plainly rather than buried: each binary embeds its JavaScript runtime, so it is 60–94 MB — much larger than an equivalent Rust or Go tool, which is the price of not forking the engine into a second language. And this is a Homebrew tap (click6067-ship-it/homebrew-fitllm), not homebrew-core: core's bar for a self-submitted project is 225 stars / 90 forks / 90 watchers, which this project does not meet yet.

Quick start

npx fitllm "Gemma 4 12b" --gpu "RTX 4090"     # one line, exit 0 fits / 1 won't — run before you download

npx fitllm --top --gpu 4090 # what can this hardware run? npx fitllm --top --detect --json # detect every supported local GPU; agent-ready JSON npx fitllm Qwen/Qwen3-32B --detect --json # public Hugging Face ID or URL; unsupported configs fail closed npx fitllm "Gemma 4 12b" --detect --json --why # include architecture, evidence, assumptions, and exact memory inputs npx fitllm scan # what you already downloaded — does it fit this machine? npm install fitllm-engine # use the same engine as a library (see Usage)

fitllm scan — start from what you already have

Instead of typing a model name, read the ones already on disk:

$ fitllm scan

Hardware: RTX 4090 (24GB) · quant Q4_K_M · ctx 8,192

ollama (3) ✓ FITS qwen3:0.6b → Qwen3-0.6B 1.2 GB used, 22.8 GB free (on disk: 0.5 GB · Q4_K_M) ? some-finetune:latest — not in the catalog, no verdict (on disk: 6.7 GB · Q5_K_M)

It reads Ollama through its loopback API and LM Studio through lms ls --json. Both are read-only: no pull, no load, no settings written, and nothing is sent anywhere. A non-loopback OLLAMA_HOST is refused rather than queried.

A model the catalog does not know gets no estimate — only the size and quant the runtime itself reported. Guessing from a similar name would hand you another model's numbers, which is worse than no number.

Verification status, stated plainly: the Ollama path is exercised end-to-end in CI against a real Ollama install (verify-install-paths.yml). The LM Studio path is covered by fixtures only — there is no LM Studio on the machine this was written on, so it has not been run against the real CLI.

Remote MCP server

Connect any Streamable HTTP MCP client to https://fitllm.run/api/mcp:

{

"mcpServers": { "fitllm": { "url": "https://fitllm.run/api/mcp" } } }

Tools

• check_llm_fit — check one model against a GPU, multi-GPU rig, or Mac and return the verdict, memory breakdown, and a fix when it does not fit.
• what_fits_on_hardware — rank the supported local models that fit the given GPU, multi-GPU rig, or Mac.
• list_supported — list the built-in model and hardware names accepted by the fit checker.

The server is read-only, stateless, and requires no authentication.

npx fitllm "GLM-4.7-Flash" --gpu 4090     # ✓ FITS — 21.9/24 GB, free 2.1 GB

npx fitllm "gpt-oss-120b" --mac 64 # ✗ WON'T FIT → what to change to make it fit npx fitllm "Qwen 3.6 35B" --gpu "5090 + 3090" # multi-GPU rig — VRAM pools (56GB), even mixed cards npx fitllm --top --detect --json # detect hardware + what fits, for people or agents npx fitllm "Gemma 4 12b" --detect # one verdict on this machine

--detect reads all nvidia-smi adapters, uses Apple Silicon unified memory on arm64 macOS, and can resolve an exact catalog GPU name through Windows/WSL PowerShell. It never uses Win32_VideoController.AdapterRAM, serials, PNP IDs, or full environment dumps. Intel-only, ambiguous, and unsupported adapters stop with exit 2 instead of borrowing a nearby GPU's memory.

Why a CLI? The "will it run?" question is born in the terminal — one line before ollama pull. No install, no tab-switching, and it reads your *actual* hardware with --detect instead of asking you to know your VRAM. Exit code 0/1 makes it a pre-download guard:

# in your model-pull script — stop BEFORE the 40 GB download:

npx fitllm "gpt-oss-120b" --detect || { echo "won't fit — aborting pull"; exit 1; }

Put the guard in the download path

The CLI exit contract composes directly with model runtimes. The download or launch runs only after a FITS or TIGHT verdict; invalid inputs stop with exit 2.

npx fitllm "Gemma 4 12b" --detect && ollama pull gemma4:12b

npx fitllm "Llama-3.1-8B-Instruct" --detect && llama-cli -m ./Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf

For CI, use the repository's composite Action. It needs no secret and returns the full --json --why result as steps.preflight.outputs.result:

permissions:

contents: read steps:

• name: Check model memory before download id: preflight uses: click6067-ship-it/fitllm-engine@v2.22.0 with: model: Gemma 4 12b gpu: RTX 4090 ctx: '8192'

Set exactly one of gpu or mac. The Action preserves the CLI contract: exit 0 means fits/tight, 1 means it will not fit, and 2 means the request is invalid. Inputs cross the GitHub expression boundary through environment variables and are passed to Node as a Bash argument array.

Agents can collect a typed measurement without uploading anything:

npx fitllm measure "Qwen 3.6 27B" --detect --measured 15.3 --kind system_total_peak --unit GiB --runtime "llama.cpp b6400"

The command validates the conditions and prints a candidate JSON object plus a prefilled GitHub issue URL. Submission remains a human action.

If you run Ollama, you do not have to measure anything yourself — it already reports the resident size of a loaded model:

ollama run qwen3:0.6b        # in one shell

npx fitllm measure --from-ollama # in another

This reads GET /api/ps and GET /api/version over loopback. It is read-only: it does not load, unload, pull, or change anything, and it submits nothing — you get the candidate and the issue link, and you decide.

What it verifies, and what it does not. The number is idle_resident: weights plus KV resident in VRAM with the model loaded and idle. It is not peak memory during generation, so it checks part of a prediction rather than all of it, and the candidate says so.

When it refuses. If size_vram differs from size, or is zero, no candidate is produced. Ollama's documentation does not define what those fields mean when they disagree, and partial residency is something this engine deliberately does not model — a number nobody can interpret is worse than no number. Models outside the catalog are skipped too: with no prediction, there is nothing to compare against. Public Hugging Face IDs (org/model) are fetched with a bounded config/index reader and accepted only when parseHfConfig() supports the architecture.

This is the open calculation core of FitLLM. The math is open so you can audit it.

Ask an LLM "does Qwen 3.6 fit my GPU?" and it pattern-matches to an architecture from its training cutoff — and usually says *no*. Catalog-based calculators lag new releases. The CLI, API, and MCP use a curated catalog pinned to official configs. The web calculator can additionally inspect a pasted Hugging Face ID's official `config.json` live, so supported architectures work on day-one releases — including the hybrid / sliding-window / MoE structures that naive formulas get wrong.

Covers Apple Silicon unified memory (M1–M6, Pro/Max/Ultra — up to the 512GB Mac Studio), NVIDIA GPUs (RTX 20/30/40/50, workstation RTX 6000 Ada / RTX PRO Blackwell 2000–6000, datacenter A100/H100/H200/B200), NVIDIA GB10 unified-memory systems (DGX Spark 128GB and the same-spec ASUS / Dell / HP / Lenovo / MSI / Acer / GIGABYTE boxes — a partner name resolves only as the full name with its capacity, e.g. `ASUS Ascent GX10 128GB`; linked 2× / 4× clusters via `--count`), AMD Radeon (RX 7000/9000, PRO W7900) and multi-GPU presets (2×3090, 2×4090, 4×3090) — with GGUF Q-tier weight quantization kept separate from KV-cache quantization. Hardware entries carry their source URLs per-value in engine.js; new entries require ≥2 independent sources (CONTRIBUTING).

---

Why most LLM memory calculators are wrong

Almost every "can I run this LLM?" calculator estimates the KV cache with the textbook formula:

KV ≈ 2 × num_layers × num_kv_heads × head_dim × context_length × bytes

That assumes every layer keeps a full-context KV cache with one uniform head shape. True for Llama-1/2 — wrong for most 2025–2026 models:

An 11× error flips the verdict: a naive calculator says Gemma 4 31B *won't fit* in 64 GB at long context, when it fits comfortably.

The six things they ignore

1. Sliding-window attention (Gemma 2/3/4, gpt-oss): most layers only keep the last *N* tokens, so their KV stops growing. Only the global layers scale with full context.
2. Hybrid / linear attention (Qwen 3.6 / 3.8, many 2026 models): linear-attention layers use a fixed-size recurrent state, not a growing KV cache. That state is modeled too, as it