Prototype · Preview: every server in the official registry, synced every 30 minutes. Sign in with Google to write reviews.Prototype · updated every 30 min
PopNx MCPMCP servers for people who use AI · explained in plain words Sign in

the-art-of-debugging Skill

Repository ↗

by stas00★ 1,7491 skills

Install

What it does

The Art of Debugging Open Book

  • Connects Instruction files the AI reads
  • Account or key No key found in README or SKILL.md
  • Category Coding
For makers: add a badge to your README

If you made this, paste this code into your README. The badge links back to this page.

[![Featured on PopNx MCP](https://mcp.popnx.com/badge/skill/stas00-the-art-of-debugging.svg)](https://mcp.popnx.com/en/skills/stas00-the-art-of-debugging)

Use together with

MCP servers that connect the AI to services for the same work

REA

Reverse engineer anything from your terminal or agent with one CLI and MCP server.

by morlutoWidely used↓ 388.7K/wk★ 468.8KUpdated 8 days ago

MCP · Connects a serviceRuns on your computer

Fallow

Codebase intelligence for TypeScript and JavaScript, exposed as structured MCP analysis tools.

by fallow-rsWidely used↓ 2.4M/wk★ 5.4KUpdated yesterday

MCP · Connects a serviceRuns on your computer

Desktop Commander

MCP server for terminal commands, file operations, and process management

by wonderwhy-erWidely used↓ 740.3K/wk★ 10.4KUpdated 2 days ago

MCP · Connects a serviceRuns on your computer

Similar Skills

Same kind of work, ordered like the Skills list

Compared with similar ones
This one
the-art-of-debugging
reaandrej-karpathy-skillsgraphifyMoneyPrinterTurbo
Makerstas00morlutomultica-aiGraphify-Labsharry0703
Skills11111
GitHub stars1,749468,733234,332150,287144,566
Last code changeOct 14, 2026Nov 19, 2026Apr 20, 2026Nov 18, 2026Nov 18, 2026
LicenseCC-BY-SA-4.0MIT–Apache-2.0MIT

1 skills (instruction sets) by stas00 that an AI reads and follows · not an MCP server

How a Skill differs from MCP

MCP = the hands

Connects the AI to an outside service or data, such as reading Notion or creating an invoice. A program or server has to be running.

Skill = the manual

A text file that teaches the AI how to do a task. It connects to nothing.

Some overlap: a few Skills run scripts that call tools, and a few MCP servers only send knowledge. Start from the task you want done.

Works with

Not stated by the maker · SKILL.md skills work in apps that support Agent Skills, such as Claude Code, Codex and Cursor

Install

Is it safe?

  • Who made it

    GitHub account stas00 (person), 11 years old

  • Use and upkeep

    ★ 1,749 stars · code last changed Oct 14, 2026

  • Any scripts?

    35 scripts that can run on a computer: build/check-new-links.sh, build/check-programs, pytorch/code/collective_mismatch.py, pytorch/code/collective_mismatch_analyze.py and 31 more

  • Uses the internet?

    Scripts that go online: build/check-new-links.sh

  • Changes machine-wide settings?

    No hooks or writes to machine settings found

Checked automatically from the repo files at commit 7e479bf (read only, nothing was run). Not a safety guarantee: read it and decide for yourself

First: which AI do you use?

Pick your app and this page shows only how to install it there

Option 1Upload to claude.ai (web and app)

  1. Download from GitHub

    This skill uses the CC-BY-SA-4.0 license, which is not on the list we redistribute, so we do not hand out a ZIP from our site. Download it from GitHub and arrange the folder yourself, or use "Let the AI install it" above, which is easier

    Download from GitHub (whole repo) ↗

    Unzip it, then zip only the the-art-of-debugging folder again. The ZIP must have a art-of-debugging folder with SKILL.md at the top level, no loose files

  2. Open Skills on claude.ai and upload

    +Create skillUpload a skillpick art-of-debugging.zip

    Open claude.ai/customize/skills ↗

  3. No Upload button? Turn on file creation first

    SettingsCapabilitiesCode execution and file creation

    Turn on Code execution and file creation first (newer menus: Create and edit files) · personal plans: Settings → Capabilities · Team/Enterprise: an admin turns it on under Organization settings → Plugins & skills · every plan, including Free

Upload once; it works in claude.ai and Claude Desktop, and in recent Claude Code signed in with a Claude account (not with an API key)

Cowork and cloud sessions use the skills turned on in your claude.ai account, not ~/.claude/skills on your computer (Claude Desktop scheduled tasks run locally and still load it)

Checked against official docs Nov 12, 2026 · Source: support.claude.com, support.claude.com, code.claude.com

Option 1Let Claude Code install it

Copy this sentence and paste it into a Claude Code chat

Where to install
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) for all my projects, at ~/.claude/skills/art-of-debugging. First show me every file and tell me whether it has scripts or changes machine settings, then tell me the full path you will install to and wait for me to agree.
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) into the project this chat is open in (at .claude/skills/art-of-debugging). First show me every file and tell me whether it has scripts or changes machine settings, then tell me the project name and the full path you will install to and wait for me to agree.

For one project, paste it into a chat that is open in that project’s folder · The AI always shows you the path before installing

The AI tells you what it will do first and asks your permission at every step

Restart Claude Code, then type /the-art-of-debugging or ask for a matching task and Claude picks it up

Checked against official docs Nov 12, 2026 · Source: code.claude.com

For Terminal users

A Skill is instructions the AI will follow. Read the files before installing, and install only from makers you trust.

  1. Open Terminal

    Mac: press Command+Space and type Terminal · Windows: the command below does not work in PowerShell; use “Let AI install it” above instead

  2. Copy the skill folder to ~/.claude/skills

    This downloads the repo to a temporary folder and copies only the folders that contain a SKILL.md

    git clone --depth 1 https://github.com/stas00/the-art-of-debugging.git /tmp/popnx-skill-stas00-the-art-of-debugging-7e479bf && mkdir -p ~/.claude/skills && cp -R '/tmp/popnx-skill-stas00-the-art-of-debugging-7e479bf/.' ~/.claude/skills/the-art-of-debugging
  3. Restart Claude Code, then type /the-art-of-debugging or ask for a matching task and Claude picks it up

Skills you install yourself work in the ChatGPT desktop app, Codex CLI and the Codex IDE extension · ChatGPT on the web and mobile cannot use skills you install yourself, only skills that come with a plugin from the directory

ChatGPT desktop app

The ChatGPT desktop app has Codex built in, and Codex reads skills from ~/.agents/skills

Option 1Let ChatGPT install it

Open Codex in the ChatGPT desktop app and paste this sentence into the chat

Where to install
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) for all my projects, at ~/.agents/skills/art-of-debugging. First show me every file and tell me whether it has scripts or changes machine settings, then tell me the full path you will install to and wait for me to agree.
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) into the project this chat is open in (at .agents/skills/art-of-debugging). First show me every file and tell me whether it has scripts or changes machine settings, then tell me the project name and the full path you will install to and wait for me to agree.

The AI always shows you the path before installing

The AI tells you what it will do first and asks your permission at every step

Option 2Put the folder there yourself

Put the art-of-debugging folder at ~/.agents/skills/art-of-debugging

~ is your home folder ($HOME)

For Terminal users

A Skill is instructions the AI will follow. Read the files before installing, and install only from makers you trust.

  1. Open Terminal

    Mac: press Command+Space and type Terminal · Windows: the command below does not work in PowerShell; use “Let AI install it” above instead

  2. Copy the skill folder to ~/.agents/skills

    This downloads the repo to a temporary folder and copies only the folders that contain a SKILL.md

    git clone --depth 1 https://github.com/stas00/the-art-of-debugging.git /tmp/popnx-skill-stas00-the-art-of-debugging-7e479bf && mkdir -p ~/.agents/skills && cp -R '/tmp/popnx-skill-stas00-the-art-of-debugging-7e479bf/.' ~/.agents/skills/the-art-of-debugging

After installing

  • ChatGPT desktop app: open Skills in the sidebar to see that the skill is there
  • To use it: in ChatGPT type @ and pick the skill · in Codex CLI or the IDE extension run /skills or type $ and the skill name
  • Codex picks up newly installed skills by itself; if one does not show up, restart Codex

Checked against official docs Nov 15, 2026 · Source: learn.chatgpt.com

ChatGPT on the web and mobile

Skills you install yourself do not work here; only skills that come with a plugin from the directory

To use this skill with ChatGPT, use the ChatGPT desktop app as above

Checked against official docs Nov 15, 2026 · Source: learn.chatgpt.com

Option 1Let Codex install it

Copy this sentence and paste it into a Codex chat · Works in Codex inside the ChatGPT desktop app, Codex CLI and the IDE extension

Where to install
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) for all my projects, at ~/.agents/skills/art-of-debugging. First show me every file and tell me whether it has scripts or changes machine settings, then tell me the full path you will install to and wait for me to agree.
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) into the project this chat is open in (at .agents/skills/art-of-debugging). First show me every file and tell me whether it has scripts or changes machine settings, then tell me the project name and the full path you will install to and wait for me to agree.

For one project, paste it into a chat that is open in that project’s folder · The AI always shows you the path before installing

The AI tells you what it will do first and asks your permission at every step

Codex picks up newly installed skills by itself; if one does not show up, restart Codex · To use it: in ChatGPT type @ and pick the skill · in Codex CLI or the IDE extension run /skills or type $ and the skill name

Checked against official docs Nov 15, 2026 · Source: learn.chatgpt.com

For Terminal users

A Skill is instructions the AI will follow. Read the files before installing, and install only from makers you trust.

  1. Open Terminal

    Mac: press Command+Space and type Terminal · Windows: the command below does not work in PowerShell; use “Let AI install it” above instead

  2. Copy the skill folder to ~/.agents/skills

    This downloads the repo to a temporary folder and copies only the folders that contain a SKILL.md

    git clone --depth 1 https://github.com/stas00/the-art-of-debugging.git /tmp/popnx-skill-stas00-the-art-of-debugging-7e479bf && mkdir -p ~/.agents/skills && cp -R '/tmp/popnx-skill-stas00-the-art-of-debugging-7e479bf/.' ~/.agents/skills/the-art-of-debugging
  3. Restart Codex, then ask for a matching task; Codex lists the skill

Option 1Let Cursor install it

Copy this sentence and paste it into a Cursor chat

Where to install
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) for all my projects, at ~/.cursor/skills/art-of-debugging. First show me every file and tell me whether it has scripts or changes machine settings, then tell me the full path you will install to and wait for me to agree.
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) into the project this chat is open in (at .cursor/skills/art-of-debugging). First show me every file and tell me whether it has scripts or changes machine settings, then tell me the project name and the full path you will install to and wait for me to agree.

For one project, paste it into a chat that is open in that project’s folder · The AI always shows you the path before installing

The AI tells you what it will do first and asks your permission at every step

In Cursor’s Agent chat, type / and pick the-art-of-debugging

Checked against official docs Nov 12, 2026 · Source: cursor.com

For Terminal users

A Skill is instructions the AI will follow. Read the files before installing, and install only from makers you trust.

  1. Open Terminal

    Mac: press Command+Space and type Terminal · Windows: the command below does not work in PowerShell; use “Let AI install it” above instead

  2. Copy the skill folder to ~/.cursor/skills

    This downloads the repo to a temporary folder and copies only the folders that contain a SKILL.md

    git clone --depth 1 https://github.com/stas00/the-art-of-debugging.git /tmp/popnx-skill-stas00-the-art-of-debugging-7e479bf && mkdir -p ~/.cursor/skills && cp -R '/tmp/popnx-skill-stas00-the-art-of-debugging-7e479bf/.' ~/.cursor/skills/the-art-of-debugging
  3. In Cursor’s Agent chat, type / and pick the-art-of-debugging

Option 1Let the AI install it

For other apps that read skills. The sentence gives the paths for Claude Code, Codex and Cursor so the AI picks the one for your app · Copy this sentence into the app’s chat

Where to install
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) for all my projects, at ~/.claude/skills/art-of-debugging (in Codex use ~/.agents/skills/art-of-debugging, in Cursor ~/.cursor/skills/art-of-debugging). First show me every file and tell me whether it has scripts or changes machine settings, then tell me the full path you will install to and wait for me to agree.
Please install the skill art-of-debugging from https://github.com/stas00/the-art-of-debugging (commit 7e479bf) into the project this chat is open in (at .claude/skills/art-of-debugging; in Codex use .agents/skills/art-of-debugging, in Cursor .cursor/skills/art-of-debugging). First show me every file and tell me whether it has scripts or changes machine settings, then tell me the project name and the full path you will install to and wait for me to agree.

For one project, paste it into a chat that is open in that project’s folder · The AI always shows you the path before installing

The AI tells you what it will do first and asks your permission at every step

Checked against official docs Nov 12, 2026 · Source: code.claude.com, learn.chatgpt.com, cursor.com

For Terminal users

Other apps that support Agent Skills (such as Gemini CLI or GitHub Copilot) use different folders. Check that app’s guide, then copy the folder with the SKILL.md there.

View on GitHub ↗

Skill folders checked against the official Claude Code, Codex and Cursor guides on Nov 11, 2026

✓ Commands checked Nov 18, 2026

View the original SKILL.md

The text below comes from the maker’s repo, not from PopNx MCP

Showing the first 8,000 characters · Open the full file on GitHub

---
name: art-of-debugging
description: >-
  Systematic methodology and concrete tool recipes for debugging Unix, Python,
  and PyTorch programs - crashes, hangs, segfaults, wrong output, CUDA OOM, NaN/Inf,
  slowness, and multi-node/multi-GPU issues. Use when a program crashes, hangs,
  deadlocks, segfaults, runs out of memory (OOM), produces NaN/Inf or wrong
  numbers, runs too slowly, or when the user mentions gdb, strace, py-spy,
  core files, CUDA_LAUNCH_BLOCKING, ldd/nm/LD_PRELOAD, cProfile, or distributed
  training hangs. Distilled from "The Art of Debugging", the latest version of which can be found at https://github.com/stas00/the-art-of-debugging

  The latest SKILL.md version can be found at https://github.com/stas00/the-art-of-debugging/blob/master/SKILL.md
---

# The Art of Debugging

> Distilled from **The Art of Debugging Open Book** by Stas Bekman - source: https://github.com/stas00/the-art-of-debugging (CC BY-SA 4.0). This skill is a condensed index; each section links back to the full chapter for depth.

Actionable methodology + copy-paste recipes for debugging Unix / Python / PyTorch programs. Apply the general loop first; then jump to the domain cheatsheet for the failure at hand. For scaling this up to large-model training/inference on real clusters (compute/storage/network, SLURM, throughput/memory, instabilities, fault tolerance, inference), pair this with [Machine Learning Engineering](https://github.com/stas00/ml-engineering/blob/master/skills/ml-engineering/SKILL.md).

## The debugging loop

The single most important idea: **most of the effort is in *locating* the cause; once you truly understand it, the fix is usually easy.** Optimize everything for reaching understanding faster.

1. **Reproduce reliably.** Get one command that triggers the bug every time. If it's flaky, pin the nondeterminism (seeds, ordering, timing, network, uninitialized memory) first - you can't debug what you can't repeat.
2. **Shrink the payload.** Make the repro *fast*: fewer layers, tiny model/data, one process, one CPU/GPU, one node. A 2-second repro beats a 2-minute one - you'll run it hundreds of times. See [methodology](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md).
3. **Localize.** Confirm you're editing the code that actually runs (the `die` trick), then bisect the search space: which commit, which file/function, which line, which input, which rank.
4. **Get a usable signal.** Turn a cryptic failure into a precise one: a real traceback (sync mode), a stack dump (py-spy/gdb), a syscall trace (strace), a printed value at the boundary, or a min/max/NaN check on a tensor.
5. **Change one thing, re-run, verify.** Fix on the fast repro, confirm, then re-widen to the full payload. Revert anything that didn't help.

### Make the loop fast and reliable

- **Atomic debug cycles.** Make each iteration a single self-contained, repeatable command (setup + run in one shot) so you never hand-redo multi-step state. See [atomic debug cycles](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#atomic-debug-cycles).
- **Automate diagnostics, minimize typing.** Alias the repro and your most-used commands; one keystroke to re-run. See [alias frequently used commands](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#alias-frequently-used-commands) and [automate diagnostics](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#automate-diagnostics-minimize-or-avoid-typing).
- **One-liner programs.** `perl`/`awk`/`python -c` to slice logs, extract fields, transform data on the fly instead of writing throwaway scripts. See [the power of one-liner programs](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#the-power-of-one-liner-programs).
- **Juggle configs cleanly** when running many debug experiments so results don't get confused. See [juggling multiple sets of configs](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#juggling-multiple-sets-of-configs-for-different-debug-experiments).

### Localization techniques

- **Am I editing the right file/class?** Insert a guaranteed break where you *think* execution goes; if the program doesn't die, you're in the wrong file/class/env:
  ```python
  def suspect():
      die   # NameError -> proves this code runs; the traceback also names the caller
  ```
  `traceback.print_stack()` shows callers without stopping (useful when the same function is reached via many paths). See [am I editing the right file and the right class?](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#am-i-editing-the-right-file-and-the-right-class).
- **Bisect a regression.** `git bisect start / bad / good <rev>` walks commits automatically to the one that broke things - script the test for `git bisect run`. See [finding a breaking commit by bisecting](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#finding-a-breaking-commit-by-bisecting-revisions).
- **Small/synthetic payload first.** Use tiny or synthetic inputs; switch to real data only when the bug is data-dependent. See [real vs random vs synthetic data](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#real-data-vs-random-data-vs-synthetic-data).
- **Race conditions.** Reordering/timing bugs hide under async; forcing synchronous execution can expose (or mask) them - note which. See [avoiding race conditions](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#avoiding-race-conditions) and [async vs sync mode](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#async-vs-sync-mode).

### Reproducing resource & environment issues

- **Cap resources on purpose** to test failure paths: emulate a nearly-full disk, limited CPU RAM, or limited GPU memory. See [running out of resources](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#running-out-of-resources-disk-space-cpu-memory-gpu-memory).
- **Watch resources live.** `watch -n1 nvidia-smi` / `free -h` / `df -h` in a second visible terminal to correlate a hang/OOM with what the machine is doing. See [watching and reproducing resource issues](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#watching-and-reproducing-resource-issues).
- **Inject `sleep`** to freeze a program at the interesting moment so you can attach a debugger or snapshot state. See [uses for sleep](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#uses-for-sleep).
- **HPC/SLURM:** keep the allocation and re-run with `srun` instead of re-`sbatch`-ing to cut per-iteration overhead. See [SLURM salloc and srun fast debug combo](https://github.com/stas00/the-art-of-debugging/blob/master/methodology/README.md#slurm-salloc-and-srun-fast-debug-combo).

## Unix / shell

Full chapter: [Unix Tools for Debugging](https://github.com/stas00/the-art-of-debugging/blob/master/unix/README.md).

- **Make shell scripts fail loudly and traceably:**
  ```bash
  set -e          # abort on first error
  set -o pipefail # a failing command anywhere in a pipe fails the whole pipe
  set -u          # abort on undefined variables (catches typos)
  set -x          # trace: print each command with expanded values as it runs
  set +x          # turn tracing back off around a noisy region
  ```
  Combine as `set -euo pipefail`. See [controlling script execution](https://github.com/stas00/the-art-of-debugging/blob/master/unix/README.md#controlling-script-execution).
- **`strace` - trace system calls** to see what a program actually does (files, network, why it's stuck):
  ```bash
  strace python -c "print('hi')"                 # trace from the start
  strace --pid PID                               # attach to a running/stuck process
  strace -o log.txt -f torchrun ...  # -f follows fork

Discussion

Ask, share a tip or report that it does not work. Sorted by what people marked helpful.

No posts yet. Be the first to ask or share.

Posts here are written by users, not the PopNx team.

Sign in to post

From GitHub

GitHub stars
1,749
Maker
stas00
License
CC-BY-SA-4.0
Last code change
Oct 14, 2026
SKILL.md files in the repo (including translated copies and templates)
1
Topics
c, debugging, gdb, linux, methodology, python, pytorch, segfault, unix