computer-use-linux Skill
What it does
Native Linux desktop control for AI agents (MCP, Pi, CLI): semantic AT-SPI trees, screenshots, window targeting, and input across GNOME, KDE, Hyprland, niri, COSMIC, i3, Sway, labwc and other wlroots compositors, and X11.
- Connects Instruction files the AI reads; connects to nothing
- Account or key No key needed
- Category 自动化与智能体
给作者:在 README 中加徽章
如果你是作者,可把这段代码贴到 README。徽章会链接回本页。
[](https://mcp.popnx.com/en/skills/agent-sh-computer-use-linux)相似的 Skill
同类工作,排序与 Skill 列表相同
agent-sh 提供的 1 个 Skill(指令集),AI 读取后照做 · 不是 MCP server
Skill 和 MCP 有什么不同
MCP = 手
把 AI 连到外部服务或数据,例如读取 Notion 或开发票。需要有程序或服务器在运行。
Skill = 手册
教 AI 怎么做某项工作的文本文件,不连接任何服务。
有些会重叠:少数 Skill 带有调用工具的脚本,少数 MCP 只提供知识。请从你想完成的工作出发。
适用的应用
Claude CodeClaude (เว็บ/แอป)CodexCursor
依据开发者 README 和 repo 中的文件夹
安装
Is it safe?
- Who made it
GitHub account agent-sh (organization) is only 8 months old: take extra care
- Use and upkeep
★ 889 stars · code last changed 2026年11月16日
- Any scripts?
No, plain text files the AI reads and follows
- Uses the internet?
No scripts, so it does not go online by itself
- Changes machine-wide settings?
No hooks or writes to machine settings found
Checked automatically from the repo files at commit f12826d (read only, nothing was run). Not a safety guarantee: read it and decide for yourself
第一步:你用哪个 AI?
选择你用的应用,本页只显示该应用的安装方法
Option 1Upload to claude.ai (web and app)
Files are byte-for-byte the same as on GitHub at commit f12826d · View on GitHub ↗
Plain text files, no programs
Once uploaded it runs on Claude’s systems, not on your computer. No need to unzip
Look inside before downloading
All 5 files · 36 KB
SKILL.md22 KBreferences/hermes-setup.md1 KBreferences/pi-setup.md5 KBreferences/selkies-setup.md6 KBLICENSE1 KB
Download computer-use-linux.zip
正在检查命令 · 通过后才会显示复制按钮
Open Skills on claude.ai and upload
+Create skillUpload a skillpick computer-use-linux.zip
No Upload button? Turn on file creation first
SettingsCapabilitiesCode execution and file creation
Turn on Code execution and file creation first (newer menus: Create and edit files) · personal plans: Settings → Capabilities · Team/Enterprise: an admin turns it on under Organization settings → Plugins & skills · every plan, including Free
Upload once; it works in claude.ai and Claude Desktop, and in recent Claude Code signed in with a Claude account (not with an API key)
Cowork and cloud sessions use the skills turned on in your claude.ai account, not ~/.claude/skills on your computer (Claude Desktop scheduled tasks run locally and still load it)
Checked against official docs 2026年11月12日 · Source: support.claude.com, support.claude.com, code.claude.com
Option 1让 Claude Code 帮你安装
复制这句话,粘贴到 Claude Code 的对话中
For one project, paste it into a chat that is open in that project’s folder · The AI always shows you the path before installing
The AI tells you what it will do first and asks your permission at every step
重新打开 Claude Code,输入 /computer-use-linux,或直接提出相关任务,Claude 会自动选用
Checked against official docs 2026年11月12日 · Source: code.claude.com
For Terminal users
Skill 是 AI 会照做的指令。安装前先阅读文件,只安装可信开发者的 Skill。
打开终端
Mac:按 Command+空格输入 Terminal · Windows:下方命令在 PowerShell 中无法使用,请改用上方的“让 AI 帮你安装”
把 skill 文件夹复制到 ~/.claude/skills
这条命令把 repo 下载到临时文件夹,只复制含 SKILL.md 的文件夹
正在检查命令 · 通过后才会显示复制按钮
重新打开 Claude Code,输入 /computer-use-linux,或直接提出相关任务,Claude 会自动选用
ChatGPT 桌面应用
ChatGPT 桌面应用内置 Codex,Codex 从 ~/.agents/skills 文件夹读取 Skill
Option 1让 ChatGPT 帮你安装
在 ChatGPT 桌面应用中打开 Codex,把这句话粘贴到对话中
The AI always shows you the path before installing
The AI tells you what it will do first and asks your permission at every step
Option 2自己放入文件夹
把 computer-use-linux 文件夹放到 ~/.agents/skills/computer-use-linux
~ 是你的主文件夹($HOME)
For Terminal users
Skill 是 AI 会照做的指令。安装前先阅读文件,只安装可信开发者的 Skill。
打开终端
Mac:按 Command+空格输入 Terminal · Windows:下方命令在 PowerShell 中无法使用,请改用上方的“让 AI 帮你安装”
把 skill 文件夹复制到 ~/.agents/skills
这条命令把 repo 下载到临时文件夹,只复制含 SKILL.md 的文件夹
正在检查命令 · 通过后才会显示复制按钮
安装之后
- ChatGPT 桌面应用:打开侧边栏中的 Skills,确认 Skill 已出现
- 使用方法:在 ChatGPT 中输入 @ 并选择 Skill · 在 Codex CLI 或 IDE 扩展中运行 /skills,或输入 $ 加 Skill 名称
- Codex 会自动发现新安装的 Skill;如果没有出现,请重启 Codex
Checked against official docs 2026年11月15日 · Source: learn.chatgpt.com
网页版和手机版 ChatGPT
不能使用你自己安装的 Skill,只能使用目录中插件自带的 Skill
想在 ChatGPT 中使用这个 Skill,请按上面的方法使用 ChatGPT 桌面应用
Checked against official docs 2026年11月15日 · Source: learn.chatgpt.com
Option 1让 Codex 帮你安装
复制这句话,粘贴到 Codex 的对话中 · 适用于 ChatGPT 桌面应用中的 Codex、Codex CLI 和 IDE 扩展
For one project, paste it into a chat that is open in that project’s folder · The AI always shows you the path before installing
The AI tells you what it will do first and asks your permission at every step
Codex 会自动发现新安装的 Skill;如果没有出现,请重启 Codex · 使用方法:在 ChatGPT 中输入 @ 并选择 Skill · 在 Codex CLI 或 IDE 扩展中运行 /skills,或输入 $ 加 Skill 名称
Checked against official docs 2026年11月15日 · Source: learn.chatgpt.com
For Terminal users
Skill 是 AI 会照做的指令。安装前先阅读文件,只安装可信开发者的 Skill。
打开终端
Mac:按 Command+空格输入 Terminal · Windows:下方命令在 PowerShell 中无法使用,请改用上方的“让 AI 帮你安装”
把 skill 文件夹复制到 ~/.agents/skills
这条命令把 repo 下载到临时文件夹,只复制含 SKILL.md 的文件夹
正在检查命令 · 通过后才会显示复制按钮
重新打开 Codex 并提出相关任务,Codex 会列出这个 skill
Option 1让 Cursor 帮你安装
复制这句话,粘贴到 Cursor 的对话中
For one project, paste it into a chat that is open in that project’s folder · The AI always shows you the path before installing
The AI tells you what it will do first and asks your permission at every step
在 Cursor 的 Agent 聊天中输入 / 并选择 computer-use-linux
Checked against official docs 2026年11月12日 · Source: cursor.com
For Terminal users
Skill 是 AI 会照做的指令。安装前先阅读文件,只安装可信开发者的 Skill。
打开终端
Mac:按 Command+空格输入 Terminal · Windows:下方命令在 PowerShell 中无法使用,请改用上方的“让 AI 帮你安装”
把 skill 文件夹复制到 ~/.cursor/skills
这条命令把 repo 下载到临时文件夹,只复制含 SKILL.md 的文件夹
正在检查命令 · 通过后才会显示复制按钮
在 Cursor 的 Agent 聊天中输入 / 并选择 computer-use-linux
Option 1Let the AI install it
适用于其他能读取 Skill 的应用。这句话列出了 Claude Code、Codex 和 Cursor 的路径,AI 会按你的应用选择 · Copy this sentence into the app’s chat
For one project, paste it into a chat that is open in that project’s folder · The AI always shows you the path before installing
The AI tells you what it will do first and asks your permission at every step
Checked against official docs 2026年11月12日 · Source: code.claude.com, learn.chatgpt.com, cursor.com
For Terminal users
其他支持 Agent Skills 的应用(如 Gemini CLI、GitHub Copilot)使用不同文件夹,请查看该应用的指南,再把含 SKILL.md 的文件夹复制过去。
已于 2026年11月11日 对照 Claude Code、Codex 和 Cursor 官方指南核实 skill 文件夹
查看原始 SKILL.md
以下文字来自开发者的 repo,不是 PopNx MCP 的指令
显示前 8,000 个字符 · 在 GitHub 打开完整文件
---
name: computer-use-linux
description: "Linux desktop observation and control via native Pi tools or the computer-use-linux MCP server: accessibility trees, screenshots, window targeting, and input synthesis (click, type, scroll)."
author: agent-sh
license: MIT
platforms: [linux]
compatibility: "Native Pi tools require Pi 0.84.4+ and Node.js 22.19+; the standalone CLI/MCP server supports Node.js 18+."
---
# computer-use-linux
Use `computer-use-linux` when an agent needs to observe or operate a local Linux desktop: inspect the accessibility tree, list/focus windows, take screenshots, click, scroll, type, press keys, or invoke AT-SPI actions.
On labwc, window targeting uses the Selkies IPC patch's `labwc.sock` when it exists. Stock labwc and other wlroots compositors that advertise wlr-foreign-toplevel-management use that protocol instead, which reports no pids or bounds. Activation requires a unique app-id/title match. Window IDs include the title and must be refreshed after it changes. Ambiguous windows are rejected. Window bounds and relative coordinate targeting are unavailable until output scaling can be verified. Window-targeted screenshots, including `screenshot_after: "window"`, fail for the same reason; capture the full screen instead.
Native X11 clicks, scrolls and drags use xdotool/XTEST on `DISPLAY`, ahead of uinput and portal input. Install xdotool for this route. Missing or failed commands return an error without a system-wide fallback. Drags send paced intermediate motion and always attempt release after a failed press or motion; verify the drop target afterward.
## When to Use
Use this skill when:
- The user wants the agent to control a Linux GUI app.
- You need desktop state from AT-SPI, screenshots, or compositor window metadata.
- You are configuring the `computer-use-linux` MCP server for your agent.
- A desktop action needs target-aware input instead of blind shell commands.
Do not use this for remote browsers, websites, or headless automation when a browser-specific tool is available. Do not assume desktop actions are safe just because the MCP connection works.
## Install
Pick the install that matches how you will run this skill. You can use both.
### Claude Code or Codex plugin
```bash
claude plugin marketplace add agent-sh/computer-use-linux
claude plugin install computer-use-linux@computer-use-linux
# or, in Codex:
codex plugin marketplace add agent-sh/computer-use-linux
codex plugin add computer-use-linux@computer-use-linux
```
The plugin registers the MCP server and downloads the pinned, sha256-verified
release binaries on first start. It does not put `computer-use-linux` on
`PATH`; shell commands in this skill need the CLI install below. Set
`COMPUTER_USE_LINUX_BIN` to run a local build instead.
### Pi native tools
```bash
pi install npm:@agent-sh/computer-use-linux
```
This enables Pi's `computer_use_linux_*` tools. It does not put
`computer-use-linux` on `PATH`. Shell commands in this skill (`doctor`,
`setup`, `setup-window-targeting`, `guard-accessibility`, the MCP `command`
config, and Verification) need the CLI install below.
### Shell CLI / MCP server
Use this when you need `computer-use-linux` on `PATH` for the commands in this
skill.
```bash
npm install -g @agent-sh/computer-use-linux
computer-use-linux doctor | jq .readiness
```
Rust users can install from crates.io:
```bash
cargo install computer-use-linux
computer-use-linux doctor | jq .readiness
```
If `doctor` reports missing input or accessibility support, run:
```bash
computer-use-linux setup
computer-use-linux setup-window-targeting
computer-use-linux doctor | jq .readiness
```
If `doctor` selects ydotool as the input backend, also enable its per-user daemon. `./install.sh` creates `ydotoold.service`; Ubuntu's ydotool package ships `ydotool.service` instead (`systemctl --user enable --now ydotool`). Direct uinput, X11 xdotool, and RemoteDesktop portal input do not require `ydotoold`. On Wayland, ydotool sends only untargeted scrolls; a scroll at a point goes through uinput or the portal, and without the daemon an untargeted scroll goes through the portal too.
On niri, the server reads `NIRI_SOCKET` from its parent processes or the systemd
user environment when the host does not pass it; without it, discovery needs an
unambiguous socket matching `WAYLAND_DISPLAY`. Window listing and
exact focus use `niri msg` with a direct IPC fallback. Missing positions or
unknown/mixed output scaling leave window-relative coordinates unavailable;
focus the target and inspect a fresh full-screen screenshot before using
desktop coordinates.
On GNOME Wayland, log out and back in after `setup-window-targeting` if the GNOME Shell extension was newly installed.
For MCP hosts with `COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1`, call the optional
`complete_interaction` tool once after finishing desktop interaction. A skipped
cue is not a task failure. This notification does not guarantee exclusive
desktop ownership or that other clients have stopped sending input.
This applies only to directly spawned MCP hosts, not the native Pi extension.
`setup_accessibility` verifies the saved GNOME `toolkit-accessibility` key
separately from runtime AT-SPI. Inspect its warning and readback before assuming
new apps can expose trees. Other accessibility tools may change the key later;
setup does not hold it enabled continuously.
### Optional foreground accessibility guard
Skip unless: the user explicitly wants GNOME's saved `toolkit-accessibility`
setting kept enabled while desktop automation runs.
Run `computer-use-linux guard-accessibility` in a foreground terminal. It
registers a passive AT-SPI window-activation listener and watches/reasserts the
saved key with readback. The setting affects all apps for the current user.
`mcp`, setup, and `get_app_state` never start this guard automatically.
Stop with Ctrl-C or SIGTERM before intentionally disabling accessibility.
Stopping ends writes and removes its listener without disabling other clients
or restoring a previous saved value. Apps launched during a reset/reassertion
race may still need restarting; do not claim a complete GNOME toggle fix.
## Configure Your Agent
The `computer-use-linux` binary is an MCP server. Configure it as a stdio MCP server in your agent of choice:
```json
{
"command": "computer-use-linux",
"args": ["mcp"]
}
```
If the binary is not on `PATH`, use the absolute path (typically `~/.local/bin/computer-use-linux` or the npm global bin directory).
Pi native tools skip this MCP `command` config; see [Pi setup](references/pi-setup.md).
### Host-specific guides
- [Hermes setup](references/hermes-setup.md)
- [Pi coding agent setup](references/pi-setup.md)
- [Selkies container setup](references/selkies-setup.md)
## Procedure
1. In Pi, call `computer_use_linux_tools` with the exact tools or capability you need. Enabled tools use the `computer_use_linux_<name>` prefix, appear starting on the next model turn, and remain active for the session.
2. Begin every desktop-control turn with `get_app_state`, scoped to the app you are working in: pass `app_name_or_bundle_identifier` or a window target (`window_id`, `pid`, `app_id`, `wm_class`, `title`). An `app_id` may be a full desktop ID or its final dot-separated segment; matching is case-insensitive, ambiguous aliases are rejected, and an unresolved explicit `app_id` returns an empty scoped tree rather than the desktop. Without a target the result is the whole desktop AT-SPI tree, `tree_scoped` is `false`, and `message` warns; that can flood context. Use `include_screenshot: false` when the accessibility tree is sufficient. To keep a screenshot out of context until you need it, pass `screenshot_file_path` (an absolute path to a new file whose extension matches `format`; an existing path is refused; stdio only, refused over HTTP): the image is saved there, `screenshot.file_path` names it, and no image content is returned. `screenshot` takes it too讨论
提问、分享技巧或报告无法使用。按“有帮助”数量排序。
来自 GitHub
- GitHub 星标
- 889
- 开发者
- agent-sh
- 许可证
- MIT
- 最近代码更新
- 2026年11月16日
- repo 中的 SKILL.md 文件(含译本和模板)
- 1
- 主题
- accessibility, agent, ai, at-spi, claude, codex, computer-use, gnome, hermes-agent, hyprland, kde, linux