OpenJudge Skill
Inicia sesión para dar me gusta0 me gusta
★ Stars are for the whole repo, not each skill
What it does
This repo is a collection of 21 skills
- Connects Instruction files the AI reads, with steps that call an outside service
- Account or key Some steps need a key for MINIMAX_API_KEY (per the README)
- Category Automatización y agentes
Maker’s original description
OpenJudge: A Unified Framework for Holistic Evaluation and Quality RewardsPara autores: añade una insignia a tu README
Si lo creaste tú, pega este código en tu README. La insignia enlaza a esta página.
[](https://mcp.popnx.com/en/skills/agentscope-ai-openjudge)Skills parecidas
Mismo tipo de trabajo, ordenadas como la lista de Skills
21 skills (conjuntos de instrucciones) de agentscope-ai que la IA lee y sigue · no es un MCP server
Pick the skill to install
Install only the ones you use. Open a skill to see how to install it
Install the whole set (21)
If you want every one, let the AI install them in one go. Know 2 things first
- Every installed skill’s description is loaded into every chat. The more you install, the easier it is for the AI to pick the wrong one
- Some skills duplicate tools the AI already has built in. Installing only the ones you use works better
Uploading to Claude on the web and apps: you need 21 uploads, one ZIP per skill. Open each skill in the list below to download it
Checked against official docs 12 nov 2026 · Source: support.claude.com, code.claude.com, learn.chatgpt.com, cursor.com
- 00-academic-router
Use when the user wants help with academic papers or citations but it's unclear which specific workflow fits — reviewing a paper, checking a BibTeX file for fake references, or benchmarking multiple LLMs on reference-recommendation accuracy. Also use when the user mentions paper review, peer review…
- 00-arena-router
Use when the user wants to compare or benchmark multiple LLMs/agents arena-style but it's unclear which specific workflow fits — a general-purpose win-rate comparison on a custom task, or a benchmark specifically about reference/citation hallucination rate. Also use when the user mentions model are…
- 00-meta-eval
Use when the user wants to build an evaluation system for an LLM/agent application but doesn't know where to start — they have traces, prompts, RAG pipelines, or nothing at all. Also use when the user mentions evaluation, eval, benchmarking, testing LLM quality, measuring agent performance, assessi…
- 01-auto-arena
Automatically evaluate and compare multiple AI models or agents without pre-existing test data. Generates test queries from a task description, collects responses from all target endpoints, auto-generates evaluation rubrics, runs pairwise comparisons via a judge model, and produces win-rate ranking…
- 01-eval-design
Use when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples, extract eval dimensions from traces/specs, or build a labeled evaluation set. Also use when the user mentions test data design, eval coverage, difficulty stratification, synthe…
- 01-graders-and-pipeline
Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating grad…
- 01-paper-review
Review academic papers for correctness, quality, and novelty using OpenJudge's multi-stage pipeline. Supports PDF files and LaTeX source packages (.tar.gz/.zip). Covers 10 disciplines: cs, medicine, physics, chemistry, biology, economics, psychology, environmental_science, mathematics, social_scien…
- 02-bib-verify
Verify a BibTeX file for hallucinated or fabricated references by cross-checking every entry against CrossRef, arXiv, and DBLP. Reports each reference as verified, suspect, or not found, with field-level mismatch details (title, authors, year, DOI). Use when the user wants to check a .bib file for…
- 02-metric-design
Use when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics, creating LLM-as-judge prompts, combining multiple metrics into a composite score, or building an automated evaluation pipeline. Also use when the user mentions grader se…
- 02-ref-hallucination-arena
Benchmark LLM reference recommendation capabilities by verifying every cited paper against Crossref, PubMed, arXiv, and DBLP. Measures hallucination rate, per-field accuracy (title/author/year/DOI), discipline breakdown, and year constraint compliance. Supports tool-augmented (ReAct + web search) m…
- 02-rl-reward
Build RL reward signals using the OpenJudge framework. Covers choosing between pointwise and pairwise reward strategies based on RL algorithm, task type, and cost; aggregating multi-dimensional pointwise scores into a scalar reward; pairwise tournament reward for GRPO on subjective tasks (net win r…
- 03-align-human
Use when the user has a judge/grader and human-labeled data, and wants to measure how well the judge agrees with humans, detect systematic biases, determine whether automatic evaluation can replace human review, or build a human-reduction roadmap. Also use when the user mentions calibration, TPR/TN…
- 03-ref-hallucination-arena
Benchmark LLM reference recommendation capabilities by verifying every cited paper against Crossref, PubMed, arXiv, and DBLP. Measures hallucination rate, per-field accuracy (title/author/year/DOI), discipline breakdown, and year constraint compliance. Supports tool-augmented (ReAct + web search) m…
- 04-eval-report
Use when the user has run multiple evaluation skills and wants a comprehensive analysis — maturity assessment, cross-skill signals, trends, prioritized actions, and an executive summary. Also use when the user mentions eval health check, evaluation audit, ship readiness, evaluation maturity, or "ho…
- 05-rag-eval
Use when the user has a RAG (Retrieval-Augmented Generation) system and wants to evaluate its quality — separating retrieval issues from generation issues. Also use when the user mentions RAG evaluation, faithfulness checking, hallucination detection in RAG, retrieval quality, chunking optimization…
- 06-prompt-regression
Use when the user has changed a prompt (system prompt, RAG template, agent instruction, etc.) and wants to know whether the candidate is better or worse than the baseline. Also use when the user mentions prompt A/B testing, prompt comparison, prompt optimization validation, "did my prompt change he…
- 07-redteam
Use when the user wants to test their LLM/agent application for safety and security vulnerabilities — jailbreaks, prompt injection, PII extraction, harmful content generation, or evaluator gaming. Also use when the user mentions security testing, adversarial testing, red teaming, safety evaluation,…
- 08-bootstrap
Use when the user has nothing — no traces, no labels, no eval set — and needs to build a v0 evaluation from scratch. Also use when the user says "I need to start evaluating my app but don't know where to begin," "I want to set up eval for a new product," or has just identified failure modes and nee…
- claude-authenticity
Detect whether an API endpoint is backed by genuine Claude (not a wrapper, proxy, or impersonator) using 9 weighted rule-based checks that mirror the claude-verify project. Also extracts injected system prompts from providers that override Claude's identity. Fully self-contained — copy the code bel…
- find-skills-combo
Discover and recommend **combinations** of agent skills to complete complex, multi-faceted tasks. Provides two recommendation strategies — **Maximum Quality** (best skill per subtask) and **Minimum Dependencies** (fewest installs). Use this skill whenever the user wants to find skills, asks "how do…
- mmx-cli
Generate text, images, video, speech, and music via the MiniMax AI platform. Covers text generation (MiniMax-M3 model), image generation (image-01), video generation (Hailuo-2.3), speech synthesis (speech-2.8-hd, 300+ voices), music generation (music-2.6 with lyrics, cover, and instrumental), and w…
En qué se diferencia un Skill de MCP
MCP = las manos
Conecta la IA a un servicio o dato externo, como leer Notion o crear una factura. Necesita un programa o servidor en marcha.
Skill = el manual
Un archivo de texto que enseña a la IA cómo hacer una tarea. No se conecta a nada.
A veces se parecen: algunas Skills ejecutan scripts que llaman herramientas y algunos MCP solo envían conocimiento. Empieza por la tarea que quieres hacer.
Funciona con
El autor no lo indica · los skills con SKILL.md funcionan en apps compatibles con Agent Skills, como Claude Code, Codex y Cursor
Instalar
Pick a skill first
This repo has 21 skills. Pick one from the list above; its page shows every way to install it, including letting the AI install it and a ZIP
Conversación
Pregunta, comparte un consejo o avisa de que no funciona. Ordenado por lo que la gente marcó como útil.
Aún no hay publicaciones. Sé la primera persona en preguntar o compartir.
Lo publicado aquí lo escriben usuarios, no el equipo de PopNx.
Desde GitHub
- Estrellas en GitHub
- 1113
Stars are for the whole repo, not each skill - Autor
- agentscope-ai
- Licencia
- Apache-2.0
- Último cambio de código
- 18 nov 2026
- Archivos SKILL.md del repo (con traducciones y plantillas)
- 21
- Temas
- agent, agent-skills, ai-agent, alignment, evaluation, grader, llm, reward, reward-model, rlhf, skill-md, skills