IA-QA — 152 QA & Dev Tools for AI Agents

What you can do with it

We have not written a plain-language summary for this one yet. Below are the facts we read from the registry.

• Connection: Runs in the cloud
• Transport: Streamable HTTP
• Category: Search & AI memory

Install

Pick the AI app you use, then copy the settings.

You need

• Runs in the cloudNothing to install; it connects over the internet

Which app do you use?

https://www.ia-qa.com/mcp

claude mcp add --transport http ia-qa-toolbox https://www.ia-qa.com/mcp

{
  "mcpServers": {
    "ia-qa-toolbox": {
      "url": "https://www.ia-qa.com/mcp"
    }
  }
}

{
  "servers": {
    "ia-qa-toolbox": {
      "type": "http",
      "url": "https://www.ia-qa.com/mcp"
    }
  }
}

https://www.ia-qa.com/mcp

Your choice stays on this device in a cookie; no account needed.

These settings are generated from the maker’s registry entry. If the README says otherwise, follow the README.

What it can do

Tool names and descriptions exactly as the server reported them on Nov 7, 2026. The maker wrote them, not us; we show them as plain text.

• get_testing_guidelines

  Query the IA-QA methodology knowledge base. Returns structured testing guidelines, assertion strategies, thresholds, best practices, and relevant MCP tools for a given topic. Call without a topic to list all available topics. Topics: llm-unit-testing, rag-pipeline, prompt-stabili

• find_tool

  Search available MCP tools by keyword or category before calling them. Returns names, descriptions and, on request, inputSchemas. Queries are logged 90 days, screened for credentials and personal data. Categories: data, dev, encoding, eval, integrations, llm, meta, qa, rag, secur

• generate_test_cases

  Generate a set of test cases (valid, edge, invalid, pairwise) for a given feature description. Declared constraints drive the boundaries: a length or numeric bound ("[8-64]", "min 8 chars", "at least 8 characters" — read from inputs, and from the feature prose when a sentence nam

• jira_to_test_suite

  Turn a Jira ticket into a test suite: Gherkin scenarios, E2E steps, API tests, test data and the ambiguities to clarify. Pass the issue object, or Jira credentials to fetch it. It never invents: a thin ticket returns an empty suite with verdict INCOMPLETE_SPEC and the questions t

• analyze_diff_bugs

  Pattern-based diff linter: flags a fixed set of risky shapes in changed code — query-string interpolation (SQL/Cypher/Mongo injection shape), shell interpolation, eval/new Function, empty catch blocks, regex built from a variable, fewer catch blocks than before, and named authori

• run_pr_gate_pipeline

  Review triage for a pull request. Takes a unified git diff (`git diff HEAD`) and returns: diff-lint findings with the lines that produced them, regression impact areas, a risk score 0–100 with the factors that built it (churn, files touched, sensitive paths, whether any test file

• diff_mappings

  Diff a baseline page mapping against a current one and return a CI-style verdict: PASS / FIX / BLOCK, plus per-element drift (ok, renamed, healable, ambiguous, lost, added, rebound). Pure and deterministic — provide two mappings as JSON with "elements" arrays of {role, name, sele

• sandbox_scenario

  Get a ready-made selector-drift test case with a known-correct answer, for testing this MCP server or an agent workflow end to end. Each scenario is a real DOM capture of a deliberately breakable app, taken before and after one specific UI change (a renamed label, two swapped but

• prompt_injection_scan

  Scan user input or prompts for prompt injection patterns. Pattern-based: it recognises known phrasings and structures, not intent — an uncovered paraphrase or language reads clean, so clean is not proof of safety. Instruction overrides are covered in 11 languages (EN, FR, ES, DE,

• hallucination_check

  Lexical hallucination check: verifies an LLM answer's words, numbers and polarity against the provided source/context. Fast, deterministic, no API key needed. Each answer sentence is aligned to its best-matching source sentence, so a number only counts as support when it sits on

• run_eval_contract

  Run an IA-EVAL test suite (.ia-eval.yaml contract): call the model per scenario; return passed (true, false, null if undecided) and a verdict (PASS, FAIL, PARTIAL). Expectations are assertions that fail a scenario whatever its score; evaluators give the score. Hosted: pass inline

• guardrail_test

  Test an LLM response against a set of guardrail rules: must-include, must-not-include, max length, required format, language, forbidden patterns, and custom regex. Returns pass/fail per rule.

• run_semantic_tests

  Semantic assertion primitive: compare actual vs expected text pairs using cosine similarity + ROUGE-L. Two modes: tfidf (default, free, no API key) or embeddings (OpenAI text-embedding-3-small, BYOK, true semantic similarity). Returns per-case PASS/FAIL verdicts and an overall ve

• generate_ci_workflow

  Generate a ready-to-commit GitHub Actions workflow that gates a build on IA-QA: "eval_contract" runs a .ia-eval.yaml through ia-qa-com/eval-action@v1 (needs a provider key as a repo secret), "cli_checks" runs deterministic tools via npx @ia-qa/cli and fails the build on their ver

• redact_pii

  Automatically detect and redact Personally Identifiable Information (PII) from text. Replaces emails, phone numbers, SSNs, IBANs, credit cards, IP addresses, and JWT tokens with [REDACTED_TYPE] placeholders. Safe to use before logging or sending to an LLM.

• secret_scan Looks like an instruction to the AI

  Scan text or code for leaked secrets: API keys (AWS, GCP, Azure, OpenAI, Anthropic, Stripe, GitHub, GitLab, Slack, Twilio, SendGrid, HuggingFace), Slack / Teams / Discord webhook URLs, private keys, JWTs, database connection strings, Bearer and Basic auth headers, and hard-coded

• needle_haystack_generate

  Generate a "needle in a haystack" test: embeds a target fact into a large block of filler text at a specified position. Use this to test LLM context window retrieval accuracy. Returns the full haystack, the question to ask, and metadata. No API key needed.

• list_local_tests

  Discover .ia-eval.yaml LLM test suite files in the project directory. Scans CWD and standard sub-directories (evals/, tests/, contracts/). Returns file paths ready to pass to run_eval_contract.

• generate_html_report

  Convert a run_eval_contract() LLM Test Runner JSON result into a fully self-contained dark-themed HTML report with Pass/Fail badges, side-by-side Input/Output/Ground-Truth panels, evaluator score bars, and a radar chart. Returns the HTML as a string.

• generate_eval_yaml

  Generate a complete .ia-eval.yaml evaluation contract from a plain-language description of what your LLM should do. Uses Groq openai/gpt-oss-20b (server-side, no API key needed). Returns ready-to-run YAML for the LLM Test Runner (run_eval_contract). Picks appropriate evaluators (

• validate_agent_trajectory

  Run declarative assertions on an agent trace (OpenAI tool calls, Anthropic tool_use, LangChain run trees, ReAct logs). Deterministic, no LLM. Assertions: order (tool A before B), must_call, must_not_call, max/min calls, no_error, recovery (a successful step after the last error),

• metamorphic_check

  Reference-free stability check: does an assistant's answer stay the same when the QUESTION is transformed (case, typo, paraphrase, reorder, translation)? You bring the outputs; no model is called (tfidf mode is deterministic and free). Default thresholds: case .95, typo .90, para

• test_skill

  Test a SKILL.md (Agent Skills format). (1) lint: the spec's frontmatter rules, no model — an error returns BLOCK; gate on lint.valid. (2) triggering: a model sees only the names + descriptions of this skill and its neighbours and picks which to load (temperature 0); trigger_accur

• diff_text

  Compute a unified line-by-line diff between two text strings (LCS algorithm). Returns added/removed/unchanged line counts and formatted diff hunks with configurable context lines (0–20). Use to compare versions of prompts, configs, code snippets, or any text where you need to see

• validate_mcp_response

  Validate that an MCP tool response conforms to expected format, schema, and content rules. Use this to QA-test any MCP server tool. Supply the tool's actual JSON result and a set of checks to perform.

• llm_output_validator

  Validate an LLM response against QA criteria: format checks (JSON, code, markdown), content rules (must-include, must-not-include), length constraints, language detection, and safety patterns. Essential for QA testing LLM-powered features.

• compare_responses

  Compare two ALREADY-PRODUCED outputs (e.g. model A vs model B on the same task) side by side. Returns deterministic metrics (token cosine, ROUGE-L, Jaccard, length/structure deltas, JSON diff) and a verdict. If a `reference` (ground truth) is given, scores each output against it

• analyze_responses

  Semantically analyze N already-produced model outputs for the SAME task (the MCP counterpart to the LLM Sandbox). Without a reference: computes consensus — pairwise cosine agreement, the most-representative output, and the outlier. With a `reference` (ground truth): also ranks ev

• prompt_test_suite

  Define a test suite for a prompt: provide the system prompt, user prompt, and expected output criteria. Returns a test plan with scored rubric — use this as input for manual or automated LLM evaluation.

• mcp_server_health_check

  Check an MCP server's tool definitions without contacting it — for a server not deployed yet, a stdio server, or a CI gate. Paste the answer to tools/list (raw JSON-RPC or its result), a bare array of tools, one tool definition, or a manifest carrying name/description; input_shap

• mcp_server_evaluate

  Check that a live MCP server over Streamable HTTP speaks the protocol, 2026-07-28 or earlier: handshake or server/discover, tools/list and its definitions, error codes, one tools/call, structuredContent against outputSchema, latency. Returns PASS/FIX/BLOCK with a 0-100 score, or

• llm_format_check

  Validate that an LLM output matches an expected format: JSON, Markdown, code block, bullet list, numbered list, table, YAML, XML, or custom regex. Essential for structured output testing.

• consistency_check

  Compare multiple LLM responses to the same prompt and detect inconsistencies using Jaccard word-overlap similarity and fact drift (number comparison). Fast, deterministic, no API key needed. Limitations: relies on surface-level word matching — "Paris is the capital of France" vs

• llm_json_schema_check

  Validate that an LLM JSON output matches a JSON Schema definition. Tests required fields, types, enums, nested objects, and arrays. Critical for function-calling and structured output testing.

• response_quality_score

  Score an LLM response against the criteria you pass: coverage of expected_keywords and compliance with max_length. Returns a 0-100 score over the criteria actually measured — or total_score null with the reason when none is given, because relevance and correctness cannot be read

• toxicity_scan

  Scan text for toxic language, hate speech, bias/stereotype framing, violence, sexual and self-harm content. Lexical + structural pattern matching (identity term + predicate), not a semantic classifier — returns per-category risk plus the named rules that fired, so every finding c

• function_call_validate

  Validate an LLM function call / tool_use output: check that function name is in allowed list, arguments match expected schema (types, enums, ranges), no extra/missing args. Reads the definitions as OpenAI (`parameters`), MCP (`inputSchema`) or Anthropic (`input_schema`) write the

• conversation_analyze

  Analyze a multi-turn conversation for context retention, topic drift, instruction following, and repetition. Accepts messages array [{role, content}]. Essential for chatbot QA.

• mcp_schema_lint

  Lint one MCP tool definition with the same rules as mcp_server_health_check: errors break the MCP spec (FAIL), documentation_issues are what its strict mode refuses — a parameter without description or type (FIX, never an A) — and suggestions are style only (snake_case) and chang

• cot_analyzer

  Analyze a Chain-of-Thought (CoT) or reasoning trace from an LLM. Detects step count, logical flow, conclusion presence, backtracking, and estimates reasoning depth. Useful for o1/o3/DeepSeek-R1 evaluation.

• ab_test_report

  Generate an A/B test report comparing two prompts or model configurations. Accepts arrays of scores and returns mean, median, std deviation, improvement percentage, and a Welch t-test at 95 %: `winner` names a variant only when the difference is significant — otherwise "no_signif

• similarity_score

  Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU. No API key needed. Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking. Supports batch mode.

• bias_detect

  Analyse a set of LLM responses generated from the same prompt template but with different demographic variants (gender, origin, age, tone). Returns a bias score (0-100), sentiment analysis per variant, pairwise Jaccard similarity, and a human-readable verdict. No API key needed —

• run_vlm_test_suite

  Test a vision model (OpenAI, Anthropic or Gemini) on one image: each test case asks a question and asserts on the answer (contains, not_contains, json_format, min_length, max_length, semantic_contains = TF-IDF cosine ≥ 0.4). Returns per-case PASS/FAIL, the pass rate, an overall P

• multimodal_eval_guide

  Multimodal evaluation: action=guide (default) interprets a CLIP/FID/VQA score against reference thresholds; clip_score, fid_score, vqa_accuracy compute the metric (HuggingFace, or your VLM key for VQA); pipeline runs all three.

• run_vlm_test_suite_batch

  Compare several vision models on the same image and test cases, in parallel (same assertions as run_vlm_test_suite). Returns per-model PASS/FAIL, pass rates, latency and a comparison table. BYOK: one key per model.

• shield_analyze

  One-call safety gate for an LLM response: grounding (with source), prompt injection on the response AND the prompt (checks.injection / checks.injection_prompt), toxicity, PII, guardrail rules, quality, plus an LLM review on Groq (server key; model "none" skips it and sends nothin

• security_headers_check

  Analyse the HTTP security headers of a public URL OR of raw response headers you paste in. Grades each header (A–F) for: Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, X-XSS-Protection, Cross-Origi

• ssl_certificate_check

  Analyse the SSL/TLS certificate of any HTTPS host. Returns certificate subject, issuer, validity dates, days until expiry, protocol version, cipher suite, key exchange info, and an overall grade (A+, A, B, C, F). Detects expired, self-signed, and weak certificates. Use this to au

• cors_test

  Audit a URL for CORS misconfigurations across several origins: wildcard with credentials, reflected origin, null origin, subdomain bypass, missing Vary. Returns risk level (safe/low/medium/high/critical) plus per-origin results. "unknown" means nothing was tested: every origin fa

• cors_checker

  Would a browser on this origin be allowed to read this URL? Verdict allowed | blocked | unknown, the reason, and the header to add. Only GET and HEAD are really sent: other methods measure the preflight alone. To audit many origins use cors_test.

• cookie_security_audit

  Audit the security attributes of cookies set by any URL. Fetches the URL and inspects all Set-Cookie headers for: HttpOnly, Secure, SameSite, Domain scope, Path scope, Max-Age/Expires, __Host-/__Secure- prefixes. Flags insecure patterns: missing HttpOnly on session cookies, missi

• web_security_audit

  Run a comprehensive web security audit combining headers, SSL, CORS, and cookies checks — then use an LLM to produce a prioritised remediation plan. Orchestrates security_headers_check + ssl_certificate_check + cors_test + cookie_security_audit in parallel, merges all findings, t

• embedding_similarity

  Compute text similarity using local algorithms (Bag of Words, TF-IDF, Character N-grams). No API key needed — runs entirely in-process. NOT real embeddings: for true semantic similarity with vector embeddings, use run_semantic_tests with mode="embeddings" and your OpenAI API key.

• rag_relevance_rank

  Rank an array of text chunks by relevance to a query using TF-IDF scoring. Simulates retrieval ranking for RAG testing without needing embeddings or an API.

• vector_similarity

  Compute similarity/distance between two float vectors: cosine similarity, dot product, Euclidean and Manhattan distance. Essential for vector DB relevance scoring, embedding evaluation, and nearest-neighbor testing.

• normalize_vector

  L2-normalize a float vector (produce a unit vector with norm=1). Required by many vector DBs (Pinecone, Qdrant cosine). Supports batch normalization of up to 1000 vectors.

• vector_quantize

  Simulate int8 or int4 quantization of float32 embedding vectors. Reduces storage by 4x (int8) or 8x (int4). Returns quantized values, scale factor, and precision loss (MSE). Useful for understanding vector DB compression trade-offs.

• vector_stats

  Compute statistics for a float vector or matrix of vectors: mean, std, L2 norm, min, max, sparsity, top-K indices. Useful for debugging embedding quality and analyzing vector distributions in a vector DB.

• bm25_score

  Compute BM25 relevance score between a query and one or more documents. BM25 is the industry-standard keyword-based ranking algorithm used in Elasticsearch, OpenSearch, and Weaviate hybrid search. Returns ranked results with normalized scores.

Basic checksi

We check information anyone can see; we have not read the code. Checks that do not apply are left out of the count. · Checked Nov 7, 2026

• ! Caution

  Maker matches the repo

  The publisher (JcJamet) is not the repo owner (ia-qa-com); someone else may have listed it

• … Not checked yet

  Still maintained

  We have not read this repo from GitHub yet

• … Not checked yet

  Has a licence

  We have not read this repo from GitHub yet

• ✓ Pass

  Endpoint responds

  Responded in 811 ms (HTTP 200)

• – Not applicable

  Sign-in with OAuth

  No sign-in needed

• ✕ Fail

  No hidden instructions in tool descriptions

  1 tool descriptions contain text that looks like instructions to the AI. Read them below before installing

Compared with others in Search & AI memoryi

• More stars than 0% of the other 5650 servers in this category
• Published to the registry more recently than 76% of the other 5650 servers in this category
• Basic checks equal to or better than 28% of the other 5650 servers in this category

Compared with every active server in this category in the registry. Stars count only when the publisher owns the repo.

Alternatives

Servers in the same category from other makers, sorted by stars.

Reading a card:★ GitHub starsiThree pillars: reviews · checks · usageiUpdated = latest version published to the registryi

README from the maker

claude mcp add --transport http ia-qa https://www.ia-qa.com/mcp

{

{