Kubently

Kubently

Kubently - Troubleshooting Kubernetes Agentically

Overview

Kubently (*Kubernetes + Agentically*) is a free, self-hosted, vendor-neutral multi-cluster Kubernetes troubleshooter. Ask one question, get AI-diagnosed answers from every cluster in your fleet in parallel — including clusters you can't reach directly: executors dial outbound to the central API, so there's no inbound ingress, no shared kubeconfig, and no per-cluster credentials to distribute.

Agents collaborate over the A2A (Agent-to-Agent) protocol, and any MCP client (Claude Code, Cursor, Claude Desktop) can use Kubently as a tool out of the box.

Key Features

• Multi-Cluster Fleet Troubleshooting: One question fans out across all registered clusters in parallel
• Outbound-Dial Executors: Reach clusters behind firewalls/NAT — no inbound ingress, no shared kubeconfig
• Natural Language Interface: Conversational Kubernetes troubleshooting and debugging
• Comprehensive Analysis: Automated issue detection, root cause analysis, and solution recommendations
• Multi-LLM Support: Compatible with Google Gemini, OpenAI, Anthropic, and other providers
• A2A Protocol: Industry-standard agent-to-agent communication for complex workflows
• MCP Server: Optional Model Context Protocol endpoint so MCP clients (Claude Desktop, Cursor, custom agents) get direct tool access
• Security-First: API key authentication, OAuth/OIDC support, and TLS with cert-manager
• Persistent Sessions: Redis-backed conversation history and context management
• Extensive Tool Suite: kubectl integration, log analysis, resource inspection, and more

Quick Start

For Users: Get Started in 5 Minutes

Point kubectl at any cluster (kind, minikube, or real) and run:

npm install -g @kubently/cli

kubently install

That's it. The CLI installs Kubently via Helm, wires up secrets and the executor, port-forwards the API, and drops you into a debug chat:

kubently> why is my nginx pod crashlooping?

You'll need an LLM API key (Anthropic, OpenAI, or Google) — the installer prompts for it, or reads ANTHROPIC_API_KEY / OPENAI_API_KEY / GOOGLE_API_KEY from your environment. Use --provider to pick the LLM, --chart ./deployment/helm/kubently to install from a local checkout, and kubently install --help for everything else.

Use from Claude Code / Cursor (MCP)

Already ran kubently install? Add Kubently to Claude Code:

claude mcp add kubently -- kubently mcp

Or connect directly over HTTP (no bridge process):

claude mcp add --transport http kubently http://localhost:8080/mcp/ \

--header "X-API-Key: "

Then ask Claude things like *"use kubently to figure out why payments pods are crashlooping"*. Any MCP client works — see docs/MCP.md for Cursor and generic configuration.

Proactive diagnosis (Alertmanager → Slack)

Set api.env.SLACK_WEBHOOK_URL to a Slack incoming-webhook URL and point Alertmanager at Kubently:

receivers:

• name: kubently webhook_configs:
• url: https:// /webhooks/alertmanager http_config: http_headers: # Alertmanager >= 0.28 X-API-Key: secrets: [" "]

Each firing alert is diagnosed by the agent and the result is posted to Slack — the bot often explains the root cause before you've opened your laptop.

Scheduled fleet health digest

Alerts are reactive. A digest sweeps *every* registered cluster on a schedule and posts one summary to the same Slack webhook — healthy clusters collapse to a single line, so what's left is what needs you.

fleetReport:

enabled: true schedule: "0 13 * * 1-5" # weekday mornings

Preview it before you schedule it — dry_run returns the digest and posts nothing:

curl -X POST https://<your-kubently-host>/webhooks/fleet-report \

-H "X-API-Key: " -H 'Content-Type: application/json' \ -d '{"dry_run": true}'

The digest question is yours to change. Pass query in that request to try one immediately, then keep the wording you like via fleetReport.query in values:

fleetReport:

query: |- Check every cluster for pods restarting more than 5 times and for PVCs above 85% usage. One line per healthy cluster. No preamble.

To run the real scheduled path once — image, secrets, in-cluster URL and all:

kubectl create job --from=cronjob/kubently-fleet-report fleet-report-test -n kubently

Deployment verification (did that deploy actually work?)

Tell Kubently what you just deployed and it watches the rollout settle, then runs a real investigation — pods ready? events clean? errors in the new logs? metrics regressed vs the pre-deploy window (when Prometheus is configured)? — and posts a PASS/FAIL verdict with the evidence to Slack. Wire it into the last step of your CI pipeline:

curl -X POST https://<your-kubently-host>/webhooks/verify-deployment \

-H "X-API-Key: " -H 'Content-Type: application/json' \ -d '{"cluster": "prod-east", "namespace": "shop", "workload": "deploy/checkout-api", "context": "v1.42.0"}'

Add "dry_run": true to get the verdict back synchronously without posting. No CI access? Label the workload instead — kubently.io/verify=enabled — and enable verifyDeployment.watch in values: Kubently notices every generation change and verifies the rollout unprompted.

Scheduled checks (your recurring questions, on cron)

The digest asks one broad question. Scheduled checks let you ask *your* questions on *their* schedules — each check is a named prompt with a cron schedule and optional target clusters, run by the agent and posted to Slack:

scheduledChecks:

enabled: true checks:

• name: cert-expiry schedule: "0 8 * * 1" # Monday mornings prompt: |- Check TLS secrets for certificates expiring within 21 days. List each as namespace/name with days remaining.
• name: pvc-pressure schedule: "0 */6 * * *" clusters: [prod-east] prompt: Find PersistentVolumeClaims above 85% usage.

A passing check posts nothing — silence means green (set notifyOnPass: true to hear about passes too). Failures always post, evidence included. Iterate on a check without waiting for cron:

curl -X POST https://<your-kubently-host>/webhooks/scheduled-check \

-H "X-API-Key: " -H 'Content-Type: application/json' \ -d '{"check": "cert-expiry", "dry_run": true}'

📖 See [QUICK_START.md](docs/QUICK_START.md) for full quick-start guide

📚 See [GETTING_STARTED.md](docs/GETTING_STARTED.md) for production deployment

For Developers: Local Testing

# Deploy to a local kind cluster (builds images from HEAD)

ANTHROPIC_API_KEY=sk-... ./deployment/scripts/kind-e2e.sh

Run comprehensive test suite

./test-automation/run_tests.sh test-and-analyze --api-key test-api-key

📖 See [CLAUDE.md](CLAUDE.md) for development guidelines

Configuration

LLM Providers

Pick a provider with LLM_PROVIDER and supply that provider's key. There is no default provider — the agent refuses to start without LLM_PROVIDER. For local development with deployment/docker-compose.yaml, put both in .env (see deployment/.env.example):

# Anthropic

LLM_PROVIDER=anthropic-claude ANTHROPIC_API_KEY=your-anthropic-api-key

OpenAI (also matches Azure and OpenAI-compatible endpoints)

LLM_PROVIDER=openai OPENAI_API_KEY=your-openai-api-key

Google Gemini

LLM_PROVIDER=google-gemini GOOGLE_API_KEY=your-gemini-api-key

In Kubernetes the keys come from the kubently-llm-secrets secret and LLM_PROVIDER goes under api.env.

Helm Deployment

Customize deployment using Helm values:

Kubently ships as a single chart. Its components are switched on and off with api.enabled, redis.enabled and executor.enabled — an executor-only install on a remote cluster is the same chart with the first two disabled.

# From a checkout

helm install kubently ./deployment/helm/kubently -n kubently \ -f deployment/helm/test-values.yaml

Or from the published chart repository

helm repo add kubently https://kubently.github.io/kubently helm install kubently kubently/kubently -n kubently -f my-values.yaml

LLM_PROVIDER is required and has no chart default — set it under api.env (anthropic-claude, openai, or google-gemini). See ENVIRONMENT_VARIABLES.md for the full configuration surface, and GETTING_STARTED.md for the production walkthrough.

Operator Runbooks

Feed your organization's tribal knowledge into investigations. Runbooks are hand-written markdown files with lightweight frontmatter; when an investigation (a chat question, an Alertmanager alert, or an A2A call) matches a runbook's criteria, the agent receives it as "the operator's runbook for this situation", follows it where applicable, notes deviations, and cites it by name in the diagnosis.

A worked example:

---

name: Payments CrashLoopBackOff match: alerts: ["KubePodCrashLooping", "PaymentsPod*"] # alert-name globs namespaces: ["payments", "payments-*"] # namespace selectors workloads: ["payment-api*"] # matches derived pod names too topics: ["crashloop", "OOMKilled", "payment service"] # free-text tags

---

1. Check recent deploys first: payment-api ships through ArgoCD, and 90% of crashloops here follow a bad config sync.
2. OOMKilled almost always means the JVM heap flag drifted from the container memory limit — compare -Xmx against resources.limits.memory before blaming traffic.
3. If the DB connection pool is exhausted, do NOT restart the pods; escalate to #payments-oncall (restarts thundering-herd the database).

Deploy runbooks as Helm values (they become a ConfigMap mounted into the API pod; edits go live without a pod restart):

# production-values.yaml

runbooks: payments-crashloop.md: |

---

name: Payments CrashLoopBackOff match: alerts: ["KubePodCrashLooping"] namespaces: ["payments"]

---

1. Check recent deploys first ...

Matching is scored: an alert-name hit outranks namespace/workload selector hits, which outrank topic hits. The best match is injected first, and the total injected size is capped (KUBENTLY_RUNBOOKS_MAX_CHARS, default 8000 characters) — one complete, best-matching runbook beats fragments of many. Outside Helm, point KUBENTLY_RUNBOOKS_DIR at any directory of .md files.

Incident History

Past diagnoses become searchable institutional memory. Whenever an investigation concludes with a root cause, Kubently stores a compact record — date, cluster, resources involved, symptom keywords, the root-cause one-liner, and the resolution when one was stated — in Redis, isolated per authenticated caller (the same namespace boundary as conversation memory, so in multi-tenant deployments one tenant's incidents are never visible to another).

The history is used two ways:

• The agent's