Agents Picked the Better-Described Tool 72% of the Time. Here's How to Write One.

Think of the tools your API exposes to agents, or the operations in your OpenAPI spec. How many of their descriptions did a person on your team write by hand?

Now count it properly. If your spec is JSON, this takes ten seconds:

# Operations with no description at all
jq '[.paths[] | to_entries[] | select(.key | IN("get","post","put","patch","delete")) | .value | select(.description == null)] | length' openapi.json

Then count the operationIds an agent would have to guess at:

# Missing, or shaped like get_api_v1_users_1
jq -r '.paths[] | to_entries[] | select(.key | IN("get","post","put","patch","delete")) | .value.operationId // "MISSING"' openapi.json \
  | grep -cE '^(get|post|put|patch|delete)_|_[0-9]+$|MISSING'

Most people are surprised by the second number.

Why those two numbers matter

An agent calling your API doesn't read your guides the way you do. It reads three things, and picks tools based on the last one:

In a February 2026 preprint, Wang et al. scanned 10,831 MCP servers. 73% had repeated tool names. Then they put functionally identical servers side by side: agents picked the one with standard-compliant descriptions 72% of the time, against a 20% baseline (arXiv 2602.18914, not yet peer-reviewed).

The descriptions in your spec usually become the descriptions on your tools. So the two counts above are a decent proxy for how often an agent will pick your API, or pick the wrong operation in it.

Here's how to fix each layer.

1. llms.txt: give the agent a way in

llms.txt is a Markdown file at the root of your docs. The spec is short: an H1, a blockquote summary, optional notes, then H2 sections that list links. A section called Optional holds links an agent can skip.

# Acme Payments API
> REST API for charges, refunds and payouts. JSON over HTTPS.
> Auth: OAuth 2.0 client credentials. Base URL: https://api.acme.com/v2

Amounts are in minor units (cents). Write calls accept an Idempotency-Key header.

## Docs
- [Quickstart](https://docs.acme.com/quickstart.md): Token and first charge
- [Authentication](https://docs.acme.com/auth.md): Scopes, expiry, errors
- [OpenAPI spec](https://docs.acme.com/openapi.json): Full contract

## Reference
- [Charges](https://docs.acme.com/reference/charges.md): Create, capture, void
- [Refunds](https://docs.acme.com/reference/refunds.md): Full and partial

## Optional
- [Changelog](https://docs.acme.com/changelog.md): Breaking changes by version

Two gotchas:

• Link to Markdown, not HTML: If your docs render with JavaScript, an agent fetching the HTML gets an empty shell. The spec suggests serving a .md version at the same URL.
• llms-full.txt isn't in the spec: It's a convention some docs platforms use, with every page in one file. Fine for small docs, too big for most API references.

2. OpenAPI: make the contract complete

Here's a typical operation, as many generators leave it:

# Before
/refunds:
  post:
    operationId: post_refunds_1
    summary: Refund
    requestBody:
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Refund'
    responses:
      '200':
        description: OK

And the same operation written for an agent:

# After
/refunds:
  post:
    operationId: createRefund
    summary: Refund a captured payment
    description: >
      Refunds all or part of a captured payment. Use after a charge succeeds.
      For a pending authorization, use voidAuthorization instead.
      Amount is in cents; omit it for a full refund.
    security:
      - oauth2: [refunds:write]
    parameters:
      - name: Idempotency-Key
        in: header
        required: true
        schema: { type: string, format: uuid }
    requestBody:
      required: true
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/RefundRequest'
          example: { payment_id: "pay_8Kd2", amount: 1500 }
    responses:
      '201':
        description: Refund created
        content:
          application/json:
            example: { refund_id: "re_91Xa", status: "pending" }
      '409':
        description: Payment already fully refunded
      '422':
        description: Amount is more than the refundable balance

What changed:

• operationId: A verb and a noun.
• description: Says when to use it, and when to use something else.
• security: On the operation, not only in a guide.
• Examples: On the request and the response. Agents copy them closely.
• Error responses: They tell the agent how to recover.

3. Tool descriptions: write the part that gets you picked

If you expose the API through MCP, each operation becomes a tool. Here's a generated one:

{
  "name": "post_refunds_1",
  "description": "Refund",
  "inputSchema": {
    "type": "object",
    "properties": {
      "payment_id": { "type": "string" },
      "amount": { "type": "integer" }
    }
  }
}

And a written one:

{
  "name": "create_refund",
  "description": "Refunds all or part of a captured payment. Use after a charge succeeds; for a pending authorization use void_authorization. Omit amount for a full refund. Returns refund_id and status. Safe to retry with the same idempotency_key.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "payment_id": { "type": "string", "description": "ID of the captured payment, e.g. pay_8Kd2" },
      "amount": { "type": "integer", "description": "Amount to refund in cents. Omit for a full refund." },
      "idempotency_key": { "type": "string", "description": "UUID. Reuse it to retry safely." }
    },
    "required": ["payment_id", "idempotency_key"]
  },
  "annotations": {
    "destructiveHint": false,
    "idempotentHint": true
  }
}

The paper scores descriptions on four tests. They make a good review checklist:

• Accuracy: Does it match what the tool does, including parameter meanings?
• Functionality: Does it say what the tool does and when to use it? This had the biggest effect in the study.
• Completeness: Are inputs, outputs and side effects covered?
• Conciseness: Is it short enough not to waste the agent's context? Two to four sentences is plenty.

4. Lint it in CI

The jq counts from the top are a start. This script checks the same things and a few more, and fails the build when it finds problems:

# lint_agent_docs.py: flag OpenAPI gaps that trip up AI agents
import json, re, sys
from collections import Counter
METHODS = {"get", "post", "put", "patch", "delete"}
GENERIC = re.compile(r"^(get|post|put|patch|delete)_|_\d+$")
spec = json.load(open(sys.argv[1]))
ops = [(path, m, op) for path, item in spec["paths"].items() for m, op in item.items() if m in METHODS]
ids = Counter(op.get("operationId") for _, _, op in ops)
problems = []
for path, method, op in ops:
    where = f"{method.upper()} {path}"
    op_id = op.get("operationId")
    desc = (op.get("description") or "").strip()
    codes = op.get("responses", {}).keys()
    if not op_id:
        problems.append(f"{where}: no operationId")
    elif GENERIC.search(op_id):
        problems.append(f"{where}: generic operationId '{op_id}'")
    elif ids[op_id] > 1:
        problems.append(f"{where}: duplicate operationId '{op_id}'")
    if len(desc) < 40:
        problems.append(f"{where}: description missing or under 40 chars")
    if not any(str(c).startswith("4") for c in codes):
        problems.append(f"{where}: no 4xx response documented")
for p in problems:
    print(p)
print(f"\n{len(problems)} problems in {len(ops)} operations")
sys.exit(1 if problems else 0)

Run against a small test spec, it prints:

POST /refunds: generic operationId 'post_refunds_1'
POST /refunds: description missing or under 40 chars
POST /refunds: no 4xx response documented
GET /refunds: duplicate operationId 'listRefunds'
GET /customers/{id}: no operationId
GET /customers/{id}: description missing or under 40 chars
GET /customers/{id}: no 4xx response documented
POST /v2/refunds: duplicate operationId 'listRefunds'
8 problems in 4 operations

The 40-character floor is a blunt rule, and that's on purpose. It catches "Get data" without arguing about style. For YAML specs, load with yaml.safe_load instead of json.load.

5. Test with an agent

Static checks catch missing fields. They don't tell you if an agent can do the job. For that:

• Pick five common tasks: For example, "refund the last charge for customer cus_123".
• Give the agent only your docs: Your llms.txt, the spec or your MCP server, plus a sandbox. No hints in the prompt.
• Log each call: Which tool it picked, what it sent and whether it worked.
• Fix the layer that failed: Wrong tool is a description problem, wrong parameters is a spec problem, and can't find anything is an llms.txt problem.

First-call success is the number to watch. If it takes three tries, there's a gap in your docs.

If you'd rather not keep three layers in sync by hand, some API platforms now generate them from the spec. DigitalAPI, for example, publishes an llms.txt index and turns OpenAPI operations into MCP tools with their schemas and examples.

Over to you

The fixes are boring, which is why they don't get done. What's the worst auto-generated operationId you've found in a spec you had to use?