MCP in Production · Chapter 5
A user once told me my AWS MCP server was "broken." I pulled up the trace. The server was fine. Every tool worked. Every handler returned valid JSON. The problem was that the model had picked the wrong tool seven times in a row, and on the eighth attempt it gave up and apologized to the user.
I spent the next two days not touching a single handler. I rewrote tool descriptions. I renamed three parameters. I added .describe() calls to fields I had assumed were self-explanatory. I tightened two enums and added a .max() on a pagination field that had been silently returning 1,000-item arrays.
I shipped @yawlabs/aws-mcp 0.3 with zero behavioral changes. The server did exactly what 0.2 did. Same handlers. Same SDK calls. Same outputs.
Tool selection accuracy on my 200-prompt eval moved up by roughly 18 points -- from the low 70s into the low 90s.
That is what this chapter is about. Schemas and descriptions are not documentation. They are the contract you sign with the model on behalf of every user who will ever talk to your server. If the contract is sloppy, the server is sloppy, no matter how well your handlers work. The handler is where you write code. The schema is where you write the prompt.
The single biggest mental shift I see engineers struggle with when they move from REST APIs to MCP is this: in MCP, the consumer is not a developer reading your docs. The consumer is a model deciding, in real time, with no human in the loop, whether your tool is the right one to call right now.
That decision is made entirely from your tool name, your tool description, your parameter names, your .describe() strings, and the structure of your input schema. The model has no other context. It cannot click through to your README. It will not read the source. It does not know that entity_id "obviously" means the EC2 instance ID because that is what your team has called it for three years.
When I onboard new engineers to MCP work at Yaw Labs, I make them write their first tool description twice. The first version is whatever they would put in a JSDoc comment. The second version is what they would say to a junior engineer who has thirty seconds to decide whether to use this function or write their own. The second version is always better, always shorter, and always more specific about when to use the tool, not what the tool does.
From the field: I have a half-joking rule that any tool description containing the words "this tool" should be deleted and rewritten. Models do not need to be told they are looking at a tool. Spend those tokens on triggers and constraints instead.
The MCP SDK gives you a description field on every tool registration. Treat it like the system prompt of a tiny model. Every word counts. Every word competes for attention against the descriptions of fifty other tools the model is also weighing. That competition has a measurable token bill, too -- see the hidden cost of 200 MCP tools in context for what a large tool surface costs before the model makes a single call.
server.registerTool(
"list_running_ec2_instances",
{
description:
"List EC2 instances currently in the 'running' state for an AWS account. " +
"Use when the user asks about active servers, current EC2 usage, " +
"what is running right now, or wants a snapshot of live compute. " +
"Does not include stopped or terminated instances -- use list_ec2_instances " +
"with a state filter for those.",
inputSchema: {
region: z
.string()
.regex(/^[a-z]{2,3}(-[a-z]+)+-\d$/)
.describe("AWS region code, e.g. us-east-1, eu-west-2, us-gov-east-1"),
max_results: z
.number()
.int()
.min(1)
.max(500)
.default(50)
.describe("Maximum number of instances to return per page"),
},
},
handler,
);
Three things about that snippet matter and we will spend the rest of the chapter on them. First, the description tells the model when to call the tool, not just what it does. Second, every parameter has a .describe() even though the names look obvious. Third, every numeric field has bounds.
When I wrote the first version of list_running_ec2_instances, the description said something like "Returns a list of running EC2 instances in the specified region." That is what every backend engineer instinctively writes. It describes the function. It does not describe the trigger.
Models pick tools by matching the user's request to a tool description. If the user says "what's running in our prod account right now," the model has to scan your description and decide: does this match? "Returns a list" matches "give me a list," sure. But "running" is doing all the work, and a model under load will sometimes prefer describe_ec2_state or get_compute_inventory because their descriptions happen to mention the word "running" in a more salient position.
The trigger-phrase pattern is what fixed this for me. Every tool description in @yawlabs/aws-mcp now follows the same structure:
That last part is underrated. When you tell the model "this tool does not handle X, use Y for that," you are simultaneously increasing the precision of this tool and the recall of the other tool. You are training a tiny classifier with one sentence.
server.registerTool(
"deprecate_npm_package",
{
description:
"Mark a published npm package version (or all versions) as deprecated " +
"with a message visible to anyone who installs it. " +
"Use when the user says 'deprecate', 'mark as deprecated', " +
"'add a deprecation notice', or 'tell users to migrate off this'. " +
"Does not unpublish or delete -- use unpublish_npm_package for removal " +
"(and warn the user about the 72-hour rule).",
inputSchema: {
package: z.string().describe("Full package name including scope, e.g. @yawlabs/old-thing"),
version_range: z
.string()
.default("*")
.describe("Semver range to deprecate. Use '*' for all versions, '<2.0.0' for old majors, etc."),
message: z
.string()
.min(1)
.max(120)
.describe(
"Short message shown to installers. Format: 'renamed to @scope/newpkg -- install that instead'. " +
"Use lowercase after the dash and skip the trailing period.",
),
},
},
handler,
);
That description is from @yawlabs/npmjs-mcp. The message field's .describe() does double duty: it documents the parameter and it teaches the model the formatting convention I learned the hard way (period-capital deprecation messages have 422'd; em-dash lowercase has shipped successfully). The schema is encoding institutional knowledge that would otherwise live in a wiki nobody reads.
Pitfall: If your tool description was copy-pasted from your internal API documentation, it is wrong. API docs are written for engineers who have already decided to use the API. Tool descriptions are written for a model that has not yet decided. The audiences are different. The text should be too.
There are three rules I follow for tool names and I have not been wrong yet to follow them.
Snake case, always. list_running_ec2_instances, not listRunningEc2Instances or list-running-ec2-instances. Models trained on tool-use traces have seen far more snake_case tool names than any other style, and the tokenizer treats underscored phrases as single semantic units more reliably than camelCase. This is empirical. I have seen tool selection accuracy degrade two to four points just from switching a server's tool naming convention from snake_case to camelCase. It is not a huge effect, but it is free, so why pay it.
Verb phrases, not noun phrases. create_repository, not repository_creation. list_secrets, not secrets_list. The tool's name should sound like an action a user would describe. When a user says "create a new repo for me," the model is looking for a verb that matches. Make it easy.
Be deliberate about namespace prefixes. The default I'd push you toward is no prefix: the MCP SDK already handles namespacing, tools are scoped to the server they belong to, and clients see the server name in the tool registry. Putting the prefix in the tool name itself is double-namespacing, and github_create_repository is just create_repository with a redundant prefix that wastes tokens in every prompt and adds nothing to the model's selection signal. That's the rule I'd give a team starting fresh.
The honest caveat is that the @yawlabs servers I currently ship have all converged the other way -- tailscale_*, npm_*, aws_*, ls_*. The reason isn't that I changed my mind on the principle; it's that once a user has three or four MCP servers connected at the same time, having delete_* mean three different things in three different namespaces stops being theoretical. A consistent per-server prefix turns out to be cheap insurance against cross-server name collisions in registries the user assembles, even if each prefix is technically redundant inside its own server. So: prefer no prefix when you can; if you do prefix, prefix per-server consistently rather than per-feature, and accept the modest token cost in exchange for collision-proof clarity in the multi-server registries your users will actually run. Chapter 11 walks through the @yawlabs servers one by one and shows the prefix decision in context -- if you want the receipts on why production drifted from the rule above, that's where to look.
The narrower rule that holds either way: when you genuinely have multiple verbs that share a noun and need disambiguation within a single server, the noun-prefix is fine. If your server has create_repo, create_user, and create_team, those are fine. The verb is the same; the noun is what disambiguates. That is structural, not namespacing.
From the field: When I migrated @yawlabs/electron-mcp from
electron_*prefixed names to bare verb phrases in 0.4, I expected nothing to change because the prefixes seemed redundant but harmless. Tool selection on my eval went up 3.5 points. The model had been confusingelectron_window_createwithelectron_create_window(both existed at one point during a refactor). Removing the prefix forced clarity. The collision risk that pushed me back to per-server prefixes on later servers was a different problem -- across servers, not within one -- so the two lessons coexist rather than contradict.
Parameter names are where I see the most preventable failures. Engineers name parameters for the API they are wrapping, not for the user who is going to talk to the model. If the AWS SDK calls it InstanceId, the engineer calls it instance_id. That is fine for AWS power users. It is not fine for "the intern who just joined and was told to find the prod database server."
I have a short list of parameter-naming rules I run through every time I add a new tool:
repo beats repository_full_name even though the latter is more specific. The model can handle the slight ambiguity; the user cannot remember the precise field name.vpc_id is fine because everyone in the AWS world says VPC. rds_cd for "RDS cluster discriminator" is not fine because no human says that.data, payload, options, or config unless you genuinely have a heterogeneous blob. These names tell the model nothing. If you have a config blob, the model has no idea what shape it should be, and you have just shifted the burden from your schema to the model's imagination.target_cluster_id, the parameter name should be environment and your handler can do the mapping.// Bad: faithful to the AWS SDK, hostile to users.
inputSchema: {
DBClusterIdentifier: z.string(),
Engine: z.string(),
MasterUsername: z.string(),
AllocatedStorage: z.number(),
}
// Good: faithful to how a user would describe the request.
inputSchema: {
cluster_name: z.string().describe("Name for the new RDS cluster, alphanumeric and dashes"),
engine: z
.enum(["postgres", "mysql", "aurora-postgres", "aurora-mysql"])
.describe("Database engine"),
admin_username: z.string().min(1).max(63).describe("Master username for the cluster"),
storage_gb: z
.number()
.int()
.min(20)
.max(65536)
.describe("Allocated storage in gigabytes"),
}
The handler does the SDK translation. The schema speaks the user's language.
A surprising amount of model failure comes down to parameter cardinality decisions. When is a parameter required? When is it optional? When does it have a default?
My rule of thumb, refined over a couple dozen production servers:
Required when there is no sensible default and the operation cannot proceed without it. A git clone tool requires a URL. There is no default URL.
Optional with a default when there is a sensible default that the user would specify 80% of the time anyway. max_results defaults to 50. If you make this required, you force the model to invent a number, and the model invents whatever it last saw, which on a long enough timeline is going to be 1000 and crash your handler.
Optional without a default when the parameter narrows the operation but its absence has a clear meaning. region on an AWS tool: if absent, use the user's configured default region. If present, override.
The trap is making things "optional" when they are actually "implicitly required because the operation makes no sense without them." I see this with filtering parameters all the time. A search_repositories tool with an optional query parameter is a tool that, if called without a query, will return everything in the universe. That is not what the user wants. Either make query required, or set a default that constrains the search to something useful (like the user's own repos).
// Trap: query is "optional" but the tool is useless without it.
inputSchema: {
query: z.string().optional(),
language: z.string().optional(),
limit: z.number().optional(),
}
// Better: query is required, language and limit have safe defaults.
inputSchema: {
query: z.string().min(1).describe("Search query, e.g. 'tailwind react' or 'org:yawlabs'"),
language: z
.string()
.optional()
.describe("Filter by programming language, e.g. 'typescript', 'python'"),
limit: z
.number()
.int()
.min(1)
.max(100)
.default(20)
.describe("Number of results to return"),
}
The default value is also a prompt. When the model sees default(20), it learns the expected order of magnitude for results. If you defaulted to 1000, the model would assume your tool was for bulk listing and would call it less often for "show me a few examples" requests. The default trains the model on usage shape.
The single highest-leverage schema decision you make on most tools is whether a string parameter should be an enum or a free-form string. Get this right and the model gets it right every time. Get this wrong and you spend the rest of your life writing handler-level validation.
My rule: enum if the set of valid values is fewer than 20 and stable. Free-form string if the set is unbounded, frequently growing, or known only at runtime.
Database engines: enum. There are a fixed number, they don't change overnight, the model needs to pick one. AWS regions: enum, even though there are 30+ - because the model otherwise hallucinates regions like us-mid-3 (does not exist) or eu-london-1 (the real one is eu-west-2). Tag values: free-form. There are infinite tag values, they are user-generated, no model knows what tags your team uses.
// Right call: small, stable set.
log_level: z.enum(["debug", "info", "warn", "error"]).describe("Log level filter"),
// Right call: medium-sized but stable, prevents hallucination.
region: z
.enum([
"us-east-1",
"us-east-2",
"us-west-1",
"us-west-2",
"eu-west-1",
"eu-west-2",
"eu-central-1",
"ap-northeast-1",
"ap-southeast-1",
"ap-southeast-2",
"ap-south-1",
// ... and so on
])
.describe("AWS region code"),
// Right call: free-form, unbounded user input.
commit_message: z.string().min(1).describe("Commit message, supports newlines"),
Where I see this go wrong is the middle case: a set of 50-200 values that feels enumerable. CSS named colors. Country codes. ICAO airport identifiers. The temptation is to enum them. Don't. The schema becomes unreadable, the model's context bloats, and any addition forces a release. Use a free-form string with a .regex() if you need format validation, and document the canonical set in .describe() if it helps.
Pitfall: When you enum a set, you commit to maintaining it. If AWS adds
il-central-1and your enum doesn't have it, your tool is silently broken until you ship. Build a small script to regenerate enums from upstream sources, or at least know which enums you'll need to refresh on each release.
Every numeric parameter on every server I run has bounds. If I see a z.number() with no .min(), no .max(), and no .int(), that is a bug in code review. There are no exceptions and the cost is zero.
Here is why each one matters.
.int() prevents the model from sending 5.0 when you wanted 5. This sounds trivial; it is not. JavaScript's loose number type means your handler can silently accept 5.0, do an array slice, and return five items, and you will never notice the bug until someone writes 5.7 and gets a runtime error inside Array.prototype.slice somewhere. Make integers integers.
.min() prevents the model from sending 0 or -1 when you wanted "at least one." Pagination tools without .min(1) on limit are responsible for an embarrassing percentage of empty-response bugs in the wild. The model thinks 0 means "default," your handler thinks it means zero, your user thinks the tool is broken.
.max() prevents the model from sending 999999 when you wanted "a reasonable page size." This is the single most important bound, because the failure mode is not a polite empty response - it is an OOM, a downstream rate limit, a 30-second timeout that propagates back through the conversation and burns the user's patience. Pick a real cap. 100 for most things, 500 for power users, 1000 only if you have tested it.
// Wrong: every numeric field is a bug waiting to happen.
inputSchema: {
limit: z.number(),
offset: z.number(),
retries: z.number(),
}
// Right: bounds, types, defaults, descriptions.
inputSchema: {
limit: z
.number()
.int()
.min(1)
.max(200)
.default(50)
.describe("Number of results per page"),
offset: z
.number()
.int()
.min(0)
.max(10000)
.default(0)
.describe("Number of results to skip for pagination"),
retries: z
.number()
.int()
.min(0)
.max(5)
.default(2)
.describe("Number of retry attempts on transient failures"),
}
For pagination specifically, I default to cursor-based instead of offset-based whenever the upstream API supports it - offsets at high values silently return inconsistent results when the underlying data changes between calls. But that is a chapter 6 topic. For now, just bound your numbers.
I have lost count of how many times I have looked at a tool that was failing in evals and realized half its parameters had no .describe(). Engineers see a parameter named region and think "obviously this is a region, what is there to describe?" Then they ship and the model sends "USA" because it does not know which region taxonomy you mean.
.describe() is not documentation. It is part of the prompt the model reads to fill in arguments. A parameter without a description is a parameter the model is guessing about.
The minimum acceptable .describe() answers two questions: what format does this take, and what does a typical valid value look like? Both. Always.
// Insufficient.
region: z.string(),
// Better.
region: z.string().describe("AWS region"),
// Right.
region: z.string().describe("AWS region code, e.g. us-east-1, eu-west-2"),
// Sometimes necessary.
region: z
.string()
.regex(/^[a-z]{2,3}(-[a-z]+)+-\d$/)
.describe(
"AWS region code, lowercase with dashes. " +
"Examples: us-east-1, eu-west-2, ap-southeast-1, us-gov-east-1. " +
"Use the user's default region from AWS_DEFAULT_REGION if they don't specify one.",
),
The third version is right because it gives the model both a regex (for format) and concrete examples (for fluency). The fourth version is necessary for parameters where the model needs guidance on how to fill in a missing value. That last sentence about AWS_DEFAULT_REGION is doing real work - it tells the model what to do when the user does not name a region, so the model does not hallucinate or refuse.
From the field: I once had a
pathparameter on a file-system MCP server with no.describe(). The model started passing relative paths like./foo, then absolute paths like/Users/jeff/foo, then half-formed Windows paths likeC:foo(no separator). I added.describe("Absolute filesystem path. On Windows, use forward slashes: C:/Users/..."). The errors stopped that day.
The hardest schema design in MCP is not designing tools in isolation; it is designing tools whose outputs become inputs to other tools. This is where most servers degrade as they grow.
The pattern looks like: list_repos returns repos, get_repo takes a repo identifier, create_branch takes a repo identifier and a branch name, and so on. The model is going to chain these together. Either the output of list_repos makes that easy, or it does not.
Make it easy. The output of any "list" or "search" tool should include, in a stable position, the exact identifier that the corresponding "get" tools will accept. If get_repo takes a full_name string in owner/repo format, then list_repos should return objects with a full_name field. Not name plus owner separately. Not id (an internal numeric ID that no other tool accepts). The exact same field name with the exact same shape.
// Bad: list returns one shape, get takes a different shape.
list_repos -> [{ owner: "yawlabs", name: "aws-mcp", id: 12345 }]
get_repo({ full_name: "yawlabs/aws-mcp" })
// Now the model has to know that full_name is owner + "/" + name.
// Sometimes it figures it out. Sometimes it sends "12345".
// Good: list returns the shape get expects.
list_repos -> [{ full_name: "yawlabs/aws-mcp", description: "...", default_branch: "main" }]
get_repo({ full_name: "yawlabs/aws-mcp" })
// The model just copies the field across. No transformation needed.
This is one of those rules that seems obvious until you have inherited a server where it was not followed. Then you discover that 30% of your tool failures are the model trying to glue together outputs and inputs that were not designed to fit. Fixing it requires either a schema migration on the consuming tool or a transform layer in the producing tool. Neither is fun.
The corollary is that you should use the same identifier shape across every tool that handles the same noun. If repo is identified by owner/name in get_repo, then clone_repo, archive_repo, delete_repo, list_repo_branches, and every other repo-related tool should all take owner/name, not numeric ID, not just name, not anything else. Pick one shape. Stick with it.
There is a recurring debate about whether you should have one big tool with 12 parameters or three specialized tools with three parameters each. The honest answer is "it depends," but I have a heuristic I trust.
If you can describe the tool's behavior in a single English sentence without using the word "or," you have a thin tool and you should keep it thin. If you find yourself writing "this tool does X or Y or Z depending on which parameters are set," you have a fat tool and you should split it.
// Fat tool that should be split.
search_things({
query: string,
type: "repo" | "user" | "issue",
in_org: string?,
language: string?,
is_archived: boolean?,
has_wiki: boolean?,
// ... and on and on
})
// Three thin tools, each cleanly describable.
search_repositories({ query, language?, in_org? })
search_users({ query, in_org? })
search_issues({ query, repo?, state? })
The fat-tool version looks tidier in code. It is harder for the model to use. The model has to figure out which parameters are valid for which type value, what happens when you pass a language filter to a user search (probably nothing? maybe an error?), and the description has to enumerate every behavioral branch. The thin-tool version is three descriptions, each unambiguous.
That said, thin can go too far. If you have list_running_instances, list_stopped_instances, list_terminated_instances, and list_all_instances, you have over-fragmented. One list_instances tool with a state enum is right because the operation is the same; only the filter changes.
The test I use: if two tools have the same handler logic with one branch on a parameter, merge them. If two tools have substantially different handler logic with shared input shape, split them.
From the field: @yawlabs/aws-mcp had a
query_awsmega-tool in 0.1 that took aserviceparameter and dispatched to whatever service handler was relevant. It was elegant in code and a disaster in production. The model kept callingquery_aws({service: "ec2", action: "list", ...})with parameters from the wrong service, and the eval was scoring in the low 40s. Splitting it into per-service-per-action tools (list_ec2_instances,describe_rds_cluster, etc.) lifted accuracy into the low 70s in 0.2 with no other changes. That was the baseline I rewrote descriptions against to get to the low 90s in 0.3.
MCP gives you several ways to return data from a tool. The big three are text content (a string the model reads as prose), JSON-shaped content (a string the model parses as data), and structuredContent (a typed object the model receives alongside the human-facing text). Picking the right one matters more than most people realize, because the output shape decides how easily the next tool call composes off your result.
My rough guide:
Text content when the user's likely next action is to read the result, not act on it. "What did the deploy log say?" returns text. "Summarize this repo" returns text. The model relays it more or less verbatim.
JSON content when the user's likely next action is to act on a specific field, but the model is smart enough to parse it inline. "List my repos and pick the one with the most stars" returns JSON because the model is going to read the array, find the max, and continue. JSON is also good when the result is large and structured, because the model is better at extracting fields from JSON than from prose.
structuredContent when you need machine-readable data and want the model to know you intend it to be machine-readable. The MCP SDK in 1.x supports structuredContent alongside content, and clients can consume it programmatically rather than relying on the model to parse prose. Use it for programmatic outputs that will feed into later tool calls. It is also what most third-party compliance suites grade for on tools whose outputs are clearly typed.
return {
content: [
{
type: "text",
text: `Found ${instances.length} running instances in ${region}.`,
},
],
structuredContent: {
instances: instances.map((i) => ({
instance_id: i.InstanceId,
instance_type: i.InstanceType,
launch_time: i.LaunchTime?.toISOString(),
private_ip: i.PrivateIpAddress,
tags: i.Tags?.reduce((acc, t) => ({ ...acc, [t.Key]: t.Value }), {}),
})),
region,
count: instances.length,
},
};
That shape gives the model a human-readable summary in content and a clean object in structuredContent. The next tool call, say terminate_instance({ instance_id: ... }), can pull the field directly from structuredContent.instances[N].instance_id without round-tripping through prose.
The anti-pattern is mixing modes - returning a giant blob of pretty-printed JSON inside a text content block. The model has to parse it, the prose summary is buried inside the data, and you have lost the ability to send a clean structured payload. Pick one mode per logical output and commit to it.
I keep a running list of schema anti-patterns I have either committed myself or had to clean up after. Here are the ones I find most often when I review other teams' MCP servers.
Descriptions copy-pasted from internal API docs. The dead giveaway is a description that uses words like "endpoint," "request," "response," or "POST." Models do not know what a POST is in this context; they know what a user wants. Rewrite for triggers.
Missing .describe() on parameters. Every parameter without a description is a roll of the dice. The model fills in something plausible. Sometimes plausible is right. Often it is not.
Unbounded numeric inputs. z.number() with no .min() or .max() is a denial-of-service waiting to happen. Even if your handler defends against it, the model should not be guessing whether 1, 100, or 10,000 is "normal."
type: any or untyped object inputs. I see this most with "config" or "options" blob parameters. The model has zero guidance on what fields exist, and the tool becomes a guessing game. If you genuinely need a heterogeneous blob, define a discriminated union with z.discriminatedUnion() instead. If you cannot enumerate the fields, you have not finished designing the tool.
Single-character parameter names. q instead of query. n instead of limit. The token savings are negligible; the comprehension cost is real.
Internal IDs leaked into user-facing schemas. If your tool accepts a cluster_id: 47823, you have leaked your database primary key into the model's context. The model has no way to derive 47823 from anything the user said. Take a name or a slug.
"Stringly-typed" enums. z.string() where the description says "must be one of: foo, bar, baz." Make it z.enum(["foo", "bar", "baz"]). The schema is the contract; the description is just supplementary.
Optional parameters that change the meaning of other parameters. "If mode is bulk, then target is a glob; if mode is single, then target is a path." Use a discriminated union or split into two tools. Conditional semantics in flat schemas are a model-confusion factory.
Default values that are sentinels. limit: -1 to mean "no limit." name: "*" to mean "all." Models read defaults as examples of typical usage. If your default is -1, the model thinks -1 is a normal value. Use Infinity, undefined, or split the tool.
Pitfall: I once shipped a tool with
max_age_daysdefaulting to0because zero meant "no filter." Within a week, the model started passingmax_age_days: 0on every call regardless of user intent because it had learned0was the typical value. I changed the default to30(a real, useful value) and made0invalid. Filtering accuracy on the next eval moved up sharply -- a double-digit jump on a single one-line change.
I want to close this chapter with the rewrite I led off with, in detail, because it is the cleanest example I have of how much description craft matters relative to handler quality.
The setup: @yawlabs/aws-mcp 0.2 had 38 tools across EC2, S3, RDS, IAM, Lambda, and CloudWatch. Handlers were in good shape - well-tested, properly typed, talking to the AWS SDK with retries and pagination. My eval suite was 200 user prompts, hand-curated from real Claude Code transcripts, each labeled with the correct tool to call. I scored on tool selection only - did the model pick the right tool out of the 38 available? - because that was the failure mode I was seeing in the wild.
0.2 scored in the low 70s. That is not bad in absolute terms, but it meant roughly a quarter of the prompts went to the wrong tool, and I was getting bug reports that traced to tool confusion.
I rewrote every description over a weekend. No handler changes. No new tools. No removed tools. Same 38 tools, same parameters, same outputs. Just descriptions and .describe() strings.
Here are the patterns I applied, ranked by how much each one moved the needle:
1. Adding "use when" trigger phrases to every description. This was the biggest single win. The 0.2 descriptions said what the tool did; the 0.3 descriptions said what the user might say to make you call it. Around nine points on the eval.
2. Adding "does not do" disambiguation pointers. Every tool that had a sibling tool got a sentence like "does not handle X, use Y for that." This eliminated about half the cross-tool confusion errors. Around six points.
3. Tightening enums on region, engine, state, and service. 0.2 had these as free-form strings with .describe() documenting valid values. The model occasionally hallucinated. Switching to z.enum() killed the hallucination class entirely. About a point and a half.
4. Adding .describe() to every previously-undescribed parameter. I had assumed instance_id, cluster_name, and bucket_name were self-explanatory. They were not. About a point.
5. Renaming three parameters for user-language alignment. DBClusterIdentifier to cluster_name, BucketName to bucket, FunctionArn to function_name (with a regex to accept either form). About half a point.
6. Numeric bounds on every paginating tool. This caught zero eval failures (the eval did not test pagination edge cases) but eliminated a bug class in production. Worth doing anyway.
The total landed in the low 90s -- roughly an 18-point gain from description craft alone. The bulk of that gain came from items 1 and 2 together: the trigger-phrase pattern and the disambiguation pointers. If you take one thing away from this chapter, take that.
The kicker: the rewrite took me a weekend. The prior month of trying to improve eval scores by tweaking handlers, adding retries, and "improving robustness" had moved the score by less than a point. Schema work is the highest-leverage time you spend on an MCP server, and most teams under-invest in it because it does not feel like "real engineering." It is.
If you maintain an MCP server in production, here is what I would do this week, in order:
z.number() without .min() or .max(). Fix all of them..describe(). Fix all of them.Those five steps will pay back more than any handler optimization you do this quarter. I have not seen an MCP server get worse from doing this work, and I have seen many get dramatically better.
In chapter 6 we move from individual schemas to how schemas compose - what happens when the output of one tool needs to slot into the input of the next, and the patterns that keep the model from losing its place between calls. Chapter 6 also picks up the protocol-level safety hints (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) -- annotations are part of the same contract this chapter has been about, but they earn their keep most clearly once you are reasoning about reads versus writes. Chapter 7 then takes on the handler layer proper: error messages that the model can act on, retry budgets, and the conventions that separate a server you trust in production from one that works in demos. The schemas you wrote in this chapter are the contract. The composition rules and the handlers are how you keep it.