The model that said no to my forced tool call

Every agent I run is a fixed set of steps written in code. Most steps are plain code: read rows, count things, compare two weeks. A few steps call a model. Those are the ones that need judgment: is this week's change meaningful, which companies does this answer name, what should the strategist tell the client.

The model steps all share one rule. They never answer in prose. The next step is code, and code can't read a paragraph. So every model step fills in a form: a boolean the next step branches on, a label from a closed list, a list it can loop over, a short text a person will read.

Since the first agent, the way I got that form was a trick.

The trick

The model APIs let you declare tools. You describe a tool with a JSON schema, and the model can "call" it by producing arguments that match. If you force the model to call one specific tool, the arguments are your structured answer. You never execute the tool. You just read what the model filled in.

json"tools": [{ "name": "weekly_judgement", "input_schema": { ...the form... } }],
"tool_choice": { "type": "tool", "name": "weekly_judgement" }

It worked across every model I used. One helper, structured(), wrapped it, and every step called that.

The day it stopped working

I wanted one step, the assistant that drafts a strategist's three lines for a client each week, on a newer, stronger model. I changed one string. The step failed with a 400:

tool_choice: type "tool" and "any" are not supported for this model.

The newer model simply doesn't accept a forced tool call. The trick that every step depended on was gone for the models I would want next.

The fix: ask for the form directly

The API now has a first-class way to say "answer in this JSON schema": a structured output format on the request itself. No fake tool, no forced call:

json"output_config": { "format": { "type": "json_schema", "schema": { ...the form... } } }

The reply's text is the JSON object. I tested it on four models, from the small fast one to the largest, and all four returned exactly the fields I asked for.

Two things needed care.

1. The schema has to be strict. The API wants every object closed (additionalProperties: false), and it refuses some keywords I had used for years, like maxItems on a list. So the helper now makes a strict copy before sending: it closes every object and takes the refused keywords out.

2. The limits still matter, so they moved into code. If my form said "at most four moves," that rule didn't disappear. It's applied to the reply after it comes back: the list is cut to four, the text to its length. That turned out to be the better split anyway. The model judges; code does the exact work. Code never miscounts.

pythondef strict_schema(schema):      # what the API accepts
    ...close every object, drop maxItems and friends...

def apply_limits(value, schema): # what the API dropped, enforced here
    ...cut lists to maxItems, text to maxLength...

The helper's signature didn't change, so none of the steps that call it changed either. One file moved to the new way, and every step came with it.

The surprise: the model thinks first

The first real run on the new model failed again, this time with "stopped early." I had given the step 600 output tokens, plenty for three short lines.

The raw reply (I keep every paid reply, more on that in another post) showed why. The model spent about 470 tokens thinking before it wrote a word of the answer. The three lines started arriving and hit the ceiling halfway through the third.

The fix was one number, 2,000 tokens. The lesson is bigger: newer models can spend output budget before they answer, so budget for the answer plus the thinking. A cut-off answer is a paid failure. The helper raises on it with the cost attached, so it shows up in the run record instead of disappearing.

Designing the form for the next step

The best idea I took from this came from how the form is designed, not from the API. Ask what the next step needs, not what the model is looking at:

• The next step makes a decision → give it a boolean.
• It does math → give it a number (and let code do the math).
• It routes → give it a label from a closed list.
• It loops → give it a list.
• A person reads it → give it short text.

My weekly drafter now returns three lines for the strategist and three routing fields for the machine: the kind of work (a closed list), which buyer question it's for, and which assistant it targets. Then code checks those fields against the week's data. The question must be one that was actually asked that week. The assistant must be one that actually answered. Anything else is blanked, never guessed.

That last part is the whole philosophy in one rule: the model proposes, code verifies, a person approves.

What it cost

A handful of test calls at a fraction of a cent each, one failed run at about two cents, and an evening. What I got back was every model available to every step, and a cleaner line between what the model decides and what code decides.