When Structured Output Ended Halfway

The GPT-5.6 path for generating roadmap topics failed three times for three different reasons.

The first failure came from the response schema. The second came from arrays that exceeded the application’s limits. I fixed both, and a focused probe passed, but the user replayed the real roadmap and generation failed again.

The third response followed the schema. It just did not reach the end.

Replacing the root array with a closed object

A Tail Villain roadmap starts from the learner’s Goal and Background. The model turns that context into topics, then adds interview angles, expected signals, follow-up prompts, and evaluation criteria. The backend validates that structured result before persisting it.

The first issue after moving this path to GPT-5.6 was the top-level response shape. The existing contract returned an array of topics directly. For OpenAI Structured Outputs, a closed top-level object was the safer boundary.

I wrapped the array in an object with a topics field, closed the nested topic and interview-module objects against undeclared properties, and added a request-level strict schema option to the provider-neutral text-generation contract.

That repaired the broad JSON shape, but application validation still failed. A captured response revealed arrays with six or seven elements where the application allowed no more than five. The JSON was syntactically valid, but it violated the product contract.

The strict setting was not reaching the actual OpenAI request. Supplying a schema does not automatically mean every constraint is enforced. Once strict Structured Outputs was connected to topic generation, blueprint materialization, and localization, the array-limit failures stopped.

Strict output can still be truncated

The focused service probe returned a valid response, so I treated the repair as complete. The user then retried the roadmap that had originally failed and reproduced the error.

This time, response length was the cause.

The real request included a longer Goal, Background, and roadmap analysis. GPT-5.6 generated more material and reached the configured ceiling of exactly 4,096 output tokens. After roughly 87 seconds, the response ended in the middle of the JSON document.

Strict Structured Outputs constrains the shape the model should generate. It does not let the model continue after the output budget is exhausted. A response can follow the schema until the token limit stops generation before the structure is complete.

Without checking completion status, all of these cases collapse into the same malformed JSON error. The application cannot distinguish a schema violation from output truncation or a valid JSON document that fails its own domain constraints.

Reducing normal output before raising the ceiling

Raising the token ceiling was the fastest available workaround, but it would have left normal generation unconstrained. If one roadmap routinely needs thousands of output tokens, latency and cost grow with it, and the same failure eventually returns at a higher limit.

I reduced the expected output first. Every generated interview module now contains exactly three concise interview angles, expected signals, follow-up prompts, and evaluation criteria. The prompt also tells the model to keep those entries specific and short.

The output ceiling moved from 4,096 to 8,192 tokens, but that capacity is safety headroom rather than a target. The schema and prompt keep normal responses compact. The higher limit protects an unusually long but otherwise valid response from ending halfway.

The OpenAI provider now checks whether a response is marked incomplete because of max_output_tokens. It reports that condition explicitly before attempting JSON parsing. Having an output_text value is not the same as having a completed response.

Replaying the failed request through the real boundary

The first two fixes survived focused checks and still failed in the product because the validation boundary was too narrow. A small service probe proved that the provider could accept the schema and return one conforming response. It did not prove the full path the user exercised.

The real flow includes authentication, controller routing, the complete roadmap context, LLM generation, application validation, and database persistence. I replayed the exact failed blueprint through the authenticated local HTTP endpoint, then checked both the response and the persisted topic and interview module.

The final request completed in about 4.9 seconds with 457 output tokens. The previous full request had consumed all 4,096 output tokens and failed after roughly 87 seconds. The largest improvement came from controlling generation size, not from granting a larger budget.

The completion criteria became explicit:

provider response completed
  → application schema validation passed
  → authenticated HTTP request succeeded
  → generated state was persisted

All four layers must pass before roadmap generation is complete. A direct provider or service probe remains useful for isolating a failure, but it cannot replace the application path.

The debugging boundary expanded with each failure. I first checked the JSON shape, then whether strict constraints reached the provider, and finally whether the response completed and survived the authenticated persistence path.

Structured output is not automatically complete output. The model can follow the schema until its output budget expires, and valid JSON can still violate application limits.

The roadmap path now checks more than whether the model returned text. It checks provider completion, domain validation, the user-facing HTTP path, and persisted state. That is the evidence required before I can call the generation successful.