fix(ai): enforce JSON output and stop failing on truncation (#72) #75

Merged

kreativmonkey merged 1 commit from fix/ai-json-generation into main

2026-06-13 14:39:34 +02:00

kreativmonkey commented

2026-06-13 13:51:53 +02:00

Owner

Problem (#72)

AI jobs intermittently ended with {"error": "LLM response was not valid JSON"}, often after a long (~55s) run.

Root cause

No enforced JSON mode. Neither the OpenAI nor the Anthropic client asked for structured output — the prompts only requested JSON, so the model sometimes wrapped it in prose/markdown.
Silent truncation. With a 1500-token budget (campaigns: 1000), long generations were cut off at the token limit and the partial — invalid — JSON was returned unchecked. OpenAI's finish_reason was ignored and Anthropic's stop_reason was only checked for refusal. The ~55s duration is the tell-tale sign of hitting the cap.
No retry on the single-shot paths (campaign, profile).

Changes

OpenAI: send response_format: {"type":"json_object"} for JSON jobs; map finish_reason == "length" to a new ErrResponseTruncated instead of returning broken content.
Anthropic (has no JSON mode): prefill the assistant turn with { to force a JSON object and stitch the brace back onto the reply; map stop_reason == "max_tokens" to ErrResponseTruncated.
Raise the default token budget 1500 → 2048 and route every JSON-parsed prompt (campaign, profile analysis/assistant, vibe preview, voice engine) through a new GenerateJSON helper.
Campaign autopilot retries on an unusable reply (re-prompting with the concrete defect) rather than failing the whole job on one noisy response.
Harden extractJSONObject with a balanced-brace scan so prose containing } no longer derails parsing.

Tests (added, TDD)

OpenAI sets response_format only for JSON requests; reports truncation.
Anthropic prefills { and stitches it back; reports truncation.
extractJSONObject survives surrounding prose with braces + nested objects.
Campaign recovers from one invalid response via retry.

go test ./... and go vet ./internal/ai/ pass.

Closes #72.

🤖 Generated with Claude Code

## Problem (#72) AI jobs intermittently ended with `{"error": "LLM response was not valid JSON"}`, often after a long (~55s) run. ## Root cause 1. **No enforced JSON mode.** Neither the OpenAI nor the Anthropic client asked for structured output — the prompts only *requested* JSON, so the model sometimes wrapped it in prose/markdown. 2. **Silent truncation.** With a 1500-token budget (campaigns: 1000), long generations were cut off at the token limit and the partial — invalid — JSON was returned unchecked. OpenAI's `finish_reason` was ignored and Anthropic's `stop_reason` was only checked for `refusal`. The ~55s duration is the tell-tale sign of hitting the cap. 3. **No retry** on the single-shot paths (campaign, profile). ## Changes - **OpenAI:** send `response_format: {"type":"json_object"}` for JSON jobs; map `finish_reason == "length"` to a new `ErrResponseTruncated` instead of returning broken content. - **Anthropic** (has no JSON mode): prefill the assistant turn with `{` to force a JSON object and stitch the brace back onto the reply; map `stop_reason == "max_tokens"` to `ErrResponseTruncated`. - Raise the default token budget **1500 → 2048** and route every JSON-parsed prompt (campaign, profile analysis/assistant, vibe preview, voice engine) through a new `GenerateJSON` helper. - **Campaign autopilot** retries on an unusable reply (re-prompting with the concrete defect) rather than failing the whole job on one noisy response. - Harden `extractJSONObject` with a balanced-brace scan so prose containing `}` no longer derails parsing. ## Tests (added, TDD) - OpenAI sets `response_format` only for JSON requests; reports truncation. - Anthropic prefills `{` and stitches it back; reports truncation. - `extractJSONObject` survives surrounding prose with braces + nested objects. - Campaign recovers from one invalid response via retry. `go test ./...` and `go vet ./internal/ai/` pass. Closes #72. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

kreativmonkey added 1 commit

2026-06-13 13:51:54 +02:00

fix(ai): enforce JSON output and stop failing on truncation

CI/CD Workflow / Test Backend (Go) (pull_request) Successful in 3m3s

Details

CI/CD Workflow / Test Frontend (Node.js) (pull_request) Successful in 1m23s

Details

CI/CD Workflow / Test Postgres Store (pull_request) Successful in 1m20s

Details

CI/CD Workflow / E2E (Playwright + Go) (pull_request) Successful in 6m41s

Details

CI/CD Workflow / Build and Publish Docker Image (pull_request) Has been skipped

Details

CI/CD Workflow / Trigger Dockhand Deployment (pull_request) Has been skipped

Details

634f58db69

AI jobs intermittently failed with "LLM response was not valid JSON"
(issue #72). Root causes were that nothing forced the model to emit JSON,
and that responses cut off at the token limit were returned silently as
broken JSON.

- OpenAI: request `response_format: {"type":"json_object"}` for JSON jobs
  and treat `finish_reason == "length"` as ErrResponseTruncated instead of
  returning the truncated body.
- Anthropic (no JSON mode): prefill the assistant turn with `{` to force a
  JSON object and stitch the brace back onto the reply; treat
  `stop_reason == "max_tokens"` as ErrResponseTruncated.
- Raise the default token budget 1500 -> 2048 and route every JSON-parsed
  prompt (campaign, profile analysis/assistant, vibe preview, voice engine)
  through the new GenerateJSON helper.
- Campaign autopilot now retries on a bad/invalid reply (re-prompting with
  the concrete defect) instead of failing the whole job on one noisy
  response.
- Harden extractJSONObject with a balanced-brace scan so prose containing
  braces no longer derails parsing.

Closes #72.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

kreativmonkey merged commit d3053819a0 into main

2026-06-13 14:39:34 +02:00

kreativmonkey referenced this pull request from a commit

2026-06-13 14:39:47 +02:00

Merge pull request 'fix(ai): enforce JSON output and stop failing on truncation (#72)' (#75) from fix/ai-json-generation into main

kreativmonkey referenced this pull request

2026-06-13 20:55:23 +02:00

fix(ai): adaptive per-model token budget on truncation #78

kreativmonkey referenced this pull request from a commit

2026-06-17 09:09:33 +02:00

Merge pull request 'fix(ai): enforce JSON output and stop failing on truncation (#72)' (#75) from fix/ai-json-generation into main