How to Get JSON Output from Claude, GPT-4o & Gemini (2026 Guide)

Why JSON from LLMs Breaks

Language models generate text token by token. They have no inherent concept of balanced brackets, valid escape sequences, or JSON grammar. Without constraints, a model optimized for human-sounding text will happily produce:

• Trailing commas (,}) copied from JavaScript code in training data
• Single-quoted strings ('value') valid in Python/JS but not JSON
• Bare undefined instead of null
• Comments (// ...) valid in JSONC but not JSON
• Truncated output when approaching the context limit
• Markdown fences (```json) wrapping the output

Modern LLMs have native structured output modes that solve most of these. Here is how to use each one.

Claude (Anthropic) — Structured Output via Tool Use

Anthropic does not have a simple json_mode flag. Instead, the correct approach is to use tool use (function calling). You define a JSON schema as a tool, and Claude is constrained to always call it — guaranteeing schema-valid output.

Python (Anthropic SDK)

import anthropic, json

client = anthropic.Anthropic()

# Define your output schema as a tool
tools = [{
    "name": "extract_data",
    "description": "Extract structured data from the text",
    "input_schema": {
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "age":  {"type": "integer"},
            "email": {"type": "string", "format": "email"}
        },
        "required": ["name", "age"]
    }
}]

message = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "tool", "name": "extract_data"},  # Force tool call
    messages=[{"role": "user", "content": "Extract: John Doe, 34, john@example.com"}]
)

# Result is always valid JSON matching your schema
result = message.content[0].input
print(json.dumps(result, indent=2))

Prompt-only approach (simpler, less reliable)

If you cannot use tool use, instruct Claude explicitly and strip markdown fences from output:

system = """You are a data extraction assistant.
Always respond with valid JSON only.
Do not include markdown code fences, comments, or trailing commas.
If a field is missing, use null."""

# Strip fences in your code as a safety net
import re
def extract_json(text):
    text = re.sub(r'^```(?:json)?
?', '', text.strip())
    text = re.sub(r'
?```$', '', text)
    return json.loads(text)

GPT-4o (OpenAI) — JSON Mode and Structured Outputs

OpenAI offers two levels: basic JSON mode (guarantees valid JSON, no schema) and Structured Outputs (schema-enforced, available on gpt-4o-2024-08-06 and later).

JSON Mode (any recent model)

from openai import OpenAI
import json

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o",
    response_format={"type": "json_object"},  # Guarantees valid JSON
    messages=[
        {"role": "system", "content": "Respond with JSON only."},
        {"role": "user",   "content": "Return user data: Alice, 28, designer"}
    ]
)

data = json.loads(response.choices[0].message.content)
print(data)

Structured Outputs (schema-enforced)

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int
    role: str

response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    response_format=User,          # Pydantic model enforces schema
    messages=[
        {"role": "user", "content": "Extract: Bob, 35, engineer"}
    ]
)

user = response.choices[0].message.parsed
print(user.name, user.age)   # Type-safe access

Gemini (Google) — response_schema

Gemini supports schema-constrained output via the response_schema parameter combined with response_mime_type: "application/json".

import google.generativeai as genai
import json

genai.configure(api_key="YOUR_API_KEY")

schema = {
    "type": "object",
    "properties": {
        "name":  {"type": "string"},
        "score": {"type": "number"},
        "tags":  {"type": "array", "items": {"type": "string"}}
    },
    "required": ["name", "score"]
}

model = genai.GenerativeModel("gemini-1.5-pro")
response = model.generate_content(
    "Rate this product: Fast, reliable, great battery",
    generation_config={
        "response_mime_type": "application/json",
        "response_schema": schema
    }
)

data = json.loads(response.text)
print(data)

Validating and Debugging LLM JSON Output

Even with native JSON modes, production code should validate output before using it. The most common issues and how to fix them:

Quick browser-based validation

Paste any LLM JSON response into the ToolPry JSON Formatter. It automatically detects the error type — trailing commas, HTML responses, undefined values, single-quoted strings — and offers a one-click auto-fix. Useful for debugging prompts during development.

Python: validate with Pydantic

from pydantic import BaseModel, ValidationError

class ProductReview(BaseModel):
    product: str
    rating: float  # must be a number
    pros: list[str]
    cons: list[str]

raw = llm_response  # whatever the model returned
try:
    data = json.loads(raw)
    review = ProductReview(**data)
    print(review.rating)  # type-safe
except json.JSONDecodeError as e:
    print(f"Invalid JSON: {e}")
except ValidationError as e:
    print(f"Schema mismatch: {e}")

JavaScript: validate with Zod

import { z } from "zod";

const ReviewSchema = z.object({
  product: z.string(),
  rating:  z.number().min(0).max(5),
  pros:    z.array(z.string()),
  cons:    z.array(z.string()),
});

const raw = llmResponse; // string from LLM
const parsed = JSON.parse(raw);
const result = ReviewSchema.safeParse(parsed);

if (result.success) {
  console.log(result.data.rating); // typed
} else {
  console.error(result.error.issues); // exact field errors
}

Best Practices

• Always use native JSON mode or tool use when available — prompt engineering alone is not reliable enough for production.
• Keep schemas flat and simple. Deeply nested schemas increase the chance of truncation and hallucinated fields.
• Validate, do not trust. Even schema-constrained output can have wrong values — a rating of 99.9 is valid JSON but invalid for a 0–5 scale. Use Pydantic/Zod range validators.
• Handle truncation. Long outputs near the context limit may produce incomplete JSON. Check for token limit warnings in the API response and increase max_tokens if needed.
• Strip markdown in fallback code. Even when using JSON mode, add a fence-stripping step as a safety net for unexpected model updates.
• Log raw responses during development. The raw string before JSON.parse() is the most useful debugging artifact when things break.

Frequently Asked Questions

What is Claude JSON mode / structured output?

Claude (Anthropic) supports structured JSON output via tool use (function calling). You define a JSON schema as a tool definition and Claude will always return data matching that schema. This is more reliable than prompting for JSON because the output is schema-validated by the API, not just text that looks like JSON.

What is GPT-4o JSON mode?

GPT-4o JSON mode is enabled by setting response_format: { type: "json_object" } in the API call. This guarantees the response is valid JSON, but does not enforce a specific schema. For schema enforcement, use Structured Outputs with a JSON Schema definition.

How do I validate JSON output from an LLM?

Paste the LLM output into the ToolPry JSON Formatter. It highlights syntax errors and shows exactly where the JSON is malformed. Common issues are trailing commas, missing quotes, and bare undefined values — all of which the formatter can auto-fix. In production code, use Pydantic (Python) or Zod (JavaScript) for schema validation.

Why does my LLM return invalid JSON even in JSON mode?

Even with JSON mode enabled, models can produce invalid JSON when the context window is too long and the model truncates mid-response, or when the prompt is ambiguous. Always wrap API calls in a try-catch and use schema validation on the output.