# Your AI Feature Is Lying to You: Stop Using JSON Mode and Use Schema-Constrained Outputs

* * *

You shipped an AI feature last month. It returns product recommendations, or extracts invoice data, or summarizes tickets — and it mostly works. But every once in a while, a field shows up empty, a number comes back as a string, or two values silently swap places. The JSON is valid. Your schema check passes. And somewhere downstream, a user sees garbage, or your database quietly stores nonsense.

That's the JSON mode trap. And it's more common than anyone admits.

* * *

## The Three Tiers of Getting Structured Data Out of an LLM

When you're building AI features into a MERN app, there's a spectrum of how tightly you can constrain what the model gives back. Most tutorials stop at tier one.

![](https://cdn.hashnode.com/uploads/covers/69d007f5e466e2b7625cd1df/24066b58-0ccf-479c-a40e-26cdfdee3056.png align="center")

**Tier 1 — JSON Mode**

You tell the model to "respond with valid JSON". OpenAI's `response_format: { type: "json_object" }` falls here. It guarantees syntactically valid JSON — no markdown fences, no prose — but that's the end of the guarantee. The keys can be whatever the model feels like, values can be the wrong types, and required fields can just be missing. Validation is on you, and it happens *after* generation, meaning the model could generate something structurally broken that slips through a loose check.

**Tier 2 — Function Calling / Tool Use**

You define a function with parameters. The model calls it. This is meaningfully better — the model at least has a schema in front of it and tries to conform. But "tries to" is doing a lot of work there. Without strict enforcement, function calling is still a soft constraint. The model can choose not to call the function at all, or fill fields with plausible-but-wrong data.

**Tier 3 — Schema-Constrained Generation**

This is the one that actually solves the problem. You define a JSON schema, the provider runs *constrained decoding* under the hood — meaning the model literally cannot emit tokens that would produce invalid output. Structure is enforced at generation time, not validation time. No post-processing surprises.

OpenAI calls this `strict: true` in their structured outputs API. Anthropic enforces it through typed tool calls. Google Gemini has `responseSchema`. The implementation differs but the concept is the same: the schema is a hard wall, not a polite suggestion.

* * *

## Why JSON Mode Will Eventually Bite You

Here's a stripped-down version of the bug pattern that's bitten plenty of MERN apps:

```javascript
// express route — extracting invoice fields from pasted text
app.post('/api/extract-invoice', async (req, res) => {
  const { text } = req.body;

  const completion = await openai.chat.completions.create({
    model: 'gpt-4o',
    response_format: { type: 'json_object' },
    messages: [
      {
        role: 'system',
        content: 'Extract invoice data from the text. Respond with JSON.'
      },
      { role: 'user', content: text }
    ]
  });

  // This looks safe. It isn't.
  const data = JSON.parse(completion.choices[0].message.content);

  await Invoice.create(data); // 🙈
});
```

The JSON parses fine. But `data.totalAmount` might be `"1,250.00"` (a string with a comma) when you expected a number. Or `data.dueDate` might be `"next Friday"` in plain English instead of an ISO string. Or — the fun one — `data.vendorName` and `data.clientName` get swapped when the invoice is formatted in an unusual way.

None of these fail loudly. They all get stored, and you find out three weeks later when someone runs a report.

* * *

## The Fix: Schema-Constrained Generation with Zod + OpenAI

![](https://cdn.hashnode.com/uploads/covers/69d007f5e466e2b7625cd1df/23c8534b-6669-4dd4-98e6-99b0a22da335.png align="center")

The good news is the fix is genuinely clean if you're already in a TypeScript or modern JavaScript project. The pattern is: define your schema once with Zod, convert it to JSON Schema, pass it to the OpenAI structured outputs API, and get back something you can trust.

First, install what you need:

```bash
npm install openai zod zod-to-json-schema
```

Then define your schema and wire it up:

```javascript
import OpenAI from 'openai';
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';

const openai = new OpenAI();

// Define your schema with Zod — readable and reusable
const InvoiceSchema = z.object({
  vendorName: z.string().describe('The company or person sending the invoice'),
  clientName: z.string().describe('The company or person receiving the invoice'),
  invoiceNumber: z.string(),
  totalAmountCents: z
    .number()
    .int()
    .describe('Total amount in cents (integer), e.g. $12.50 → 1250'),
  dueDateISO: z
    .string()
    .describe('Due date in ISO 8601 format, e.g. 2026-11-01'),
  lineItems: z.array(
    z.object({
      description: z.string(),
      quantityCents: z.number().int(),
    })
  ),
});

export async function extractInvoice(text) {
  const response = await openai.chat.completions.create({
    model: 'gpt-4o-2024-08-06', // or your preferred model
    messages: [
      {
        role: 'system',
        content:
          'You are an invoice extraction assistant. Extract all invoice fields exactly as described.',
      },
      { role: 'user', content: text },
    ],
    response_format: {
      type: 'json_schema',
      json_schema: {
        name: 'invoice_extraction',
        strict: true,
        schema: zodToJsonSchema(InvoiceSchema),
      },
    },
  });

  const raw = JSON.parse(response.choices[0].message.content);

  // Still worth validating — catches model refusals that return partial data
  return InvoiceSchema.parse(raw);
}
```

A few things worth noting in this pattern:

*   **Storing amounts as cents (integers)** is a common trick. Floats and comma-formatted strings from LLMs are a recurring headache. An integer field forces the model to give you `1250` not `"12.50"` or `"$12.50"`.
    
*   **Field descriptions matter more than you'd think.** The model uses them to understand intent. `totalAmountCents` with a description is dramatically more reliable than `total`.
    
*   **Still call** `InvoiceSchema.parse(raw)` **after.** Schema-constrained generation is the hard guarantee, but Zod's parse gives you runtime-typed data and catches edge cases like model refusals that return `{"error": "insufficient information"}`.
    

* * *

## Wiring It Into Express Without Making a Mess

The extract function above is clean, but a common mistake is to scatter the schema definitions around your routes. A better approach is to keep schemas co-located with their service logic:

```plaintext
src/
  services/
    invoice/
      schema.js      ← Zod schema lives here
      extractor.js   ← extractInvoice() lives here
  routes/
    invoice.js       ← thin route, calls extractor
```

Your route stays thin:

```javascript
// routes/invoice.js
import { extractInvoice } from '../services/invoice/extractor.js';

router.post('/extract', async (req, res, next) => {
  try {
    const invoice = await extractInvoice(req.body.text);
    res.json({ success: true, data: invoice });
  } catch (err) {
    // ZodError means the model returned something structurally wrong
    if (err.name === 'ZodError') {
      return res.status(422).json({
        error: 'extraction_failed',
        details: err.errors,
      });
    }
    next(err);
  }
});
```

This separation means when OpenAI updates their API or you want to swap in a different model, you change one file. The route doesn't care how the data was extracted — only that it comes back in the right shape.

* * *

## Practical Takeaways

A few things to walk away with:

*   **Audit your existing AI routes.** If you're using `json_object` mode and downstream code assumes a specific shape, you have latent bugs. The model just hasn't hit the edge case that surfaces them yet.
    
*   **Descriptions are prompts, not decorations.** Write your Zod field descriptions like you're prompting the model, because functionally you are. Be specific about units, formats, and fallback behavior.
    
*   **Integer money values.** Always. Floating point and LLMs don't mix.
    
*   **Don't skip Zod parse after extraction.** Schema-constrained generation is very reliable but `strict: true` can still return a valid-but-empty fallback on refusals. Parse it.
    
*   **Think about MongoDB schema alignment.** Your Zod schema and your Mongoose model should describe the same shape. If they drift, you'll be debugging at 2am.
    

* * *

## Wrapping Up

JSON mode was fine when you were prototyping. In production, it's a liability. The migration to schema-constrained outputs is smaller than you'd expect — a few hours to add Zod schemas and update your `response_format` calls — and the reliability improvement is real enough that you'll notice it in logs within a week.

The pattern here works for more than invoice extraction. Product classification, content tagging, sentiment analysis, customer data enrichment — anything where you need a predictable shape out of an LLM. Get the schema right once and stop thinking about it.

If you've shipped something using this approach or have a pattern that works better for your stack, drop it in the comments. Always curious what's actually running in production versus what sounds good in a blog post.