Can Jev do structured outputs?

Results

Time by schema level

Median end-to-end time. Overall: Jev 418 ms, Luna 1.69 s, Mixed 1.96 s (16% slower than Luna).

Cost by schema level

Estimated API cost per 1000 objects, with each run’s actual cache discount. Overall: Luna $0.17, Mixed $0.30, all-Jev $0.44.

The valid but wrong workflow

workflow-4 asked for a wait, then an email. All-Jev and Mixed both picked the notification branch twice. The schema allows two notifications, and Luna only fills the shape Jev picked, so it could not fix the choice.

{
  "project": "cedar-web",
  "workflow": {
    "enabled": true,
    "steps": [
      {
        "kind": "notify",
        "delivery": {
          "channel": "email",
          "to": "noah@example.test",
          "subject": "Your invoice",
          "html": false
        }
      },
      {
        "kind": "notify",
        "delivery": {
          "channel": "email",
          "to": "noah@example.test",
          "subject": "Your invoice",
          "html": false
        }
      }
    ],
    "approvals": [],
    "policy": {
      "failure": "stop",
      "limits": {
        "attempts": 1,
        "timeout": 300
      }
    }
  }
}

How Mixed splits a schema

Mixed sorts every schema node into fixed values, closed choices, and open slots. Code fills object keys, constants, single-value enums, fixed array lengths, and unambiguous numeric bounds. Jev gets one call per round of independent choices, with shared state. Once branches and array slots exist, one strict Luna call fills the open fields, and code stitches the answers back by path. Empty question sets and empty fill schemas make no call.

Example from the native-mode preflight, outside the 24 scored requests: “Use version 1. Lea Martin wants seven units urgently.” Memory is empty. Version is fixed, so Jev sees only the priority choice:

{
  "model": "jev-1.13.0",
  "state": {
    "request": "Use version 1. Lea Martin wants seven units urgently.",
    "memory": {},
    "partial_output": {
      "version": "1",
      "customer": {
        "$fill": "f0"
      },
      "quantity": {
        "$fill": "f1"
      },
      "priority": "PENDING"
    },
    "policy": "Build the object requested by `request`, following `schema`. Copy open strings and numbers from `memory`: put VARIABLE:<slug> in that slot, never write a new string or number. Closed enums, booleans, literal constants, and null use their actual values. A VARIABLE reference is replaced with its memory value only after the whole object is built. Check the original field constraints when choosing a variable. Do not use a display title where a slug is required. Preserve the order of requested array items. Do not add unrequested items. Null means explicitly absent. No repair, fallback, or explanation."
  },
  "questions": {
    "q0": {
      "type": "choice",
      "instructions": {
        "question": "For output path `priority`, which value satisfies the request and field constraints?",
        "field_schema": {
          "type": "string",
          "enum": [
            "normal",
            "urgent"
          ]
        }
      },
      "criteria": {
        "normal": {
          "value": "normal",
          "meaning": "Use the literal value \"normal\"."
        },
        "urgent": {
          "value": "urgent",
          "meaning": "Use the literal value \"urgent\"."
        }
      }
    }
  }
}

Luna then sees only the two open slots, with their original types and bounds:

{
  "model": "gpt-6-luna",
  "instructions": "Fill only the open string/number slots marked $fill in `scaffold`, following the original field constraints and `request`. Return native strings and numbers. Copy known values from memory when requested; other values may come from the request. Preserve array order. Closed values and structure are already decided: do not change them. No repair or explanation.",
  "input": "{\"request\":\"Use version 1. Lea Martin wants seven units urgently.\",\"memory\":{},\"scaffold\":{\"version\":\"1\",\"customer\":{\"$fill\":\"f0\"},\"quantity\":{\"$fill\":\"f1\"},\"priority\":\"urgent\"}}",
  "reasoning": {
    "effort": "high"
  },
  "max_output_tokens": 8192,
  "stream": true,
  "store": false,
  "text": {
    "format": {
      "type": "json_schema",
      "name": "mixed_open_slots",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "f0": {
            "type": "string",
            "description": "Output path customer. "
          },
          "f1": {
            "type": "integer",
            "minimum": 1,
            "maximum": 20,
            "description": "Output path quantity. "
          }
        },
        "required": [
          "f0",
          "f1"
        ],
        "additionalProperties": false
      }
    }
  }
}

Code assembles both responses and the fixed literal:

{
  "version": "1", // fixed by the schema; code fills it, no model call
  "customer": "Lea Martin", // Luna writes a native string from the request
  "quantity": 7, // Luna writes a native integer, not a memory reference
  "priority": "urgent" // Jev picks the enum value
}

A separate literal-only preflight produced version “1”, active false, quantity 7, and an empty array with 0 API calls and $0 API cost. The scored comparison used reference mode so all three methods did the same task; native mode is there for apps that need new strings or numbers.

Method

The original requests, schemas, memory, and answer key are unchanged, as are the original results, raw traces, source hashes, and protocol. Mixed has its own frozen protocol and audit. All-Jev already skipped literal constants; Mixed also collapses fixed object edges and keeps open fields out of Jev’s questions. It makes one final Luna fill, not a round trip at every depth. Bounded array length is a closed choice for Jev, not an open number.

Reference mode filters memory candidates by base type only, not by answer, field name, regex, or range. Validation runs after substitution. Native mode keeps string and number types and constraints in Luna’s projected schema. Both modes support strict required objects, bounded arrays, and disjoint unions. Neither handles recursive refs, overlapping unions, dynamic keys, or unbounded arrays.

Run and cost

The original methods began 30 Sept 2026, 4:35 pm UTC; Mixed began 30 Sept 2026, 5:29 pm UTC, on the same Mac in Montreal. All use pinned Jev 1.13.0 and GPT-6 Luna with high reasoning. Luna has no dated snapshot, so traces keep the returned model IDs. Time covers compilation, model waits, trace IO, assembly, substitution, and Zod validation. Mixed runs its two phases in sequence.

Original evaluation spend: $0.01476. Mixed evaluation: $0.00718 ($0.00361 Jev + $0.00357 Luna). Smoke tests: $0.00177 original, $0.00085 for Mixed’s four cases. Costs use recorded tokens at published Standard rates, including cached input and reasoning output, not invoices. Controller cost, tax, and human time are excluded.

Audit and downloads

Both audits passed: 72 unique trials and 145 evaluation request/response traces. They check request replay, field ownership, Ajv/Zod agreement on the original schemas, zero-call fixed cases, batching, usage, and unchanged original hashes. Inputs are hand-written synthetic cases under CC0. This is a small feasibility test, not a full benchmark.

Three-method resultsOriginal two-method snapshotInputsAnswer keyOriginal protocolMixed protocolBoth auditsOriginal frozen hashesMixed frozen hashesOriginal sourceMixed sourceOriginal tracesMixed traces