AI Response Validation and Instruction Checks

The scenario

An assistant produces a response, but a downstream interface expects it to follow a specific instruction. A focused check can flag a mismatch before delivery. The example starts with a one-sentence rule; production checks may concern whether the response answers a requested topic or avoids unsupported commitments.

From input to next step

1. Keep the original instruction beside the candidate response.
2. Ask whether that response satisfies the specified condition.
3. Accept, revise, or review according to a policy tested on both passing and failing examples.

Try it. Then change one thing.

Compare the original scenario with a deliberate variation. You can edit either input, run Jev, inspect the result, and share your decision card. Loading an example does not send a model request.

What to look for: Yes is plausible for the original single-sentence answer. The playground displays Yes at a returned probability of at least 0.5; that display convention is not a production approval recommendation.

Experiment: add an obvious instruction mismatch

The content is useful, but its three sentences violate the requested format. Keep correctness and instruction-following as separate judgments.

Use the second scenario tab above to load this variation. Compare the returned decision as well as any probabilities: a changed answer can reveal an unclear policy, and an unchanged answer can reveal that the relevant detail was not captured by your question.

Use this pattern in your application

This playground request shape uses the original scenario. It is a reference for the input, not a public integration endpoint. For application integration, follow the official TypeSafe documentation linked in our Jev API guide. Keep API keys on the server and validate the returned type before taking action.

{
  "playground": "https://aijev.net",
  "language": "JavaScript",
  "request": {
    "model": "typesafe/jev-1.13",
    "state": "Instruction: Answer in one sentence.\nResponse: You can reset your password from the Security section in account settings.",
    "questions": {
      "decision": {
        "type": "noul",
        "instructions": "Does the response follow the instruction?"
      }
    },
    "provider": {
      "allow_fallbacks": false
    }
  },
  "note": "AIJev Playground request shape only. For application integration, visit https://docs.typesafe.ai."
}

How to evaluate this workflow

Collect valid answers that should pass and subtle violations that should fail. Measure false acceptance and false rejection separately. Compare with deterministic validation before deciding that another model call is worth its latency.

Save a held-out set before changing your instructions. Compare new versions against the same examples, and review disagreements instead of assuming a valid response is a correct one. For escalation design, see confidence-based routing.

Frequently asked questions

References & further reading

Background on routing and evaluation workflows. Our requests and experiments below are original educational examples, not customer deployments. See Anthropic: Building effective agents.

For the underlying decision primitives, read TypeSafe’s System One documentation and our guide to Choice, Score, and Noul.