Fast mode and queues

Most verdicts take about a millisecond. The exception is the gray zone, roughly one submission in twenty, where a language model is asked for a second opinion and the request can take up to three seconds. Fast mode lets you choose where that wait happens.

What fast mode does

Send "mode": "fast" with the request. The pipeline runs memory, rules and the classifier as usual. If the score lands in the gray zone, instead of waiting on the language model the API returns the classifier’s nearest label immediately with two extra codes:

• LLM_SKIPPED: the second opinion was not requested.
• ML_GRAY_FALLBACK: the label is the classifier’s nearest side, not a confident verdict.

Everything else is identical. Confident verdicts carry the same codes they would in full mode.

{
  "id": "9b1e…",
  "label": "spam",
  "score": 0.41,
  "stage": "ml",
  "reasons": ["URLS_PRESENT", "LLM_SKIPPED", "ML_GRAY_FALLBACK"],
  "latency_ms": 1.2
}

The two-step pattern

1. On the ingest path (the endpoint that answers the visitor’s browser), classify with mode: "fast". Accept the submission and return to the visitor at once.
2. In your delivery worker, before mail or webhooks go out, look for LLM_SKIPPED in the stored reasons. Only for those, classify again in full mode and replace the stored verdict.

// ingest
const quick = await classify(fields, { mode: "fast" });
await submissions.insert({ ...fields, verdict: quick });
respond(200);

// delivery worker
const sub = await submissions.next();
let verdict = sub.verdict;
if (verdict?.reasons.includes("LLM_SKIPPED")) {
  verdict = (await classify(sub, { mode: "full" })) ?? verdict; // fail open to the fast verdict
  await submissions.update(sub.id, { verdict });
}
if (verdict?.label === "spam") quarantine(sub, verdict); else deliver(sub);

The second call counts as a classification against your quota, so about one submission in twenty costs two checks. The visitor never waits, and nothing is delivered without the full verdict.

When to skip the re-check

If you simply never want to wait on the language model, use fast mode everywhere and treat ML_GRAY_FALLBACK verdicts as the review bucket described in Handling verdicts. Precision in the confident zones is unaffected; you only give up the second opinion on the borderline cases.