Meerada Cross-check

🔎 CROSS-CHECK — every answer examined by a jury from the other labs 1. gpt-5.6-luna (openai) · instruction_following █████████░ 0.90 · ✅ holds up · examined by 3 models · jurors agree 100% · confidence 0.86 - claude-sonnet-5: complete, no invented numbers - gemini-2.5-flash: follows every constraint; slightly long - grok-4.6: correct; the last paragraph is padding 2. deepseek-v4-flash (deepseek) · instruction_following ██████░░░░ 0.62 · ❔ jury split · examined by 3 models · jurors agree 67% ⚑ flagged: cites a 2024 figure not in the prompt | skipped the required table 3. qwen3.8-max (alibaba) · instruction_following ███░░░░░░░ 0.31 · ❌ does not hold up · examined by 3 models · jurors agree 100% ⚑ flagged: answers a different question WINNER: gpt-5.6-luna — score 0.90, examined by 3 models. Jury cost: $0.0041 · every juror is from a lab other than the answer it judged.

Illustrative report. Real reports come from your own keys; the jury's cost sits on your ledger like any other call.

How it works

1️⃣

Send to all

In the Manager, one task goes to several models in parallel — or from the terminal: meerada crosscheck "…" --models a,b,c.

2️⃣

Seat a neutral jury

For each answer, up to three jurors are chosen from the other labs you have keys for. Kin never judges kin. Fewer keys? A thin panel of 1–2 still scores, labelled "examined by N models".

3️⃣

Read the verdict

Score, the jurors' own one-line reasons, flagged spans, agreement, a winner — or "no winner" when no panel agreed. A score chip lands on every examined session.

The rules the jury cannot break

Rule	Why
A juror never judges its own family	OpenAI grading OpenAI is not neutral — the whole point is that it is not
Three labs = a full panel; 1–2 = a thin panel, labelled	Use what you have today, never pretend it is more
Pairwise comparisons run twice, in both orders	Position bias cancels
The rubric deducts explicitly for padding	Length bias cancels
Fleiss' kappa reported; split panel → "unknown"	A disagreement is information, not a score
Hard daily budget on the jury	It is your money
A programmatic check always outranks a jury	Tests beat opinions, even expert ones

From the terminal

meerada crosscheck "Summarize this memo in 3 bullets" \
  --models gpt-5.6-luna,claude-sonnet-5,gemini-2.5-flash \
  --context memo.txt --budget 1 --store ~/.meerada/jury.sqlite

Every verdict stays on your machine (~/.meerada/jury.sqlite) and becomes the training set for Meerada Judge.