View 7 · operations

Cost and latency

What each configuration costs and takes per message, versus what it gets right. The cost is API cost only; the open models cost 0 in API but need their own GPU (here, a DGX Spark), whose cost is not included.

Quality measured on

Single pass With reviewer Colour: family of the 1st-pass model. Labelled: the configurations no other beats on both quality and latency at once (if the label does not fit, the detail is on hover).

Full table

Latency: median per message. On configurations with a reviewer it is the sum of the two passes. Jev is measured via API from Europe; the open models, on a GB10 or Intel Arc GPU (CLM and Strands) with the model already loaded. Jev's cost: the one OpenRouter returns, or input tokens × $0.042/M on the TypeSafe API; Span-01's, input tokens × $0.02/M (OpenRouter; Lite is free). On OpenAI's native decisions route (JEV-78) the cost is computed with input tokens × $0.10/M because the response does not report an amount; internal rejected attempts are still counted. Microsoft-Decision-1 (JEV-90): endpoint-recorded cost (~$0.00412 per full pass, ~$0.0212/1000 cases); summary client latency 693 ms. Claude Haiku 5.5 (JEV-83): $0.10/M input and $0.50/M output, with thinking inside the output; on the adaptive cell the published cost is a lower bound (E11's first answer (adv5) was lost in a session cut and the resume repeated it, with no recovered cost). Universe of the table's latencies: on Haiku they are the median of the 194 unique records (1,657 / 1,289 / 2,656 ms); the scoreboard prints the median of the scorer's summary over 154 base+new cases (1,656 / 1,285 / 2,639 ms) — they are not the same figure. OpenMed Ministral-3B (JEV-85, incomplete coverage): cost not recorded — its records store a null cost (local endpoint with no API pricing; electricity and hardware not measured)— and its published latency is the median of the scorer's summary over the 9 base+new phases (4,780 / 1,913 ms over 154 records per mode), the same figure as the manifest and the scoreboard, not the automatic median over the error-free phases. GPT-OSS-120B on Cerebras (JEV-54): the recorded ms include the spacing imposed by the account rate limits (5 req/min, 150 req/h) — they do not measure the model's latency; no median is published («not available for comparison» on the profile). The 13 JEV-106 runs (Oct 16) measure OpenRouter's decisions endpoint with the jev adapter: endpoint-recorded cost on each of the 194 records and latency = median of the scorer's summary over the 9 base+new phases (154 records).