GPT-4o-mini (2024-07-18)
Timed at 43 tok/s, with results on 2 openly licensed boards, and not scored by Artificial Analysis.
retrieved sha256 · 0bb53a77713f
Measured speed
3,601 requests · 1 endpoint · 30-minute windowOpenRouter's own timing of its routed traffic, weighted by requests, over one 30-minute window read . Every route is on the endpoint ledger below.
Every endpoint
1 priced · 1 timed · cheapest first- OpenAIopenai · listed$0.15$0.60$0.075not declared16k43 tok/s32498ms3,601 req100%
Uptime is OpenRouter's own figure for each seller over the trailing day, read once a day, and the column prints the latest reading, taken . The one seller with a reading was at 99% or better on each of the 23 days held. Source: OpenRouter's public endpoints API, one reading a day for each seller: 1,211 sellers carried one on the latest of the 23 days held.
Source: OpenRouter's public endpoints API. 1257 sellers across 359 models when the mirror ran, 186 of them served by more than one. List prices, not what anybody paid; a discount is the seller's own. Read . Speed is OpenRouter's own timing over one 30-minute window, read .
What each seller accepts
16 parameters · 1 seller- frequency_penaltyevery seller
- logit_biasevery seller
- logprobsevery seller
- max_tokensevery seller
- predictionevery seller
- presence_penaltyevery seller
- response_formatevery seller
- seedevery seller
- stopevery seller
- structured_outputsevery seller
- temperatureevery seller
- tool_choiceevery seller
- toolsevery seller
- top_logprobsevery seller
- top_pevery seller
- web_search_optionsevery seller
The one seller that publishes a list accepts these 16. Each list is the seller's own declaration in OpenRouter's endpoints API, read , and nothing here tested one.
Price position
342 priced modelsBlended $0.262 per million tokens, 3 in to 1 out: cheaper than 251 of the 342 priced models in the catalog. Each bar is one stretch of the catalog on a logarithmic price scale; the mark is this listing.
Measured quality
145 models with an index · 132 on a design arena boardNot ranked in any Design Arena category.
Epoch AI has not placed GPT-4o-mini (2024-07-18) on its Capabilities Index, or places it under a name the pairing does not reach. Epoch places a model once four of its benchmarks carry a result.
Benchmark results
2 of 6 boards · 8 results| Task | All runs | Score |
|---|---|---|
| Chess Puzzles | 167 | 0.03 |
| FrontierMath-Tiers-1-3-v2-Private | 128 | 0.01 |
| GPQA diamond | 342 | 0.38 |
| MATH level 5 | 108 | 0.53 |
| Mystery Game Puzzles | 149 | 0.12 |
| OTIS Mock AIME 2024-2025 | 319 | 0.07 |
| SimpleQA Verified | 106 | 0.08 |
Mean accuracy, 0 to 1, on the task the row names, as Epoch AI runs it and averaged over repeated runs. All eighteen of their tasks, including the held-out FrontierMath tiers, named as they name them. The model id carries the effort setting the run used. Epoch AI Benchmarking Hub names this run gpt-4o-mini-2024-07-18, with no effort setting written on it. All runs is how many runs the publisher scored on the task, every model's. Read from epoch.ai ↗, which dates its own data . Epoch AI Benchmarking Hub data, by Epoch AI, under CC BY 4.0.
| Task | Score |
|---|---|
| polyglot pass rate | 3.60 |
Percent of 225 Exercism exercises solved with a second attempt allowed, higher is better. The id carries the edit format, because the publisher says a model alone does not identify a run. Aider polyglot names this run gpt-4o-mini-2024-07-18 (whole), with no effort setting written on it. Read from raw.githubusercontent.com ↗, which dates its own data . Aider polyglot benchmark, Apache-2.0, by the Aider project. Results are submitted by contributors rather than produced by the publisher.
2 of the 6 openly licensed boards Vitna mirrors carry a result for GPT-4o-mini (2024-07-18), each printed as its publisher published it and none added to another. MTEB, LiveBench, HELM and Vectara hallucination leaderboard have not published one under a name that matches its listing, and names are matched to the catalog exactly and never guessed. A task that is a link opens its column on /capability, with every run scored on it; an effort that is a link opens its run's row there, where the run is ranked on Vitna's scale. Every board, with its licence and what its publisher measures, is on /benchmarks.
Source: OpenRouter (openrouter.ai/models), as of 17 Nov 16:29 UTC. sha256 0bb53a77713f over 853,270 bytes.
Catalog: OpenRouter's public model list.
Intelligence, coding and agentic indexes: Artificial Analysis, as published by OpenRouter.
Elo, win rate and rank: Design Arena, as published by OpenRouter.
Vitna ran none of these evaluations and adjusted none of these figures.
