Fu Index

Ambition

Sort by

By lab

# Model Ambition Hard tests Fresh rivals Weight class Independent tests Fu
How the Ambition Score works

For each launch we take the headline benchmark table or chart of the official launch post or model card: the benchmarks shown and the models the new one is compared with. Each part is rated against the database as it stood on the launch date and scored 0 to 100.

Hard tests: for each benchmark (percent scores only), the field level: the best score of the second-best lab among models of its weight class by the launch, the new model included. A saturated test was hard and is no longer: the field has caught up, so one lab’s lead does not saturate it. A test counts only once at least 5 models of 3 other labs had results before the launch (a brand-new test has no top score yet); for a model with published size, the weight class is models up to four times as large. A test with a field level of 90% or more counts 0, one at 50% or less counts 1, linear in between; the part is the mean of the hardest half of the rated tests, but at least three (all of them if fewer), so easy tests next to hard ones cost nothing, while one hard test among two does not make a hard table. Fresh rivals: a rival up to one month old counts 1, one twelve months or older counts 0. Weight class: an open-weights rival of an open-weights model counts 1 when it has as many total parameters or more and 0 at a quarter of the size or less. All other rivals are compared by Fu Score (best variant) with a bar: the new model’s own Fu Score, or, if lower, that of the best model of the second-strongest other lab at the time. A rival at the bar counts 1, one 6 points or more below it counts 0. Independent tests: the share of benchmarks not made by the launching lab; internal evals count as in-house.

Rivals are the compared models of other labs. The lab’s own older models show progress, not a fight, and its own models from the same launch are siblings; neither is rated. A launch post with no rival from another lab scores 0 on fresh rivals; weight class is then not rated. The Ambition Score is the weighted mean of the parts that could be rated: hard tests count three times, fresh rivals and weight class once, independent tests half; it needs at least three parts, hard tests among them. Models with too few scores for the Fu Ranking get an estimated Fu Score, marked *.