The model operates as an agent on an electrical engineering design task. The input material can be complex โ it includes abstract circuit schematics and system diagrams, so it demands a rigorous integration of visual and textual content. The resulting design then passes three kinds of check: build checks verify that the design builds at all; simulation measurements verify electrical parameters; BOM constraints police whether the components used fit the stated requirements. Each check yields a deterministic verdict, so the same result can be reproduced independently. Cost and time per task and output token counts are measured alongside, allowing comparison not only of effectiveness but of a model's economics.
Evaluating models in engineering domains usually rests on closed questions or a model judge, neither of which says anything about whether the designed circuit actually works. EEBench closes that gap: it checks the engineering artefact with the same tooling used to check a real design.
Verify that the produced design can be built at all โ the first, hard correctness barrier.
Simulation measures whether the circuit meets the required parameters, instead of relying on the model's claim.
The bill of materials must fit the stated constraints โ ruling out designs that are electrically sound but not realizable.
The leaderboard reports cost per task, time per task and output token counts alongside the score itself.
The score speaks to electrical engineering, not general engineering ability; transferring it to other domains is unwarranted.
Deterministic verification requires a specific build and simulation environment; a change of tool versions can shift results.
atopile runs a public leaderboard with an option to submit your own model; in the published snapshot GPT-6 Astra leads with 69.3%.
