We have uploaded some initial, objective benchmarks at https://gertlabs.com/?mode=oneshot_coding

TL;DR: the open weights one is severely benchmaxxed (possibly the most benchmaxxed we've tested), cannot adapt to unorthodox benchmarks, is verbose and often runs out of output tokens before completing solutions to difficult problems. Probably not going to be your daily driver, trailing far behind other recent open weights model releases.

Long context agentic work and other benchmarks will be finalized in the combined scoring in 24 hours.

So far, strange results (especially for the proprietary Pro version) -- the open source version is pretty weak in initial testing. But the Pro version is definitely intelligent, but struggles with formatting and has a low success rate in our tests, which drags down its score. This does suggest a fair amount of overfitting. Basically, when it works (plays the eval without compilation failures, violating the sandbox, or throwing some other kind of error), it's a smart model. But it's not at the same level as Kimi K2.6 or GLM 5.1. Both the open weights and free version are extremely verbose.

Still, the quantity and quality of open weights options is incredible. Moats are closing.