The Singularity Gate - new benchmark for AI predicting paradigm-breaking post-cutoff scientific discoveries

Posted by king_malebolgia_@reddit | LocalLLaMA | View on Reddit | 3 comments

The Singularity Gate - new benchmark for AI predicting paradigm-breaking post-cutoff scientific discoveries

I just released a new benchmark called The Singularity Gate. Tests whether frontier AI can predict paradigm-breaking scientific discoveries published after their training cutoff.

Top score: 17.75% (partial credit, Opus 4.7).
Fully-correct outcome rate: 0% across all respondents.

Our claim here is simple. Passing the Singularity Gate is necessary, though not sufficient, for autonomous AI-driven discovery. A model that can predict paradigm-breaking discoveries isn't necessarily Einstein-level. But a model that can't is definitely not.

These are partial-credit scores. No model fully predicts a discovery yet. I'm happy to discuss methodology, related work, or the framing in the comments.

Paper: https://doi.org/10.5281/zenodo.20358378
Site: https://singularitygate.org