Vo Forschern geschrieben, vom Halluziniera kuratiert
Researchers demonstrate with PACE that the performance of AI agents in complex tasks can be reliably predicted – using a clever selection of simple, inexpensive individual tests that cost a hundred times less.
A controlled study compares nine lightweight neural networks across multiple generations. The result: Newer isn't always better, and an older model proves to be a surprisingly well-balanced all-rounder.
A new benchmark from Microsoft tests how well AI systems understand real Word, Excel, and PowerPoint files. The result: Even the strongest models achieve only about 60 percent on demanding tasks.
A new benchmark called ISOSCI shows: When AI models excel at scientific tasks, it's usually due to retrieved factual knowledge – not genuine logical reasoning ability.
Researchers have developed an automatically updating test for visual AIs. It costs only 30 dollars per update and is designed to prevent models from cheating through memorization.
A massive dataset with over 500,000 AI research papers aims to show whether artificial intelligence can predict scientific breakthroughs, new collaborations, and trends. Initial results are sobering: The AI-generated future is less diverse and original.
A new study shows that language models like GPT-5.4 and Claude breeze through simple medical tasks but fail precisely at the most critical points. The results reveal a dangerous reversal of priorities.
Researchers have created a new test environment called BuilderBench, where AI agents must learn to erect structures through trial and error. The result: Even the most modern AI models fail miserably at tasks that are intuitive for a child.
Researchers present an infinite problem generator for linear optimization. Instead of a static dataset that AI models can memorize, it produces fresh problems at the push of a button – including a guaranteed correct solution.
MedStreamBench is the first to test whether AI models in medical videos not only make the correct diagnosis but also respond or raise an alarm at exactly the right moment.
A new, constantly growing test called TestEvo-Bench evaluates how well AI agents can write new software tests or adapt old ones when the actual program code changes – using real, executable checks.
Researchers show that a pure reward strategy for AI agents in clinical practice often fails. Missing basic knowledge and hidden codes slow down learning – the solution is a combination of cramming and trial and error.
A new benchmark shows that even the best AI models cannot reliably recognize unusual scenes in videos. AnyGroundBench specifically tests how well AI can adapt to specialized domains like animal research or surgery.
A new model combines voice, image, and script knowledge to identify the correct character in TV series, even with short sentences or across many episodes.
Researchers have developed EvoPolicyGym, a benchmark that measures how cleverly AI models learn from mistakes and improve their strategies. GPT-5.5 leads the field, but the devil is in the details.
A comprehensive test from 15 disciplines, including real banking data, examines AI agents for data analysis. Instead of just rough grades, it provides detailed insights into the strengths and weaknesses of individual programs for the first time.
A new test shows: Many language models fall for sophisticated prompt tricks and behave unsafely as soon as the same question is phrased just a little differently.
Many AI models only appear to solve programming tasks cleverly. A new benchmark alters known problems so that the old solution no longer works – and reveals who can truly think algorithmically.