SemiAnalysis has released the general agent capability benchmark GDPx; the best model achieved only 12%.
SemiAnalysis partners with 14 professional reinforcement learning environments to launch the GDPx benchmark, covering 11 fields including chip design, cybersecurity, biology, finance, law, quantitative trading, RSI, hardware design, materials science, alignment, and software engineering. The goal is to establish it as the leading benchmark for measuring general agent capabilities.
Continue looking at this topic
OpportunitiesOmicOS turns biological agents into a usable entry point ↗Open-source projectIf Agent can read the accessible structure of a desktop interface, why does it still click the wrong button? ↗News and AnalysisAI agents generated only hundreds of orders: how should we view DoorDash's pilot? ↗Hotly discussed viewsConcerned that agents may become less capable after data desensitization? Which task details should be retained first? ↗Return to original text
A short introduction is retained here; the full text from the source website is not reposted. For specific wording and subsequent updates, please refer to the original article.
Open AIHOT original