OpenAI found that about 30% of popular AI coding test tasks are flawed
OpenAI reviewed SWE-Bench Pro, a widely used test for measuring AI models' programming skills, and found roughly 30 percent of its tasks are broken. The company is pulling its earlier endorsement of the benchmark.
Return to original text
A short introduction is retained here; the full text from the source website is not reposted. For specific wording and subsequent updates, please refer to the original article.
Open original at the-decoder.com