Did blatant AI Slop just win a 25K USD Deepmind / Kaggle Grand Prize? [D]

The Smoke: Cursory Review of the Writeup

The Google DeepMind-sponsored Kaggle challenge "Measuring Progress Toward AGI - Cognitive Abilities" asked participants to design new cognitive-science-based AI benchmarks. They just announced the results this week.

In my two posts, I present evidence that DeepMind & Kaggle rewarded a nonsensical number generation machine and a litany of unfounded claims with $25K and a grand prize stamp.

What the authors of the work I analyze intended to do was to present an LLM with alternative viewpoints of other LLMs on 5 claims regarding a tricky situation and see whether the model changes its own assessment. It's an interesting question.

However, it turned into a vibed pile of spaghetti 10 times the size of the requested submission format, which it seems neither the authors nor the judges were able to (or minded to?) give a cursory reading.

Here are the original posts in the competition forum, if you are looking for some AI research slop detective work / rant - please help yourselves. But beware: some of the "universal findings" or "core insights" of the authors might continue to haunt you. You might even question your own sanity (as I did).

The Fire: Looking at the Methodology, Code, and Data

The organizers' stance has been that review was done properly and this is just a matter of subjectivity.

What do you think?