Can Codex Build an Entire x86 Assembly Game Autonomously? A Real-World Experiment

The Rules

Before starting the experiment, I established a set of rules that would remain unchanged until the project was complete. The objective wasn't simply to build an Asteroids-style game. The objective was to evaluate how autonomous Codex could be when acting as a software engineering agent. To keep the experiment fair, I intentionally limited my own involvement.

My responsibilities:

• Define the project objective.
• Answer questions about gameplay when clarification was required.
• Play-test every completed milestone.
• Report gameplay bugs or unexpected behavior.

Codex responsibilities: Everything else. This included:

• Project planning
• Software architecture
• Assembly implementation
• Refactoring
• Debugging
• Documentation
• Git repository management
• Building the project
• Running the game after each build
• Maintaining the project roadmap
• Verifying completed milestones

One rule never changed throughout the experiment: I never wrote a single line of Assembly code. In fact, I never built the project manually or launched the game during development. Every build and every execution was performed autonomously by Codex. My role was simply to evaluate the delivered result as if I were testing software developed by another engineer.

Reality Check

Although Codex handled the implementation independently, this was not a "hands-off" experiment. After each milestone I played the game, looked for unexpected behaviour and reported anything that didn't work correctly. During the entire project I reported only two gameplay bugs that had not been detected automatically. Both were analyzed, fixed and verified by Codex before development continued. Apart from defining the original goal and validating each milestone, I had no involvement in the implementation itself.

Why these rules?

One of the biggest challenges when evaluating AI-generated software is separating the work performed by the AI from the work performed by the human. Many demonstrations claim that an AI built an application, but the human developer still makes architectural decisions, edits the generated code, fixes compilation errors and completes the final implementation. I wanted to eliminate as much of that influence as possible. The goal was not to see whether Codex could assist me, to see how far Codex could go without me writing the code.

The Development Process

Once the rules were established, I created an empty GitHub repository and described the project. From that point on, development became an iterative process. Codex planned the work, implemented a milestone, built the project, ran it, verified the result, committed the changes, and then handed the project back to me for testing. My job was simply to play the latest version and answer one question: Does it behave as expected?

If I found a problem, I described what I observed. I never suggested how to fix it or where the bug was located. Codex analyzed the report, identified the cause, implemented the fix and presented a new version for testing. The cycle then repeated until the next milestone. Over time, the project grew from an empty repository into a complete DOS game with structured source code, documentation, persistent high scores and a fully playable gameplay loop.

Results

By the end of the experiment, Codex had produced:

• 18 Assembly source files
• ~2,500 lines of Assembly
• Complete project documentation
• GitHub repository with structured commits
• Persistent high score system
• Playable Asteroids-style game

During development:

• Human-written Assembly: 0 lines
• Human builds: 0
• Human launches: 0
• Gameplay bugs reported: 2
• Gameplay bugs fixed by Codex: 2

One thing surprised me the most. Codex behaved much less like a code generator and much more like a software engineer. Instead of immediately writing gameplay code, it spent time planning the project, documenting the architecture, organizing the repository and verifying each completed milestone before moving on.

Summary

The "Active Time" represents the time Codex spent actively implementing each milestone. It does not include my play-testing, discussions, or the time between development sessions.

Source code: https://github.com/mbostjan/AssemblyAsteroids

The entire repository, including the complete commit and documentation, is publicly available for anyone who wants to review the experiment or reproduce the results.