GameForgeBench tests coding agents building and repairing games in real engines

GamePhanes Studio has published GameForgeBench, an MIT-licensed benchmark that evaluates coding agents on the complete engineering loop of building, debugging and repairing interactive games inside real game engines. The repository, which the studio also refers to as Game Terminal-Bench, was opened on 21 August and is at 536 stars and 22 forks as of 6 September.

The design premise, quoting the project's own bench page: "Terminal benchmarks often stop at files, commands, and exit codes. Interactive software has a second truth: the project must import, launch, accept controlled input, change runtime state, and produce the intended behavior." Every task is scored on the whole sequence of inspect, edit, build, run, observe, diagnose, repair and verify, not on whether the code compiles.

Twenty tasks, five kinds of work, real runtime tests

The first production suite is 20 tasks organised into five families of four: gameplay and feel, engine and runtime, UI and interaction, content and systems, and delivery and quality. The public open release includes 81 completed executable tasks across Godot, Unity, Roblox, Minecraft, Unreal, web and generic engine projects, with one representative task, godot-battle-status-bars-gloss, checked in so the full workflow can be inspected without downloading the sealed corpus.

Each task is a Harbor-compatible directory: a task.toml with schema, timeouts and resource budgets, an instruction.md describing the outcome without prescribing the patch, an environment/ folder with a pinned engine and a Dockerfile, an independent tests/ folder that runs after the agent submits, and an oracle solution/ used to prove solvability that is never mounted into the agent's workspace. The reference task requires Godot 4.6.1 and Docker.

Scoring runs across five evidence surfaces, in the project's own words: build (import, parse and headless build complete inside the budget), runtime (the project launches and stays responsive under controlled probes), behavior (named events and state fields satisfy deterministic user-visible requirements), regression (existing behavior survives the change) and trajectory (every terminal action, patch, feedback message, cost, failure and repair attempt is auditable). A passing build is only the first of the five.

The leaderboard is empty, and labelled as such

The bench page shows a projected score chart for four combinations, with GamePhanes and Fable 5 at 84.2 percent at the top. Every one of those numbers carries the label "ESTIMATED BASELINE NOT MEASURED", and the caption states in full: "These values are product-planning estimates, not measured benchmark results or model capability claims." No coding agent has yet been scored against the suite in public.

The public JSON registry is the authoring contract, not a Harbor release yet, and the studio says the benchmark identity stays separate from Terminal-Bench 2.0 and TB-Science rankings even though it uses Harbor's execution and packaging layer. That separation is worth accepting as stated: the same task package format is a fair way to share infrastructure without letting one leaderboard flatter another.

An eval harness made for interactive software meets a real need. General coding benchmarks reward passing tests, and game code passes tests by exiting cleanly with no window opened. The parts that hurt a real player, a jump that misses its brief allowed timing window, a save file that gets corrupted when the save format changes, an upgrade choice that stops behaving the same way against the boss, are the exact tasks GamePhanes is scoring. A team choosing which agent to point at a Godot project can, once measured runs are published, compare on the full sequence that actually matters instead of on files changed.

Source

Primary source: GamePhanesStudio/GamePhanes on GitHub, the Game Terminal-Bench overview page and the GamePhanes Studio homepage. Star and fork counts from the GitHub REST API, retrieved 2026-09-07.