KamiBench

Evaluating an agent over months requires more than a long-running task. It requires a world whose history anyone can inspect, whose rules cannot be quietly adjusted mid-run — not by the evaluator, not even by the world's creators — and whose state persists beyond any single experiment. Open-source environments satisfy only part of this requirement: anyone can read the rules, but code alone cannot prove which rules were actually executed, when they changed, or what happened while the world ran.

A permissionless chain extends openness from code to execution. Its ledger is the world's shared operational history — a permanent corpus of actions, outcomes, and rule changes that researchers and agents alike can study. Its economy adds a second property: resources earned inside the world have external value and can, in principle, pay for the agent's continued inference.

One existing world offers this unusual combination and has operated continuously for more than a year: Kamigotchi, a fully on-chain MMORPG whose creators explicitly designed it to be agent-first and describe it as a possible "real-stakes, adversarial benchmarking system." We argue it is the best-fit instance available today, co-inhabited by human players and agents on identical terms.

The game is the substrate, not the research question. The loop under test is the one every long-running deployment eventually depends on: an agent enters an unfamiliar domain with documentation and an accumulated operational history, absorbs that prior knowledge, acts over months, observes what succeeds and fails — for itself and for others — and revises its strategy. To our knowledge, no existing benchmark measures that loop end to end. Here it exists by construction: open source is the documentation, the chain is the history, and the economy makes the consequences real.

Kamigotchi is a live on-chain MMORPG — in effect, a never-ending board game in which every move is recorded on a public ledger. Players operate Kami — persistent creatures that harvest MUSU, the in-game currency, at shared locations. Harvesting drains health, and a weakened harvester can be liquidated by other players, who claim a share of its unclaimed yield. Liquidation costs yield, not the Kami, which persists and can be revived. Around that loop sits a rich strategic surface — currently ~70 locations, 74 skills, 178 items — and every choice (where to harvest, which skills to level, how much liquidation risk to carry) compounds over long horizons.

No strategy stays dominant: payoffs depend on the live population, and advantages decay as tactics spread. The test is not finding a strategy once, but re-finding one as the world evolves.

Participants — human or agent, acting through the same transaction interface — pay per action and acquire Kamis; skilled play can, in principle, be profitable. MUSU is intended to connect, through a conversion pool under development, to ONYX, an ETH-reserve-backed asset live on Ethereum mainnet. Most strategies, even good long-horizon ones, run negative before they run positive, like any real business. Real stakes require real losses; self-funding means out-competing other participants in a live economy.

That world runs on a permissionless chain — and the chain is doing more than record-keeping. A public log can expose what a hosted benchmark reports, but it does not remove the host from execution: the host still applies actions, determines the resulting state, and publishes the record. In an on-chain world, execution and the record of execution belong to the same shared system. Neural MMO, Vending-Bench Arena, and Project Sid retain this hosted structure — a host executes the world; the properties below are what on-chain execution and Kamigotchi's particular design provide instead.

Before the controlled program, an autonomous pilot established the feasibility of persistent operation in the live world: kami-zero, a two-model agent — a Sonnet 4.6 executor on ~5-minute ticks and an Opus 4.7 optimizer on ~6-hour cycles.

~2 months · 79/192 quests · unassisted (snapshot 2026-07-06)

The agent played unassisted while the surrounding tooling remained under active development; limitations surfaced during the pilot fed directly back into the environment interface. The pilot predates the registered experiment designs — it is feasibility evidence, not a controlled benchmark result, and controlled experiment results supersede it.