Rubik Arena

Rubik Arena

Watch LLMs solve a Rubik's cube in a live 3D dashboard, each with its own timer. Every move is checked by the harness.

Rubik Arena gives several models the same seeded scramble and shows each one working on a 3D cube with a running timer. You can run the models one after another or in parallel. It records time to solve, time per turn, time to first token, moves, distance to solved, tokens, and credits or cost.

Models are reached through CLIs you are already logged into, so no API key is needed: Pi (which routes to Kiro, Codex/ChatGPT, Copilot and other providers), and Kiro CLI. API keys (BYOK) and local models through OpenAI-compatible endpoints (Ollama, OpenRouter, DeepSeek, vLLM) are also supported. Future models work without code changes: the dashboard reads each CLI's model list, and a new CLI needs one adapter file.

Quick start

Requires Node ≥ 22.

git clone https://github.com/Wayan123/AI-Rubik-Solver.git
cd AI-Rubik-Solver
npm install
npm start          # builds the dashboard and starts the local runner on 127.0.0.1:8787

Open the link printed in the terminal (http://127.0.0.1:8787/#token=…). Pick a preset, for example Claude Opus 5.5 · high (Pi → Kiro), set a scramble depth (start with 3–5), and press Start race.

If the runner isn't running, the dashboard switches to demo mode and only the solver baselines can run. The Kociemba and random-mover baselines always work, with no login.

Command-line run without the UI (this calls the real model and uses credits):

npm run test:e2e                                                   # Pi → kiro/claude-opus-5-5, thinking high, depth 3
RUBIK_E2E_DEPTH=5 RUBIK_E2E_SEED=7 npm run test:e2e
RUBIK_E2E_ADAPTER=kiro-cli RUBIK_E2E_MODEL=claude-opus-5.5 npm run test:e2e

Model sources

All of these use a login you already have. No API key is needed except for the BYOK row.

pi --list-models shows every model your Pi logins can reach. Pi provider extensions (Kiro, Antigravity, …) are found automatically from ~/.pi/agent/settings.json, so a newly installed provider shows up without code changes. The runner also scans supported CLI and IDE harnesses at startup; use Harnesses & models → Refresh models after installing or updating one. Discovery never sends a prompt or consumes model credits. See the harness connection and support guide.

Presets are in config/contestants.json and grouped by login in the dashboard. Discovery does not rewrite them. To add another CLI, see docs/adding-a-model.md.

Not usable on this machine right now:

• Codex CLI and Claude Code are not logged in.
• Gemini CLI personal tier was retired by Google; use Antigravity instead.
• The OpenCode free model returned 401.
• Pi's nvidia/… models did not answer within 180 s.

Choosing the scramble

You decide the starting cube. The Scramble panel has four sources:

Every input is checked before a race starts: allowed moves, 9 stickers per colour, fixed centres, twisted corners, flipped edges and permutation parity. An impossible cube is rejected with a readable reason. The panel shows the unfolded start cube and how far it is from solved (exact up to 6 moves, otherwise an upper bound).

API: POST /api/races with scramble: { source: "seeded" | "moves" | "state" | "random-state", seed?, depth?, moves?, state? }. Older clients that send only moves or seed/depth still work.

Replay

A finished race (live, or opened from History) switches to replay mode, showing each cube's final position.

• Replay all from start plays every contestant simultaneously from the scrambled cube to the end. Next to it are Pause all, Resume all and Show final.
• Each lane has its own controls: restart, previous move, play/pause, next move, and a slider to jump to any move. Lanes can be played one at a time, independently of the others.
• Pacing:
  • Real time: each move appears when the model actually answered (turn start + latency). The replay reproduces the race, so faster models finish first. Speed 1×–300×.
  • Move by move: every lane advances at the same rate (0.5–16 moves/s), which compares the solution paths side by side.
• While replaying, the timer, turns, moves, distance and sparkline follow the replay position. The status line shows the current move, the previous moves and the next one.

How a race works

1. You pick the start cube (see above). Every contestant gets exactly the same state.
2. Each model gets the same system prompt (PROMPT_VERSION v1). The prompt covers Singmaster notation, an unfolded net, a per-face listing and the 54-character facelet string. The model must answer with {"moves": [...]}.
3. Two modes:
   • Interactive (default): the model returns 1–10 moves per turn and receives the new state after each turn.
   • One-shot: the model returns the full solution in a single answer.
4. The harness parses and applies every move itself; the model's own claims about the cube are never used. An invalid token makes the whole answer invalid. Three invalid answers in a row end the run.
5. Distance to solved is exact up to 6–8 moves and a Kociemba upper bound (≤) beyond that. It gives a progress score, (initial − best) / initial, so a model that doesn't finish still gets partial credit. This follows Cube Bench (arXiv:2512.20595); CubeBench (arXiv:2512.23328) reports 0% on long-horizon tasks for leading LLMs. See docs/research.

Metrics recorded per run: wall time, latency and time to first token for each turn, turns, moves, invalid answers, initial, best and final distance, progress, tokens in and out, cost (USD or credits), the reported model, and the effort level the CLI reports back.

First results (2026-09-29, this machine)

Each row is a single run (n = 1). The table shows the pipeline works; it is not a ranking. GPT-5.6 Sol's first turn made a wrong move, which it corrected on turn 2. DeepSeek 3.2 used all 30 turns: it applied 237 moves, got within 3 moves of solved, but never finished.

With state feedback in interactive mode, Claude Opus 5.5 solved even a fully random cube. The literature reports 0% on long-horizon one-shot tasks, so the interactive result shows how much per-turn feedback matters; single runs cannot quantify the effect.

Security model

• The runner listens on 127.0.0.1 only. Each launch creates a random bearer token that every /api/* call must send. A Host-header check blocks DNS-rebinding attacks. No CORS headers are sent. A strict CSP is set.

• CLIs are started without a shell (arguments passed as an array) in a fresh empty temp directory, with tools disabled:

  • Pi: --no-tools --no-skills --no-context-files --no-extensions --no-session
  • Kiro: --trust-tools=. An answer that attempted a tool call is rejected.
  • Hermes: -t clarify. An empty -t "" falls back to your configured tools; in a test that config wrote a file into $HOME.

  Flags such as --yolo, --trust-all-tools or --dangerously-* are never used. A timeout or cancel sends SIGTERM, then SIGKILL.

• API keys never reach the browser. Presets and requests hold only the env var name (apiKeyEnv). Option keys that look like secrets are rejected.

• Race logs are written to data/, which is gitignored. They contain raw model answers.

Details: SECURITY.md. Discovery-specific privacy, PATH-shadowing, manifest, WSL, SSRF and supply-chain analysis is in the discovery threat model.

Project layout

packages/cube-engine   facelet model, moves, scramble, exact/bounded distance, Kociemba wrapper
packages/bench-core    prompt, parser, run loop (timers, limits, cancel), race, ranking
packages/adapters      pi, kiro-cli, openai-compatible, baselines; spawnCli sandbox helper
apps/runner            Fastify server: auth, SSE live events, JSON race store, serves the dashboard
apps/web               React + react-three-fiber dashboard, demo mode in the browser
config/                contestant presets
docs/                  BMAD brief, research, design spec, implementation plan, how-to docs
scripts/e2e-pi.ts      opt-in live run against a real model

Development

npm run dev          # runner with reload (port 8787)
npm run dev:web      # Vite on 127.0.0.1:5173, proxies /api using data/.runner-token
npm run check        # biome lint + tsc -b + vitest + build (same as CI)

The spec is in docs/superpowers/specs. Contributions are welcome; see CONTRIBUTING.md.

License

MIT. Uses cubejs (MIT) for the two-phase solver.