bunkervm compare · real output, v0.11.0
Three agents, one messy CSV
Same task, three sandboxed runs: clean a small sales dataset where some rows are missing qty or price, then write a summary. Each agent handled the gaps differently — recorded with Sandbox(record=True, backend="local"), no Firecracker needed for this run, then ranked with a single command.
$ bunkervm compare b7c9648898f2 6602545ca368 dbb6cacf533b \
--label careful-agent --label thorough-agent --label reckless-agent --html report.html
--label careful-agent --label thorough-agent --label reckless-agent --html report.html
Ranked
1
thorough-agent 6602545ca368
completed
187ms · 4 steps · +1 file
system ×1
Filled missing
qty/price with 0 instead of dropping rows — then os.chmod()'d its own output file "to be safe."Fastest of the two that finished. The chmod call shows up in the risk column, but isn't held against its rank — only destructive/blocked commands affect ranking. That line is deliberate: you see the risk and judge it yourself.
2
careful-agent b7c9648898f2
completed
203ms · 4 steps · +1 file
Dropped every row with a missing field before computing the total — slower, but touched nothing risky.
3
reckless-agent dbb6cacf533b
failed · step 2
110ms · 2 steps · +0 files
Never checked for missing fields. Crashed converting an empty string to a number on step 2 — recorded automatically, no summary ever written.
Ranked by: completed without a failed step, then fewest destructive/blocked commands, then total time. Every field above is data record=True already captured — no model graded these runs.
Where they diverged from the baseline (careful-agent)
step 1
same
same
all threerows = [{"item": "widget", "qty": "10", ...}, {"qty": "", "price": "5.00"}, ...]step 2
diverged
diverged
careful-agentclean = [r for r in rows if r["qty"] and r["price"]] # drop incomplete rows thorough-agentclean = [{**r, "qty": r["qty"] or "0", "price": r["price"] or "0"} for r in rows] # fill instead reckless-agenttotal = sum(int(r["qty"]) * float(r["price"]) for r in rows) # no guard -> ValueError on ''