Where it went wrong
Every miss, false alarm and failed recall in the saved results, counted from the rows each time you load this page. I'd rather you see them here than find them yourself.
Real groups it missed
2 of 20 real groups missed, reading the oldest 50 transactions per wallet
| Group | What it missed | Reading 200 transactions |
|---|---|---|
| rs-15 | It found no link to the group's shared fee payer (0x3431...7fdb). | Still missed |
| rs-19 | It found no link to the group's shared first funder (0x13ff...2bd5). | Found |
Inside the groups, 13 of 70 wallets were not tied back to their group: rs-05 (1 of 6), rs-15 (4 of 6), rs-19 (3 of 3), rs-20 (5 of 6).
Counted from data/real-set/results.json, data/real-set/results-tx200.json
Ordinary wallets it flagged
3 of 20 ordinary wallets got something other than a plain "pay"
| Wallet | Answer | Code | Why |
|---|---|---|---|
| rc-03 | caution | ROUND_TRIP_TO_FUNDER | The wallet sent money straight back to the wallet that funded it. |
| rc-12 | caution | ROUND_TRIP_TO_FUNDER | The wallet sent money straight back to the wallet that funded it. |
| rc-17 | caution | ROUND_TRIP_TO_FUNDER | The wallet sent money straight back to the wallet that funded it. |
None of them got "do not pay". A round trip on its own gives "caution", which is a false alarm all the same.
Counted from data/real-set/results.json
Held-back cases it lost
4 of 30 held-back cases where the checker with memory did worse than the plain AI or got it wrong
| Case | What happened |
|---|---|
| rs-15 | The checker found no link and said "pay". The shared transactions aren't in the oldest 50 it reads. |
| rs-20 | The AI said "do not pay". The checker found the shared fee payer and said "pay". The wallet reported as bad has a different fee payer in the checker's reading, so the report was saved under another address and memory found nothing for this wallet. |
| rc-12 | An ordinary wallet that sent money back to whoever funded it. The checker said "caution" and the AI said "pay". |
| rs-19 | The AI named the wallet that funded this group. The checker didn't, though memory still stopped the payment. |
Counted from bench/results/real.json
Pre-scan memories that did not come back
29 of 200 pre-scan memories not counted as recalled
| How many | What happened |
|---|---|
| 23 | recall worked but returned no pre-scan verdict for this address |
| 2 | Walrus Memory request timed out after 15000ms during POST /api/recall. The relayer accepted the connection but did not answer in time. |
| 2 | fetch failed |
| 2 | Unexpected status code: 404 |
They are stored in Walrus Memory. They are left out of the count because asking for them by exact address did not bring back the stored verdict.
Counted from data/prescan/verified.json
Stress test (synthetic, wallets I built)
Missed 0 of 65 wallets, flagged 0 of 16 ordinary wallets. These links were built by me and are easy, so this says little about real wallets.
Counted from bench/results/stress.json