ratemy.sh

ratemy.sh · evidence-backed release review

Writing it was the easy part.

AI builds your product in a weekend. Nobody tells you whether you'd dare put real users, real data, or real money behind it. These five skills do — each auditing one layer of what you just built, all sharing one evidence contract and one willingness to say NOT READY.

Audit · R-014 target: real-money ref: sha256:9f3c…e2a1

Issue list

BLOCKER · F-002Cancellation does not stop tool calls — the agent keeps spending money after the user presses Stop.

HIGH · F-005Retry re-executes a non-idempotent charge — one approval can bill twice.

To verify

UNKNOWN · U-003Tenant isolation has not been exercised — one user's data may reach another.

Evidence lanes

deterministic-checksFAIL
critical-journey-e2ePASS
probabilistic-evalUNVERIFIED
continuous-evidenceN/A — no deployment yet
Maximum safe target: internal-demo
Blocking gates: duplicate-real-charge, runaway-execution
NOT READY

The five

Five layers. One evidence contract.

Every skill is a standalone plugin for Claude Code, Codex, Cursor, and any Agent Skills client. Install only the layer you're shipping.

Why they say no when others say yes

The review is a contract, not a vibe.

All five share these clauses. They are the reason a verdict here is worth reading.

§ 1

No score averages away a veto.

Hard gates — authority bypass, cross-boundary leakage, irreversible data loss, duplicate real charges, fabricated completion — block the release regardless of the weighted average. Accepting the risk doesn't turn a gate into a pass.

§ 2

Evidence has states.

Every claim is labeled E3 reproduced, E2 instrumented, E1 static, or E0 unverified. A reachable code defect is real, but its runtime consequence stays labeled as inference until it's exercised.

§ 3

A plausible diff is not a fix.

Findings advance to verified-fixed only on fresh passing evidence from an independent retest context — which also audits the fix code itself. The pass that wrote the fix never grades its own work.

§ 4

Green CI proves structure, not behavior.

Repository checks and schema validation cannot masquerade as a critical-journey run. Public or privileged targets require fresh runtime evidence on the exact build under review — a mutable latest is not an identity.

Which one

Pick by what you're about to ship.

"I vibe-coded an app and I want to charge for it."

ratemycode at the real-stakes level. It verifies payment invariants, retries, authorization, recovery, and monitoring before you take a cent.

"The API is mine and it's about to face real traffic."

ratemyserver. Tenant isolation, transactions, idempotency, backpressure, migrations, and whether a restore actually restores.

"I built an agent that can touch real accounts."

ratemyagent for what it decides, ratemyharness for whether the runtime can stop it. Both, if the answer matters.

"I wrote a Skill and I want to publish it."

ratemyskill. Trigger precision, with-skill versus without-skill uplift, injection resistance, and package integrity.