📊 Metrics & KPIs for Consumer ATO Prevention (CAP)

Use this guide to define success, build dashboards, and report impact. Start with a short baseline, set targets, and iterate.

🎯 What to measure

Category	KPI	Why it matters
Outcomes	ATO Incidents ↓	The core “bad thing” you’re trying to stop.
	ATO Prevented ↑	Proactive blocks/resets that stopped credential-based account abuse.
Efficiency	Time to Detect (TTD) ↓	Time from new exposure observed → user flagged.
	Time to Remediate (TTR) ↓	Time from user flagged → action completed (reset/step-up).
Coverage	Population Evaluated ↑	% of active users evaluated by CAP controls.
	Identifier Coverage ↑	% of users with matchable identifiers (email/phone/username/IP).
Quality	Policy Precision ↑	% of CAP actions on accounts with confirmed exposure.
	False Positive Rate ↓	% of CAP actions later reversed as “not risky.”
User Experience	Friction Rate ↔︎	% of logins/registrations impacted by CAP actions.
	Password Reset Completion ↑	% of forced resets completed within SLA (e.g., 24h).

Baseline: 2–4 weeks pre-enforcement to record ATO incidents, fraud losses, and login/registration conversion.
Targets (first 90 days):
- ATO incidents: ↓ 20–40% (credential-based)
- TTR: ↓ 30–50% (from flag to reset/step-up)
- Friction rate: ≤ 1–3% of total logins (tune to business tolerance)
Cadence: Weekly ops review; monthly KPI roll-up; quarterly ROI update.

Aim for steady improvements, not perfection on day one. Tighten thresholds as precision improves.

Adjust to your data model; keep formulas stable over time for trend integrity.

1) ATO Incidents (credential-based) Number of confirmed account takeovers that used exposed/reused credentials during the period.

2) ATO Prevented Prevented = ForcedResets_PreAccess + High-Risk_Login_Blocks + Registration_Blocks

3) Time to Detect (TTD) TTD (median) = time(user_flagged_by_CAP) − time(exposure_available)

4) Time to Remediate (TTR) TTR (median) = time(action_completed) − time(user_flagged_by_CAP)

5) Population Evaluated Evaluated % = distinct(users_evaluated_by_CAP) / distinct(active_users)

6) Policy Precision Precision % = CAP_Actions_On_Exposed / Total_CAP_Actions

7) False Positive Rate (FPR) FPR % = Reversed_CAP_Actions / Total_CAP_Actions

8) Friction Rate Friction % = (Forced_Resets + Step-Up_Auth_Prompts) / Total_Logins

9) Password Reset Completion Completion % = Resets_Completed_within_SLA / Resets_Triggered

Capture these events (or equivalents) to power KPIs:

ExposureEvaluated (user_id, identifiers, exposure_source: breach/malware/phished, exposure_count, severity, ts)
LoginEvaluated (user_id, credential_risk, action_applied, ts)
RegistrationEvaluated (user_id/email, action_applied, ts)
ActionApplied (type: forced_reset | step_up | registration_block, reason, ts)
ResetCompleted (user_id, ts)
ATOConfirmed (user_id, ts, vector: credential)
Conversion (type: login_success | registration_complete, ts)

Keep identifiers consistent so you can join SpyCloud results with app/fraud/IdP logs.

📈 Executive top line (always-on)

🧰 Operations (daily)

🧪 Policy tuning

Segment reports to see where to tighten and where to lighten controls.

KPI	This Month	Prev Month	Δ	Target	Status
ATO Incidents (cred)	42	61	−31%	−20%	✅
ATO Prevented	3,210	2,780	+15%	+10%	✅
TTR (median)	3.6h	6.1h	−41%	≤ 4h	✅
Precision	92.4%	88.7%	+3.7pp	≥ 90%	✅
FPR	1.3%	1.9%	−0.6pp	≤ 2%	✅
Friction	1.8%	1.7%	+0.1pp	≤ 2%	⚠️
Reset Completion (24h)	74%	69%	+5pp	≥ 75%	▢

pp = percentage points

Map exposure severity/counts to actions (tune to your risk tolerance):

Condition	Suggested Action
High severity or multiple recent exposures	Force password reset immediately
Medium severity, first-time exposure	Step-up auth + optional reset nudge
Low severity or stale exposure	Monitor; nudge user to refresh password
Repeat exposure within 30 days	Mandatory reset + education banner

Start conservative; widen enforcement as precision stabilizes.

ROI = (Avg Incident Cost × ATO Prevented) − (Ops Cost + Tooling Cost)

Report quarterly; socialize wins with Security, Fraud, and CX.

Fail-open async checks at login; sync checks at registration where policy requires.
Track attempt → success for forced resets (this powers TTR and completion).
Log policy route and reason for every action to enable precision/FPR reporting.
Revisit thresholds monthly; keep a changelog for experiments.

Keep it boring and repeatable. The program wins by consistency.