Grading on a curve: How to assess a pentest

Source: The Red Canary Blog: Information Security Insights

Author: Brian Donohue

URL: https://redcanary.com/blog/testing-and-validation/pentesting/

ONE SENTENCE SUMMARY:

Effective defense disrupts multi-stage attack chains by prioritizing high-fidelity, intent-rich behaviors, not exhaustive detection of every atomic action.

MAIN POINTS:

  1. Breaches result from campaigns of sequential actions, not single attacker successes.
  2. Detecting any critical step can hinder adversaries, evict threats, and prevent incidents.
  3. Depth and redundancy help, but complete coverage of all behaviors isn’t required.
  4. Testing is often misused as an exhaustive scorecard demanding alerts for every action.
  5. Real threats are adaptive, persistent campaigns; emulations are usually partial and constrained.
  6. Defensive focus should be “detect to disrupt” by breaking the attack chain early.
  7. Early or mid-chain detection can outperform noisy reconnaissance detection in outcomes.
  8. Isolated atomic events lack context; patterns reveal malicious intent and progression.
  9. High-fidelity TTPs like LSASS dumping and persistence provide reliable intervention points.
  10. Over-alerting to catch everything increases false positives and reduces analyst effectiveness.

TAKEAWAYS:

  1. Measure success by stopping attacker objectives, not by maximizing alert counts.
  2. Prioritize chokepoints and intent-rich techniques that reliably indicate malicious progression.
  3. Treat pentests and red teams as validation inputs, not comprehensive real-threat replicas.
  4. Use contextual correlation to distinguish benign activity from adversary behavior patterns.
  5. Expand coverage thoughtfully to scale, avoiding alert floodgates that bury true threats.