Security Logging, Monitoring & Observability — Deep Dive

CIPHER Training Module — Security Logging Architecture Last updated: 2026-03-14

---

Table of Contents

1. Logging Fundamentals
2. OWASP Logging Standards
3. OWASP Logging Vocabulary
4. Windows Audit Policy Configuration
5. Sysmon Event Types & Rule Writing
6. Linux auditd Rule Patterns
7. Cloud Provider Log Sources
8. Log Shipping Architecture
9. OSSEM Data Model
10. EVTX Analysis & Attack Samples
11. Tool-to-Event-ID Mapping
12. Minimum Viable Logging Checklist

---

1. Logging Fundamentals

Why Log (Security Perspective)

Logs serve four security functions:

1. Detection — identify attacks in progress or after the fact
2. Investigation — reconstruct what happened, when, and by whom
3. Compliance — satisfy regulatory and audit requirements (PCI DSS Req 10, HIPAA, GDPR Art. 30)
4. Deterrence — non-repudiation and accountability

What Makes a Log Entry Useful

Every log entry must answer four questions:

Question	Fields
When	UTC timestamp (ISO 8601), event time vs. log time, interaction/correlation ID
Where	Application ID/version, hostname, IP, service name, URL/endpoint, code location
Who	Source IP, user identity (authenticated or claimed), session ID (hashed), user type
What	Event type, severity, security relevance flag, action, affected object, result status, reason

What NOT to Log

Never log these — they create liability and attack surface:

• Passwords, authentication tokens, API keys, encryption keys
• Session IDs in cleartext (hash if needed for correlation)
• Credit card numbers, bank account details (PCI DSS)
• Health data, government IDs, SSNs (HIPAA, GDPR)
• Application source code
• Database connection strings
• Cryptographic material (private keys, certificates)
• Data users opted out of collecting

Handle with care (consider de-identification): names, email, phone numbers, file paths, internal network topology.

Attacks Against Logging Infrastructure

Attack Vector	Category	Mitigation
Log injection (CRLF, delimiter injection)	Integrity	Input sanitization, structured logging (JSON)
Log flooding (disk exhaustion)	Availability	Rate limiting, log rotation, separate partitions
Log tampering/deletion	Integrity	Append-only storage, WORM, hash chains, separate network segment
Credential harvesting from logs	Confidentiality	Never log secrets, access controls on log stores
Log forging (false identity)	Accountability	Authenticated log sources, TLS mutual auth
Performance degradation via logging	Availability	Async logging, sampling, buffering

Log Protection Requirements

At rest:

• Tamper detection (hash chains, digital signatures)
• Write-once media where possible
• Access controls with periodic review
• Encryption for sensitive log stores

In transit:

• TLS for all log shipping
• Mutual authentication between agents and collectors
• Integrity verification at destination

Retention:

• Define per regulatory requirement (PCI: 1 year accessible, HIPAA: 6 years, SOX: 7 years)
• Automated deletion after retention window
• Legal hold capability

---

2. OWASP Logging Standards

Source: OWASP Logging Cheat Sheet

Events to Always Log

Authentication & Access

• Login success and failure (with user ID)
• Authentication failures reaching threshold
• Account lockouts
• Password changes (success and failure)
• Token creation, revocation, reuse attempts

Authorization

• Access control failures
• Privilege escalation attempts
• Entitlement changes

Session Management

• Session creation, renewal, expiration
• Cookie/token modification attempts
• JWT validation failures

Input Validation

• Protocol violations
• Unacceptable encodings
• Invalid parameter names/values
• Output validation failures

Application Events

• Startup, shutdown, restart
• Configuration changes
• Logging initialization and failures
• Runtime errors, connectivity issues
• Third-party service failures

Higher-Risk Operations

• User administration (create, delete, modify privileges)
• System administration actions
• Default or break-glass account usage
• Sensitive data access (PII, financial, health)
• Encryption key use and rotation
• Data import/export operations
• File uploads
• Deserialization failures
• Backend TLS failures and certificate issues

Business Logic

• Sequence/ordering failures
• Out-of-context actions
• Rate limit violations
• Suspicious patterns (impossible travel, anomalous volumes)

Logging Architecture (OWASP Recommendation)

┌─────────────┐    ┌──────────────┐    ┌─────────────┐
│  FRONTEND   │    │  MIDDLEWARE   │    │   BACKEND   │
│  (Viewing)  │◄──►│ (Collection) │◄──►│  (Storage)  │
│             │    │              │    │             │
│ Log viewer  │    │ Aggregator   │    │ Write-once  │
│ Dashboards  │    │ Normalizer   │    │ Long-term   │
│ Search UI   │    │ Router       │    │ SIEM ingest │
└─────────────┘    └──────────────┘    └─────────────┘
     ▲                    ▲                    ▲
     │                    │                    │
  Separate           Separate            Separate
  network            network             network
  segment            segment             segment

Each tier runs on a separate network segment with distinct access controls.

Standard Log Formats

Format	Vendor	Use Case
CEF (Common Event Format)	ArcSight/Micro Focus	SIEM ingestion
LEEF (Log Event Extended Format)	IBM QRadar	SIEM ingestion
RFC 5424 (Syslog)	IETF	Universal transport
CLF/CLFS	W3C/Microsoft	Web server logs
JSON structured logging	Various	Modern applications
ECS (Elastic Common Schema)	Elastic	Elasticsearch stack

---

3. OWASP Logging Vocabulary

Source: OWASP Application Logging Vocabulary

A standardized event taxonomy for application-level security logging. Use these event names for consistent, machine-parseable security events across applications.

Authentication Events [AUTHN]

Event	Level	Description
authn_login_success[:userid]	INFO	Successful authentication
authn_login_successafterfail[:userid,retries]	INFO	Success after prior failures — track brute force that succeeded
authn_login_fail[:userid]	WARN	Failed authentication attempt
authn_login_fail_max[:userid,maxlimit]	WARN	Maximum retry threshold reached
authn_login_lock[:userid,reason]	WARN	Account locked (maxretries/suspicious/customer/other)
authn_password_change[:userid]	INFO	Password changed successfully
authn_password_change_fail[:userid]	CRITICAL	Password change attempt failed
authn_impossible_travel[:userid,region1,region2]	CRITICAL	Login from geographically impossible locations
authn_token_created[:userid,entitlements]	INFO	API/service token issued
authn_token_revoked[:userid,tokenid]	INFO	Token invalidated
authn_token_reuse[:userid,tokenid]	CRITICAL	Revoked token reuse attempt — active attack indicator
authn_token_delete[:appid]	WARN	Token permanently removed

Authorization Events [AUTHZ]

Event	Level	Description
authz_fail[:userid,resource]	CRITICAL	Unauthorized resource access attempt
authz_change[:userid,from,to]	WARN	User entitlements/role modified
authz_admin[:userid,event]	WARN	Privileged user activity — all admin actions logged

Cryptographic Events [CRYPT]

Event	Level	Description
crypt_decrypt_fail[:userid]	WARN	Decryption operation failed
crypt_encrypt_fail[:userid]	WARN	Encryption operation failed

Excessive Use [EXCESS]

Event	Level	Description
excess_rate_limit_exceeded[:userid,max]	WARN	Rate limit or quota exceeded

File Upload Events [UPLOAD]

Event	Level	Description
upload_complete[:userid,filename,type]	INFO	Upload finished
upload_stored[:filename,from,to]	INFO	File moved to storage location
upload_validation[:filename,(validator):(result)]	INFO/CRITICAL	Validation result (virus scan, magic bytes, etc.)
upload_delete[:userid,fileid]	INFO	Uploaded file deleted

Input Validation Events [INPUT]

Event	Level	Description
input_validation_fail[:fields,userid]	WARN	Server-side validation failure
input_validation_discrete_fail[:field,userid]	WARN	Invalid value for constrained field (dropdown/enum) — may indicate parameter tampering

Malicious Behavior Events [MALICIOUS]

Event	Level	Description
malicious_excess_404[:userid|IP,useragent]	WARN	Excessive requests for non-existent resources — directory enumeration
malicious_extraneous[:userid|IP,inputname,useragent]	CRITICAL	Unexpected form field submitted — parameter injection attempt
malicious_attack_tool[:userid|IP,toolname,useragent]	CRITICAL	Known attack tool signature detected (sqlmap, nikto, etc.)
malicious_sqli[:userid|IP,parameter,ruleid,useragent]	CRITICAL	SQL injection pattern detected
malicious_cors[:userid|IP,useragent,referer]	CRITICAL	Unauthorized cross-origin request
malicious_direct_reference[:userid|IP,useragent]	CRITICAL	IDOR attempt — accessing other users' objects

Privilege Events [PRIVILEGE]

Event	Level	Description
privilege_permissions_changed[:userid,object,fromlevel,tolevel]	WARN	Object access control modified

Sensitive Data Events [DATA]

Event	Level	Description
sensitive_create[:userid,object]	WARN	New sensitive data created
sensitive_read[:userid,object]	WARN	Sensitive data accessed
sensitive_update[:userid,object]	WARN	Sensitive data modified
sensitive_delete[:userid,object]	WARN	Sensitive data marked for deletion

Sequence Events [SEQUENCE]

Event	Level	Description
sequence_fail[:userid]	CRITICAL	Business logic attack — out-of-order workflow steps

Session Events [SESSION]

Event	Level	Description
session_created[:userid]	INFO	New authenticated session
session_renewed[:userid]	INFO	Session extended/refreshed
session_expired[:userid,reason]	INFO	Session ended (logout/timeout/revoked)
session_use_after_expire[:userid]	CRITICAL	Expired session token reuse — session hijacking indicator

System Events [SYS]

Event	Level	Description
sys_startup[:userid]	WARN	Application/service started
sys_shutdown[:userid]	WARN	Application/service stopped
sys_restart[:userid]	WARN	Application/service restarted
sys_crash[:reason]	WARN	Unexpected termination
sys_monitor_disabled[:userid,monitor]	WARN	Security monitoring agent stopped — tampering indicator
sys_monitor_enabled[:userid,monitor]	WARN	Security monitoring agent started

User Management Events [USER]

Event	Level	Description
user_created[:userid,newuserid,attributes]	WARN	New account provisioned
user_updated[:userid,onuserid,attributes]	WARN	Account attributes modified
user_archived[:userid,onuserid]	WARN	Account deactivated (preferred over deletion)
user_deleted[:userid,onuserid]	WARN	Account permanently removed

MCP Server Events [MCP]

Event	Level	Description
mcp_prompt_injection[:userid]	WARN	Prompt injection indicators in AI/LLM context
mcp_resource_exhaustion[:userid]	WARN	Resource depletion attack pattern
mcp_tool_poisoning[:userid]	WARN	Tampered or malicious tool detected

Standard Log Entry Format

All entries use ISO 8601 timestamps with UTC offset:

{
  "datetime": "2026-03-14T10:30:00.000Z",
  "appid": "myapp-prod",
  "event": "authn_login_fail",
  "level": "WARN",
  "description": "Failed login for user jsmith from 10.0.1.42",
  "useragent": "Mozilla/5.0...",
  "source_ip": "10.0.1.42",
  "host_ip": "10.0.2.10",
  "hostname": "auth-server-01",
  "protocol": "https",
  "port": 443,
  "request_uri": "/api/v1/login",
  "request_method": "POST",
  "region": "us-east-1",
  "geo": "39.7392,-104.9903"
}

---

4. Windows Audit Policy Configuration

Advanced Audit Policy Categories

Configure via secpol.msc → Advanced Audit Policy Configuration, or via GPO at: Computer Configuration\Policies\Windows Settings\Security Settings\Advanced Audit Policy Configuration

Account Logon

Subcategory	Recommended	Key Event IDs	Detection Value
Credential Validation	Success, Failure	4774, 4775, 4776, 4777	NTLM authentication monitoring
Kerberos Authentication Service	Success, Failure	4768, 4771, 4772	TGT requests, Kerberoasting detection
Kerberos Service Ticket Operations	Success, Failure	4769, 4770	Service ticket requests, Golden/Silver ticket detection
Other Account Logon Events	Success, Failure	4649, 4778, 4779, 4800, 4801	Replay attacks, session reconnects

Account Management

Subcategory	Recommended	Key Event IDs	Detection Value
Computer Account Management	Success	4741, 4742, 4743	Machine account manipulation (RBCD attacks)
Distribution Group Management	Success	4749-4753	Distribution group changes
Other Account Management Events	Success	4782, 4793	Password policy bypass attempts
Security Group Management	Success	4727, 4728, 4729, 4730, 4731, 4732, 4733, 4734, 4735, 4737, 4754-4758	Group membership changes — privilege escalation
User Account Management	Success, Failure	4720, 4722, 4723, 4724, 4725, 4726, 4738, 4740, 4765, 4766, 4767, 4780, 4781, 4794	Account creation, modification, lockout, deletion

Detailed Tracking

Subcategory	Recommended	Key Event IDs	Detection Value
DPAPI Activity	Success, Failure	4692, 4693, 4694, 4695	Credential/key theft via DPAPI
PNP Activity	Success	6416, 6419, 6420, 6421, 6422, 6423, 6424	USB device connections
Process Creation	Success	4688	Critical — command line logging (enable "Include command line in process creation events")
Process Termination	Success	4689	Process lifecycle tracking
RPC Events	Success, Failure	5712	Remote procedure call monitoring
Token Right Adjusted	Success	4703	Token privilege manipulation

DS Access (Domain Controllers)

Subcategory	Recommended	Key Event IDs	Detection Value
Detailed Directory Service Replication	Success, Failure	4928, 4929, 4930, 4931, 4934, 4935, 4936, 4937	DCSync detection
Directory Service Access	Success, Failure	4661, 4662	AD object access monitoring
Directory Service Changes	Success	5136, 5137, 5138, 5139, 5141	AD modifications — GPO changes, OU manipulation
Directory Service Replication	Success, Failure	4932, 4933	Replication monitoring — DCSync indicator

Logon/Logoff

Subcategory	Recommended	Key Event IDs	Detection Value
Account Lockout	Failure	4625 (with sub-status)	Brute force detection
Group Membership	Success	4627	Group membership at logon time
IPsec Extended Mode	No audit	4978, 4979	IPsec troubleshooting
IPsec Main Mode	No audit	4646, 4650, 4651	IPsec troubleshooting
IPsec Quick Mode	No audit	4977	IPsec troubleshooting
Logoff	Success	4634, 4647	Session duration tracking
Logon	Success, Failure	4624, 4625, 4648	Critical — all logon events, logon type tracking
Network Policy Server	Success, Failure	6272-6280	RADIUS/NPS authentication
Other Logon/Logoff Events	Success, Failure	4649, 4778, 4779, 4800, 4801, 4802, 4803, 5378, 5632, 5633	Session reconnects, screen lock
Special Logon	Success	4964, 4672	Critical — admin logon, special privileges assigned
User/Device Claims	Success	4626	Claims-based access

Object Access

Subcategory	Recommended	Key Event IDs	Detection Value
Application Generated	Success, Failure	4665-4668	Application-specific audit events
Certification Services	Success, Failure	4868-4898	Certificate operations — ADCS abuse (ESC1-ESC8)
Detailed File Share	Failure	5145	Detailed share access — lateral movement detection
File Share	Success, Failure	5140, 5142, 5143, 5144	Share access — critical for lateral movement
File System	Success, Failure	4656, 4658, 4660, 4663, 4664	File access auditing (requires SACL)
Filtering Platform Connection	Failure	5031, 5150-5159	Firewall connection events
Filtering Platform Packet Drop	Failure	5152, 5153	Firewall drops
Handle Manipulation	Success	4658, 4690	Handle duplication tracking
Kernel Object	Success, Failure	4659, 4660, 4661, 4663	Kernel object access
Other Object Access	Success, Failure	4671, 4691	Miscellaneous object events
Registry	Success, Failure	4657, 4660, 4663	Registry modification (requires SACL)
Removable Storage	Success, Failure	4656, 4658, 4663	USB/removable media access
SAM	Success, Failure	4661	SAM database queries — credential dumping indicator

Policy Change

Subcategory	Recommended	Key Event IDs	Detection Value
Audit Policy Change	Success	4715, 4719, 4817, 4902, 4904, 4905, 4906, 4907, 4908, 4912	Critical — audit policy tampering detection
Authentication Policy Change	Success	4706, 4707, 4713, 4716, 4717, 4718, 4739, 4864, 4865, 4866, 4867	Authentication/trust changes
Authorization Policy Change	Success	4703, 4704, 4705, 4670, 4911, 4913	User rights assignment changes
Filtering Platform Policy Change	Failure	Various	Firewall policy changes
MPSSVC Rule-Level Policy Change	Success, Failure	4944-4958	Windows Firewall rule changes
Other Policy Change Events	Failure	4714, 4819, 4826	Miscellaneous policy changes

Privilege Use

Subcategory	Recommended	Key Event IDs	Detection Value
Non Sensitive Privilege Use	No audit	4673, 4674, 4985	Very noisy — enable selectively
Sensitive Privilege Use	Success, Failure	4673, 4674, 4985	Debug privilege, impersonation, backup privilege use
Other Privilege Use Events	No audit	—	Reserved

System

Subcategory	Recommended	Key Event IDs	Detection Value
IPsec Driver	Success, Failure	4960-4968	IPsec driver events
Other System Events	Success, Failure	5024, 5025, 5027, 5028, 5029, 5030, 5032, 5033, 5034, 5035, 5037	Firewall service events
Security State Change	Success	4608, 4616, 4621	System startup, time change — anti-forensics detection
Security System Extension	Success	4610, 4611, 4614, 4622, 4697	Critical — security package/driver loaded, service installed
System Integrity	Success, Failure	4612, 4615, 4618, 4816, 5038, 5056, 5057, 5058, 5059, 5060, 5061, 5062	Code integrity failures, cryptographic operations

Critical GPO Settings Beyond Audit Policy

# Enable command line in process creation events (CRITICAL)
Computer Configuration\Administrative Templates\System\Audit Process Creation
→ "Include command line in process creation events" = Enabled

# PowerShell logging
Computer Configuration\Administrative Templates\Windows Components\Windows PowerShell
→ "Turn on Module Logging" = Enabled (all modules: *)
→ "Turn on PowerShell Script Block Logging" = Enabled
→ "Turn on PowerShell Transcription Logging" = Enabled (with output directory)

# WinRM logging
Computer Configuration\Administrative Templates\Windows Components\Windows Remote Management
→ Enable logging

Key Windows Security Event IDs — Quick Reference

Event ID	Description	Detection Use
1102	Audit log cleared	Anti-forensics
4624	Successful logon	Lateral movement (Type 3, 10), service accounts (Type 5)
4625	Failed logon	Brute force, password spray
4627	Group membership at logon	Privilege tracking
4634	Logoff	Session duration
4648	Logon with explicit credentials	Credential use / runas
4656	Handle requested to object	File/registry access
4662	Operation on AD object	DCSync (replicating directory changes)
4663	Object access attempt	File access auditing
4672	Special privileges assigned	Admin logon detection
4688	Process created	Command execution (with command line)
4689	Process terminated	Process lifecycle
4697	Service installed	Persistence, lateral movement
4698	Scheduled task created	Persistence
4699	Scheduled task deleted	Cleanup/anti-forensics
4700	Scheduled task enabled	Persistence
4702	Scheduled task updated	Persistence modification
4703	Token privileges adjusted	Privilege escalation
4719	System audit policy changed	Audit tampering
4720	User account created	Persistence
4722	User account enabled	Account manipulation
4724	Password reset attempt	Account takeover
4725	User account disabled	Destructive action
4726	User account deleted	Anti-forensics
4728	Member added to security-enabled global group	Privilege escalation
4732	Member added to security-enabled local group	Local admin addition
4738	User account changed	Account modification
4740	User account locked out	Brute force indicator
4756	Member added to universal group	Domain-wide privilege change
4768	Kerberos TGT requested	Authentication monitoring
4769	Kerberos service ticket requested	Kerberoasting (RC4 encryption type 0x17)
4770	Kerberos service ticket renewed	Service ticket tracking
4771	Kerberos pre-authentication failed	Password spray
4776	NTLM credential validation	NTLM relay, pass-the-hash
5136	Directory service object modified	GPO modification, AD attribute changes
5140	Network share accessed	Lateral movement
5145	Detailed file share access	SMB access with specific file paths
7045	Service installed (System log)	Persistence, PsExec
4104	PowerShell Script Block (Microsoft-Windows-PowerShell/Operational)	Malicious PowerShell
4103	PowerShell Module Logging	PowerShell execution tracking

Logon Type Reference

Type	Name	Significance
2	Interactive	Console logon
3	Network	SMB, net use — lateral movement indicator
4	Batch	Scheduled tasks
5	Service	Service account logon
7	Unlock	Workstation unlock
8	NetworkCleartext	IIS Basic auth — password exposed
9	NewCredentials	RunAs /netonly — alternate credentials
10	RemoteInteractive	RDP
11	CachedInteractive	Cached domain credentials

---

5. Sysmon Event Types & Rule Writing

Installation & Management

# Install with config
sysmon64.exe -accepteula -i sysmonconfig.xml

# Update running config
sysmon64.exe -c sysmonconfig.xml

# Print schema (shows all fields per event type)
sysmon64.exe -s

# Uninstall
sysmon64.exe -u

# Sysmon for Linux
# https://github.com/microsoft/SysmonForLinux
sudo dpkg -i sysmonforlinux_*.deb
sudo sysmon -accepteula -i sysmonconfig-linux.xml

Event log location: Applications and Services Logs/Microsoft/Windows/Sysmon/Operational

Complete Event ID Reference

ID	Tag	Event	Key Fields	Detection Value
1	ProcessCreate	Process creation	Image, CommandLine, ParentImage, ParentCommandLine, User, Hashes, ProcessGuid, LogonGuid, IntegrityLevel, CurrentDirectory	Critical — command execution, process trees
2	FileCreateTime	File creation time changed	TargetFilename, CreationUtcTime, PreviousCreationUtcTime	Timestomping detection
3	NetworkConnect	Network connection	Image, SourceIp, SourcePort, DestinationIp, DestinationPort, DestinationHostname, Protocol	C2 communication, lateral movement (disabled by default)
4	—	Sysmon service state change	State (Started/Stopped)	Cannot be filtered — service tampering
5	ProcessTerminate	Process terminated	Image, ProcessGuid	Process lifecycle
6	DriverLoad	Driver loaded	ImageLoaded, Hashes, Signature, Signed	Rootkit/driver-level persistence
7	ImageLoad	DLL/image loaded	ImageLoaded, Image, Hashes, Signature, Signed	DLL side-loading, hijacking (noisy — filter carefully)
8	CreateRemoteThread	Remote thread created	SourceImage, TargetImage, StartAddress, StartModule, StartFunction	Critical — process injection
9	RawAccessRead	Raw disk access	Image, Device	MBR reading, direct disk access for exfil
10	ProcessAccess	Process memory accessed	SourceImage, TargetImage, GrantedAccess, CallTrace	Critical — LSASS credential dumping (target: lsass.exe, access: 0x1010, 0x1410, 0x1438, 0x143a)
11	FileCreate	File created/overwritten	TargetFilename, Image	Malware drops, autostart locations
12	RegistryEvent	Registry key/value create/delete	EventType, TargetObject, Image	Persistence (Run keys, services)
13	RegistryEvent	Registry value set	TargetObject, Details, Image	Registry-based configuration changes
14	RegistryEvent	Registry key/value renamed	TargetObject, NewName	Registry manipulation
15	FileCreateStreamHash	Named file stream created	TargetFilename, Hash, Contents	ADS (Alternate Data Streams), Mark-of-the-Web
16	—	Sysmon config change	Configuration, ConfigurationFileHash	Cannot be filtered — config tampering
17	PipeEvent	Named pipe created	PipeName, Image	Cobalt Strike (default pipes: \postex_*, \msagent_*, \MSSE-*-server)
18	PipeEvent	Named pipe connected	PipeName, Image	IPC communication, C2 channels
19	WmiEvent	WMI filter registered	EventNamespace, Name, Query	WMI persistence
20	WmiEvent	WMI consumer registered	Name, Type, Destination	WMI persistence
21	WmiEvent	WMI consumer-to-filter binding	Consumer, Filter	WMI persistence chain completion
22	DNSEvent	DNS query	QueryName, QueryResults, Image	DNS-based C2, domain reputation
23	FileDelete	File deleted (archived)	TargetFilename, Image, Hashes, IsExecutable	Evidence destruction, malware cleanup (file preserved to ArchiveDirectory)
24	ClipboardChange	Clipboard content changed	ClientInfo	Clipboard data theft
25	ProcessTampering	Process image change detected	Image, Type	Critical — process hollowing, herpaderping
26	FileDeleteDetected	File deleted (logged only)	TargetFilename, Image, Hashes	Same as 23 but without archiving
27	FileBlockExecutable	Executable file creation blocked	TargetFilename, Image	Proactive blocking of PE drops
28	FileBlockShredding	File shredding blocked	TargetFilename, Image	Anti-forensics prevention (SDelete)
29	FileExecutableDetected	Executable file created	TargetFilename, Image, Hashes	PE file creation monitoring
255	Error	Sysmon error	—	Internal Sysmon errors

Configuration Schema

<Sysmon schemaversion="4.82">
  <!-- Global configuration -->
  <HashAlgorithms>sha256,imphash</HashAlgorithms>
  <CheckRevocation>true</CheckRevocation>
  <DnsLookup>false</DnsLookup>
  <ArchiveDirectory>SysmonArchive</ArchiveDirectory>
  <CopyOnDeletePE>true</CopyOnDeletePE>

  <EventFiltering>
    <!-- RuleGroup with explicit AND/OR logic -->
    <RuleGroup name="Credential Dumping" groupRelation="or">
      <ProcessAccess onmatch="include">
        <TargetImage condition="is">C:\Windows\System32\lsass.exe</TargetImage>
      </ProcessAccess>
    </RuleGroup>

    <!-- Standard include/exclude (exclude overrides include) -->
    <ProcessCreate onmatch="exclude">
      <Image condition="is">C:\Windows\System32\svchost.exe</Image>
    </ProcessCreate>

    <!-- Named rules for tracking which rule matched -->
    <FileCreate onmatch="include">
      <TargetFilename name="Startup Folder Drop" condition="contains">\Start Menu\Programs\Startup\</TargetFilename>
      <TargetFilename name="Temp EXE Drop" condition="end with">.exe</TargetFilename>
    </FileCreate>
  </EventFiltering>
</Sysmon>

Filter Condition Operators

Condition	Description
is	Exact match (default)
is any	Matches any of ;-delimited values
is not	Not equal
contains	Substring match
contains any	Contains any of ;-delimited values
contains all	Contains all of ;-delimited values
excludes	Does not contain substring
excludes any	Does not contain any of ;-delimited values
excludes all	Does not contain all of ;-delimited values
begin with	Prefix match
end with	Suffix match
not begin with	Does not start with
not end with	Does not end with
less than	Lexicographic less than
more than	Lexicographic greater than
image	Match image name or full path (e.g., lsass.exe matches C:\Windows\System32\lsass.exe)

All conditions are case insensitive.

Rule Logic

• Same field name rules within a filter: OR (any match triggers)
• Different field name rules within a filter: AND (all must match)
• RuleGroup with groupRelation="and": forces AND across all rules
• RuleGroup with groupRelation="or": forces OR across all rules
• When both include and exclude exist for same event type: exclude takes precedence

Community Configs Comparison

Config	Philosophy	Best For
SwiftOnSecurity/sysmon-config	Conservative noise reduction, heavily commented, baseline	Production environments, getting started
Neo23x0/sysmon-config (Florian Roth)	Detection-focused, merged 40+ community PRs, includes blocking rules variant	Threat detection, includes Cobalt Strike/PrintNightmare/HiveNightmare patterns
olafhartong/sysmon-modular	Modular per-event-type rules, ATT&CK mapped, merge scripts	Custom configs, advanced users, research

High-Value Sysmon Detection Rules

LSASS Credential Dumping (Event ID 10)

<RuleGroup groupRelation="or">
  <ProcessAccess onmatch="include">
    <TargetImage name="LSASS Access" condition="is">C:\Windows\System32\lsass.exe</TargetImage>
  </ProcessAccess>
  <ProcessAccess onmatch="exclude">
    <!-- Legitimate LSASS accessors -->
    <SourceImage condition="is">C:\Windows\System32\svchost.exe</SourceImage>
    <SourceImage condition="is">C:\Windows\System32\lsm.exe</SourceImage>
    <SourceImage condition="is">C:\Windows\System32\csrss.exe</SourceImage>
    <SourceImage condition="is">C:\Windows\System32\wininit.exe</SourceImage>
  </ProcessAccess>
</RuleGroup>

Cobalt Strike Named Pipes (Event ID 17)

<PipeEvent onmatch="include">
  <PipeName name="CobaltStrike Pipe" condition="begin with">\postex_</PipeName>
  <PipeName name="CobaltStrike Pipe" condition="begin with">\msagent_</PipeName>
  <PipeName name="CobaltStrike Pipe" condition="begin with">\MSSE-</PipeName>
  <PipeName name="PsExec Pipe" condition="begin with">\PSEXESVC</PipeName>
</PipeEvent>

Suspicious Process Creation (Event ID 1)

<ProcessCreate onmatch="include">
  <!-- LOLBAS execution -->
  <Image name="LOLBAS certutil" condition="image">certutil.exe</Image>
  <Image name="LOLBAS mshta" condition="image">mshta.exe</Image>
  <Image name="LOLBAS regsvr32" condition="image">regsvr32.exe</Image>
  <Image name="LOLBAS rundll32" condition="image">rundll32.exe</Image>
  <Image name="LOLBAS msiexec" condition="image">msiexec.exe</Image>
  <Image name="LOLBAS bitsadmin" condition="image">bitsadmin.exe</Image>
  <!-- PowerShell encoded commands -->
  <CommandLine name="Encoded PowerShell" condition="contains">-enc</CommandLine>
  <CommandLine name="Encoded PowerShell" condition="contains">-EncodedCommand</CommandLine>
  <!-- Suspicious parent-child -->
  <ParentImage name="Office spawning cmd" condition="contains any">winword.exe;excel.exe;powerpnt.exe</ParentImage>
</ProcessCreate>

---

6. Linux auditd Rule Patterns

Architecture

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Kernel Audit │────►│   auditd     │────►│  Log Files   │
│  Framework   │     │  (Daemon)    │     │ /var/log/    │
│              │     │              │     │  audit/      │
│  - syscalls  │     │  - audisp    │     │  audit.log   │
│  - file ops  │     │  (plugins)   │     │              │
│  - network   │     │              │     │  - ausearch  │
│  - process   │     │  - audispd   │     │  - aureport  │
└──────────────┘     └──────────────┘     └──────────────┘

Configuration Files

• /etc/audit/auditd.conf — daemon configuration
• /etc/audit/rules.d/*.rules — persistent rules (loaded at boot)
• auditctl — runtime rule management

auditd.conf Key Settings

# /etc/audit/auditd.conf
log_file = /var/log/audit/audit.log
log_format = ENRICHED          # Include resolved UIDs/GIDs
log_group = adm                # Group that can read logs
num_logs = 10                  # Number of rotated logs to keep
max_log_file = 100             # Max log file size in MB
max_log_file_action = ROTATE   # Action when max size reached
space_left = 200               # MB remaining before space_left_action
space_left_action = email      # Alert when disk space low
admin_space_left_action = halt # Stop system if critically low (extreme)
disk_full_action = HALT        # Action when disk full
disk_error_action = HALT       # Action on disk error
flush = INCREMENTAL_ASYNC      # Performance vs. reliability tradeoff
freq = 50                      # Flush frequency for INCREMENTAL_ASYNC
name_format = HOSTNAME         # Node identifier format

Rule Types

1. Control rules — configure audit system behavior
2. File system rules (-w) — watch files/directories for access
3. System call rules (-a) — audit specific syscalls

Essential auditd Rules

# /etc/audit/rules.d/security.rules

## ============================================
## Control Rules
## ============================================

# Delete all existing rules (clean start)
-D

# Set buffer size (increase for busy systems)
-b 8192

# Set failure mode (1=printk, 2=panic)
-f 1

# Rate limit (0=unlimited, careful with this)
-r 0

## ============================================
## Self-Auditing (audit the audit system)
## ============================================

# Watch audit configuration changes
-w /etc/audit/ -p wa -k audit_config
-w /etc/audisp/ -p wa -k audit_config
-w /etc/libaudit.conf -p wa -k audit_config
-w /sbin/auditctl -p x -k audit_tools
-w /sbin/auditd -p x -k audit_tools
-w /usr/sbin/augenrules -p x -k audit_tools

## ============================================
## Identity and Authentication
## ============================================

# User/group modification
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/gshadow -p wa -k identity
-w /etc/security/opasswd -p wa -k identity

# PAM configuration
-w /etc/pam.d/ -p wa -k pam_config
-w /etc/security/ -p wa -k pam_config

# SSH configuration
-w /etc/ssh/sshd_config -p wa -k sshd_config
-w /etc/ssh/sshd_config.d/ -p wa -k sshd_config
-w /root/.ssh/ -p wa -k ssh_root

# Login configuration
-w /etc/login.defs -p wa -k login_config
-w /etc/securetty -p wa -k login_config
-w /etc/faillog -p wa -k login_config
-w /etc/lastlog -p wa -k login_config
-w /var/log/faillog -p wa -k login_config
-w /var/log/lastlog -p wa -k login_config
-w /var/log/tallylog -p wa -k login_config

## ============================================
## System Configuration
## ============================================

# Network configuration changes
-w /etc/hosts -p wa -k network_config
-w /etc/hostname -p wa -k network_config
-w /etc/resolv.conf -p wa -k network_config
-w /etc/sysconfig/network -p wa -k network_config
-w /etc/NetworkManager/ -p wa -k network_config
-w /etc/netplan/ -p wa -k network_config

# Cron/scheduled tasks
-w /etc/crontab -p wa -k cron
-w /etc/cron.d/ -p wa -k cron
-w /etc/cron.daily/ -p wa -k cron
-w /etc/cron.hourly/ -p wa -k cron
-w /etc/cron.monthly/ -p wa -k cron
-w /etc/cron.weekly/ -p wa -k cron
-w /var/spool/cron/ -p wa -k cron
-w /etc/anacrontab -p wa -k cron

# Systemd service files
-w /etc/systemd/ -p wa -k systemd
-w /usr/lib/systemd/ -p wa -k systemd
-w /lib/systemd/ -p wa -k systemd

# Kernel modules
-w /etc/modprobe.d/ -p wa -k kernel_modules
-w /etc/modules -p wa -k kernel_modules
-w /etc/modules-load.d/ -p wa -k kernel_modules

# Sysctl configuration
-w /etc/sysctl.conf -p wa -k sysctl
-w /etc/sysctl.d/ -p wa -k sysctl

# Time configuration
-w /etc/localtime -p wa -k time_config
-w /etc/timezone -p wa -k time_config

## ============================================
## Persistence Mechanisms
## ============================================

# Init scripts and rc.local
-w /etc/init.d/ -p wa -k init
-w /etc/rc.local -p wa -k init

# LD preload hijacking
-w /etc/ld.so.conf -p wa -k ld_preload
-w /etc/ld.so.conf.d/ -p wa -k ld_preload
-w /etc/ld.so.preload -p wa -k ld_preload

# Shell profiles (user persistence)
-w /etc/profile -p wa -k shell_profiles
-w /etc/profile.d/ -p wa -k shell_profiles
-w /etc/bashrc -p wa -k shell_profiles
-w /etc/bash.bashrc -p wa -k shell_profiles
-w /etc/shells -p wa -k shell_profiles
-w /etc/environment -p wa -k shell_profiles

## ============================================
## Privilege Escalation
## ============================================

# Sudoers
-w /etc/sudoers -p wa -k sudoers
-w /etc/sudoers.d/ -p wa -k sudoers

# SUID/SGID execution
-a always,exit -F arch=b64 -S execve -F euid=0 -F auid>=1000 -F auid!=4294967295 -k priv_esc
-a always,exit -F arch=b32 -S execve -F euid=0 -F auid>=1000 -F auid!=4294967295 -k priv_esc

# setuid/setgid calls
-a always,exit -F arch=b64 -S setuid -S setreuid -S setresuid -F a0!=0 -k setuid
-a always,exit -F arch=b64 -S setgid -S setregid -S setresgid -F a0!=0 -k setgid

## ============================================
## Process Execution
## ============================================

# All execve calls (very verbose — use on high-value targets)
-a always,exit -F arch=b64 -S execve -k exec
-a always,exit -F arch=b32 -S execve -k exec

# Specific suspicious binaries
-w /usr/bin/wget -p x -k download_tools
-w /usr/bin/curl -p x -k download_tools
-w /usr/bin/base64 -p x -k encoding_tools
-w /usr/bin/nc -p x -k netcat
-w /usr/bin/ncat -p x -k netcat
-w /usr/bin/nmap -p x -k recon_tools
-w /usr/bin/rawshark -p x -k recon_tools
-w /usr/bin/tcpdump -p x -k recon_tools
-w /usr/sbin/iptables -p x -k firewall_tools
-w /usr/sbin/nft -p x -k firewall_tools
-w /usr/bin/wireshark -p x -k recon_tools

# Package management
-w /usr/bin/dpkg -p x -k package_mgmt
-w /usr/bin/apt -p x -k package_mgmt
-w /usr/bin/apt-get -p x -k package_mgmt
-w /usr/bin/yum -p x -k package_mgmt
-w /usr/bin/dnf -p x -k package_mgmt
-w /usr/bin/rpm -p x -k package_mgmt
-w /usr/bin/pip -p x -k package_mgmt
-w /usr/bin/pip3 -p x -k package_mgmt

## ============================================
## Kernel and Module Loading
## ============================================

-a always,exit -F arch=b64 -S init_module -S finit_module -k kernel_module_load
-a always,exit -F arch=b64 -S delete_module -k kernel_module_unload
-a always,exit -F arch=b32 -S init_module -S finit_module -k kernel_module_load
-a always,exit -F arch=b32 -S delete_module -k kernel_module_unload

## ============================================
## Network Activity
## ============================================

# Socket creation (very verbose — use selectively)
-a always,exit -F arch=b64 -S socket -F a0=2 -k network_socket_ipv4
-a always,exit -F arch=b64 -S socket -F a0=10 -k network_socket_ipv6

# Network connections by non-root
-a always,exit -F arch=b64 -S connect -F euid!=0 -F auid>=1000 -F auid!=4294967295 -k network_connect

## ============================================
## File Deletion and Anti-Forensics
## ============================================

-a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -k file_deletion
-a always,exit -F arch=b32 -S unlink -S unlinkat -S rename -S renameat -k file_deletion

# Log tampering
-w /var/log/ -p wa -k log_tampering

## ============================================
## Container Activity
## ============================================

-w /usr/bin/docker -p x -k container
-w /usr/bin/dockerd -p x -k container
-w /usr/bin/containerd -p x -k container
-w /usr/bin/runc -p x -k container
-w /usr/bin/kubectl -p x -k container
-w /usr/bin/podman -p x -k container
-w /etc/docker/ -p wa -k container_config

## ============================================
## Make rules immutable (MUST BE LAST RULE)
## ============================================
-e 2

File Watch Permissions

Permission	Meaning
r	Read access
w	Write access
x	Execute access
a	Attribute change

Key ausearch / aureport Commands

# Search by key
ausearch -k identity --interpret

# Search by event type
ausearch -m USER_LOGIN --interpret

# Search by time range
ausearch --start 2026-03-14 00:00:00 --end 2026-03-14 23:59:59

# Summary reports
aureport --summary
aureport --auth --summary          # Authentication summary
aureport --login --summary         # Login summary
aureport --executable --summary    # Executed binaries
aureport --anomaly                 # Anomalous events
aureport --key --summary           # Events by audit key
aureport --failed                  # All failed events

---

7. Cloud Provider Log Sources

AWS CloudTrail

Log Type	Description	Key Events
Management Events	Control plane API calls	CreateUser, AttachUserPolicy, CreateAccessKey, AssumeRole, ConsoleLogin, StopLogging, DeleteTrail
Data Events	Data plane operations	S3 GetObject/PutObject, Lambda Invoke, DynamoDB GetItem/PutItem
Insights Events	Anomalous API activity	Unusual volume of specific API calls
Network Activity Events	VPC endpoint activity	Network-level API routing

Key fields: eventName, eventSource, sourceIPAddress, userIdentity, requestParameters, responseElements, errorCode, eventTime, awsRegion

Critical CloudTrail Events to Monitor

# IAM Abuse
CreateUser, CreateAccessKey, AttachUserPolicy, AttachRolePolicy
PutUserPolicy, CreateLoginProfile, UpdateLoginProfile
CreateRole, UpdateAssumeRolePolicy

# Persistence
CreateEventSubscription, PutBucketNotification, CreateFunction
CreateTrail, UpdateTrail, StopLogging, DeleteTrail (anti-forensics)

# Defense Evasion
StopLogging, DeleteTrail, PutEventSelectors (reduce logging scope)
DeleteFlowLogs, DisableGuardDuty, DeleteDetector
ModifyInstanceAttribute (disable IMDSv2)

# Credential Access
GetSecretValue, GetParameter, GetPasswordData
AssumeRole (cross-account), GetSessionToken, GetFederationToken

# Exfiltration
PutBucketPolicy (public access), CreateSnapshot, ShareSnapshot
ModifySnapshotAttribute, CopySnapshot (cross-account)

Additional AWS Log Sources

Source	Description	Service
VPC Flow Logs	Network traffic metadata (src/dst IP, port, protocol, action)	VPC
GuardDuty Findings	Threat detection findings	GuardDuty
S3 Access Logs	Bucket access logs	S3
WAF Logs	Web application firewall logs	WAF
Route 53 Query Logs	DNS query logs	Route 53
ELB Access Logs	Load balancer access logs	ELB/ALB/NLB
CloudWatch Logs	Application and system logs	CloudWatch
Config	Resource configuration changes	AWS Config
SSM Session Manager Logs	Interactive session logs	Systems Manager

Azure

Log Type	Description	Key Events
Azure Activity Log	Subscription-level control plane operations	Resource CRUD, RBAC changes, policy assignments
Azure AD Sign-in Logs	Authentication events	Success/failure, MFA status, conditional access, risky sign-ins
Azure AD Audit Logs	Directory changes	User/group/app modifications, role assignments
Azure AD Provisioning Logs	Identity provisioning	SCIM operations
NSG Flow Logs	Network traffic at NSG level	Src/dst IP, port, protocol, allow/deny
Azure Firewall Logs	Firewall rule processing	Application rules, network rules, threat intelligence
Key Vault Logs	Secret/key/certificate operations	GetSecret, SetSecret, DeleteSecret, Decrypt, Sign
Storage Analytics Logs	Storage account access	Read, write, delete operations on blobs/tables/queues
Microsoft Defender for Cloud	Security recommendations and alerts	Vulnerability findings, threat detections

Critical Azure Events to Monitor

# Identity
Add member to role, Add user, Delete user
Reset user password, Disable account
Add service principal credentials
Consent to application (OAuth app consent phishing)
Add owner to application

# Conditional Access
Update conditional access policy, Delete conditional access policy

# Resource
Create or update resource, Delete resource
Create or update role assignment
Create or update policy assignment

# Storage
Put Blob, Get Blob (sensitive containers)
Set Blob Service Properties (disable logging)

GCP

Log Type	Description	Key Events
Admin Activity Audit Logs	API calls that modify resources (always on)	SetIamPolicy, Insert, Delete, Patch operations
Data Access Audit Logs	API calls that read resources (configurable)	Get, List operations on sensitive resources
System Event Audit Logs	Google-initiated system events	Live migration, maintenance events
Policy Denied Audit Logs	Access denied by VPC Service Controls	Denied requests at perimeter boundaries
VPC Flow Logs	Network traffic metadata	Sampled network flows
Firewall Rules Logging	Firewall rule matches	Allow/deny decisions per rule
DNS Logs	Cloud DNS queries	Query name, type, response
Load Balancer Logs	HTTP(S) request logs	URL, status, latency, client IP

Critical GCP Events to Monitor

# IAM
SetIamPolicy (any resource)
CreateServiceAccountKey
CreateServiceAccount
AddMember (roles/owner, roles/editor)

# Persistence
compute.instances.setMetadata (startup scripts)
cloudfunctions.functions.create
pubsub.subscriptions.create

# Defense Evasion
logging.sinks.delete, logging.sinks.update
compute.firewalls.delete, compute.firewalls.patch
organizations.setIamPolicy

# Exfiltration
storage.buckets.setIamPolicy (public access)
compute.snapshots.setIamPolicy
bigquery.datasets.setIamPolicy

---

8. Log Shipping Architecture

Windows Event Forwarding (WEF)

Source: Palantir WEF, NSA Event Forwarding Guidance

Architecture

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│  Source       │     │  WEC Server  │     │    SIEM      │
│  (Endpoint)  │────►│  (Collector) │────►│  (Analysis)  │
│              │     │              │     │              │
│  WinRM Svc   │     │  Subscriptions│    │  Splunk/ELK/ │
│  Event Fwd   │     │  Custom       │    │  Sentinel    │
│  Svc         │     │  Channels     │    │              │
└──────────────┘     └──────────────┘     └──────────────┘
     ▲ GPO                                      ▲
     │                                          │
  Domain Controller                     Log shipper
  (Audit Policy)                       (WinLogBeat/
                                        NXLog/Fluent Bit)

WEF Configuration Steps

1. Audit Policy GPO — configure Advanced Audit Policy on endpoints
2. WinRM GPO — enable WinRM service, configure firewall rules
3. WEC Server — install Windows Event Collector service, create custom event channels
4. Subscriptions — XML-based event selection using XPath queries
5. Log Shipper — forward from WEC to SIEM

Subscription Types

Type	Mechanism	Use Case
Source-initiated	Endpoints push to collector	Scalable, GPO-driven, preferred for domain environments
Collector-initiated	Collector pulls from endpoints	Small environments, specific high-value targets

Palantir Subscription Categories

Palantir organizes WEF subscriptions by security function:

• Account lockout events
• Log deletion and audit policy changes
• Object access events
• Process execution events
• Service installation events
• Authentication events (NTLM, Kerberos)
• Group/user management events
• PowerShell operational events
• Task scheduler events
• Firewall events
• Sysmon events (forwarded through WEF)
• Autoruns baseline changes

NSA Recommended Event Categories

The NSA guidance ("Spotting the Adversary") recommends forwarding:

• Application crashes and errors
• Application whitelisting events (AppLocker/WDAC)
• Certificate services events
• DNS server logs
• Group Policy events
• Log clearing events
• NTLM authentication events
• Operating system events (startup, shutdown, time changes)
• PowerShell events (Script Block, Module, Transcription)
• Process creation events (4688 with command line)
• Service events (install, start, stop)
• Share access events
• Smart card events
• Software/service installation
• Task scheduler events
• Windows Defender events
• Windows Firewall events

Fluent Bit

Lightweight log processor and forwarder (< 1MB, written in C).

Pipeline Architecture

Input → Parser → Filter → Buffer → Router → Output
  │         │        │        │        │        │
  │         │        │        │        │        └─ Destination (ES, Loki, Splunk, S3, Kafka)
  │         │        │        │        └─ Tag matching (glob patterns)
  │         │        │        └─ Memory or filesystem buffering
  │         │        └─ Modify, grep, lua, record_modifier, multiline
  │         └─ JSON, regex, LTSV, logfmt, decoders
  └─ tail, systemd, forward, winlog, tcp, syslog, kmsg

Key Input Plugins

Plugin	Description	Security Use
tail	Follow log files	Application logs, auth.log, syslog
systemd	Read journald	System events, service logs
forward	Receive from Fluentd/Fluent Bit	Agent chaining
winlog / winevtlog	Windows Event Log	Security, Sysmon, PowerShell event logs
tcp / udp	Network listeners	Syslog receivers
syslog	RFC 3164/5424 syslog	Network device logs
kmsg	Kernel ring buffer	Kernel events
node_exporter_metrics	System metrics	Performance monitoring

Key Output Plugins

Plugin	Description	Use Case
es / opensearch	Elasticsearch/OpenSearch	ELK/OpenSearch SIEM
loki	Grafana Loki	Label-based log storage
splunk	Splunk HEC	Enterprise SIEM
kafka	Apache Kafka	Event streaming pipeline
s3	AWS S3	Long-term log archive
azure_blob	Azure Blob Storage	Azure log archive
stackdriver	GCP Cloud Logging	GCP integration
forward	Fluentd/Fluent Bit	Agent chaining
http	HTTP endpoint	Webhook/custom API
file	Local file	Local storage/backup

Key Filter Plugins

Plugin	Description	Security Use
grep	Include/exclude by regex	Drop noisy events, select security events
modify	Add/remove/rename fields	Field normalization
record_modifier	Add static fields	Add environment tags (hostname, cluster)
lua	Custom Lua scripting	Complex transformation, enrichment
multiline	Merge multi-line entries	Stack traces, multi-line log events
nest	Nest/lift fields	Schema normalization
parser	Re-parse field content	Extract structured data from message field

Buffering Strategy

Mode	Description	Trade-off
Memory (default)	Events stored in memory	Fast, risk of data loss on crash
Filesystem	Events written to disk	Reliable, slower throughput
Hybrid	Memory with filesystem overflow	Balance of speed and reliability

Example: Security Log Collection Config

[SERVICE]
    Flush         5
    Daemon        Off
    Log_Level     info
    Parsers_File  parsers.conf
    storage.path  /var/log/fluent-bit-buffer/

[INPUT]
    Name          tail
    Path          /var/log/auth.log
    Tag           auth.*
    Parser        syslog-rfc3164
    DB            /var/lib/fluent-bit/auth.db
    Mem_Buf_Limit 10MB

[INPUT]
    Name          systemd
    Tag           systemd.*
    Systemd_Filter _SYSTEMD_UNIT=sshd.service
    Systemd_Filter _SYSTEMD_UNIT=sudo.service
    Read_From_Tail On

[INPUT]
    Name          tail
    Path          /var/log/audit/audit.log
    Tag           audit.*
    Parser        audit
    DB            /var/lib/fluent-bit/audit.db

[FILTER]
    Name          record_modifier
    Match         *
    Record        hostname ${HOSTNAME}
    Record        environment production
    Record        cluster security-monitoring

[FILTER]
    Name          grep
    Match         auth.*
    Regex         log (Failed|Accepted|Invalid|session opened|session closed)

[OUTPUT]
    Name          loki
    Match         *
    Host          loki.internal.corp
    Port          3100
    Labels        job=security-logs, host=${HOSTNAME}
    TLS           On
    TLS.Verify    On

[OUTPUT]
    Name          es
    Match         *
    Host          elasticsearch.internal.corp
    Port          9200
    Index         security-logs
    TLS           On
    HTTP_User     fluent-bit
    HTTP_Passwd   ${ES_PASSWORD}

Grafana Loki

Log aggregation system that indexes labels, not log content.

Architecture

┌──────────┐     ┌──────────────┐     ┌──────────────┐     ┌──────────┐
│  Alloy/  │────►│ Distributor  │────►│  Ingester    │────►│ Storage  │
│ Promtail │     │              │     │              │     │          │
│ FluentBit│     │  Validates   │     │  In-memory   │     │ S3/GCS/  │
│          │     │  Distributes │     │  WAL         │     │ Azure/   │
└──────────┘     └──────────────┘     │  Flush→store │     │ Filesystem│
                                      └──────────────┘     └──────────┘
                                             ▲
┌──────────┐     ┌──────────────┐            │
│ Grafana  │────►│ Query        │────────────┘
│  UI      │     │ Frontend     │
│          │     │  → Querier   │
└──────────┘     └──────────────┘

Key Components

Component	Function
Distributor	Receives incoming log streams, validates, distributes to ingesters
Ingester	Stores logs in memory, flushes to long-term storage
Querier	Retrieves logs from ingesters and storage for queries
Query Frontend	Caches results, splits large queries, optimizes
Compactor	Compacts index files for storage efficiency
Index Gateway	Serves index queries (optional)

Deployment Modes

Mode	Description	Scale
Monolithic	Single binary, all components	Dev/small (< 100GB/day)
Simple Scalable	Read, write, backend targets	Medium (100GB-few TB/day)
Microservices	Each component independently scaled	Large (multi-TB/day)

Storage Backends

Backend	Use Case
Filesystem	Local dev/testing
AWS S3	Production AWS
GCS	Production GCP
Azure Blob Storage	Production Azure
MinIO	Self-hosted S3-compatible

LogQL Query Language

# Stream selector (required — selects log streams by labels)
{job="security-logs", host="web-server-01"}

# Filter expressions (pipe operators)
{job="security-logs"} |= "Failed password"           # Contains
{job="security-logs"} != "CRON"                       # Does not contain
{job="security-logs"} |~ "Failed.*root"               # Regex match
{job="security-logs"} !~ "session (opened|closed)"    # Negative regex

# Parser expressions
{job="security-logs"} | json                          # Parse JSON
{job="security-logs"} | logfmt                        # Parse logfmt
{job="security-logs"} | regexp `user=(?P<user>\w+)`   # Extract with regex

# Metric queries (aggregations over time)
count_over_time({job="auth"} |= "Failed password" [5m])    # Count failures in 5m windows
rate({job="auth"} |= "Failed password" [1m])                # Rate of failures per second
sum by (host) (count_over_time({job="auth"} |= "Failed" [5m]))  # Failures by host

# Top-K queries
topk(10, sum by (src_ip) (count_over_time({job="firewall"} |= "DENY" [1h])))

# Alert-ready queries (detect brute force)
sum by (src_ip) (count_over_time({job="auth"} |= "Failed password" [5m])) > 10

Label Best Practices

• Use static labels (environment, job, host, cluster) not dynamic (user, IP, request_id)
• High-cardinality labels destroy performance — Loki is not designed for them
• Use filter expressions (|=, |~) for high-cardinality filtering, not labels
• Keep label count per stream under 15-20

---

9. OSSEM Data Model

Source: OSSEM, OSSEM-CDM

Project Structure

OSSEM (Open Source Security Events Metadata) is a community-led framework for documenting, standardizing, and modeling security event logs across platforms.

OSSEM
├── OSSEM-DD (Data Dictionaries)     — Field-level documentation per log source
├── OSSEM-CDM (Common Data Model)    — Normalized entity schemas
└── OSSEM-DM (Detection Model)       — Relationships between events for detection

Components

Data Dictionaries (OSSEM-DD)

Reference documentation organized by OS and data provider. Each dictionary covers a single event log and catalogs every field name, type, and description.

Coverage: Windows Security Events, Sysmon, PowerShell, Linux auditd, macOS, AWS CloudTrail, Azure Activity Logs, Zeek (formerly Bro).

Common Data Model (OSSEM-CDM)

Provides standardized entity schemas for log normalization. Entities and schema tables map heterogeneous log sources to common field names.

Core Entities:

Entity	Description	Key Fields
Process	Process execution events	process_name, process_id, process_guid, process_command_line, process_path, process_integrity_level
File	File system operations	file_name, file_path, file_extension, file_directory, file_hash_md5, file_hash_sha256
Network	Network connections	dst_ip_addr, src_ip_addr, dst_port, src_port, network_protocol, network_direction
Registry	Registry operations (Windows)	registry_key, registry_value_name, registry_value_data, registry_value_type
User	User identity	user_name, user_domain, user_sid, user_logon_id, user_session_id
Host	Host/system identity	host_name, host_ip, host_os_family, host_os_version
Hash	Hash values	hash_md5, hash_sha1, hash_sha256, hash_imphash
HTTP	HTTP request/response	http_request_method, http_url, http_status_code, http_user_agent, http_content_type
DNS	DNS queries/responses	dns_query_name, dns_query_type, dns_response_code, dns_rdata
Authentication	Authentication events	logon_type, auth_protocol, auth_package
Service	Service operations	service_name, service_type, service_start_type, service_state

Schema Tables aggregate entities to normalize related data sources. For example, a Network Session table combines Network, User, Host, and Process entities to normalize firewall, proxy, and IDS logs.

Detection Model (OSSEM-DM)

Maps relationships between security events for building detection analytics. Organized around adversary techniques — connects specific data sources to ATT&CK techniques, identifying which events provide coverage for which attack behaviors.

OSSEM in Practice

Raw Log (Sysmon Event ID 1)           Normalized (OSSEM-CDM)
─────────────────────────────         ─────────────────────────
Image: C:\Windows\cmd.exe       →     process_name: cmd.exe
CommandLine: cmd /c whoami       →     process_command_line: cmd /c whoami
ParentImage: C:\...\winword.exe  →     parent_process_name: winword.exe
User: CORP\jsmith                →     user_name: jsmith
                                       user_domain: CORP
LogonGuid: {abc-123}             →     user_logon_id: abc-123
Hashes: SHA256=abc123...         →     hash_sha256: abc123...

Adopters: HELK (Hunting ELK), Microsoft Sentinel normalization schema.

---

10. EVTX Analysis & Attack Samples

python-evtx — EVTX Parsing

pip install python-evtx

import Evtx.Evtx as evtx

# Parse EVTX file and extract XML
with evtx.Evtx("Security.evtx") as log:
    for record in log.records():
        print(record.xml())

# Available command-line tools:
# evtx_dump.py Security.evtx          — XML output
# evtx_dump_json.py Security.evtx     — JSON output
# evtx_info.py Security.evtx          — Metadata and checksums
# evtx_templates.py Security.evtx     — Event templates

EVTX-ATTACK-SAMPLES

Source: sbousseaden/EVTX-ATTACK-SAMPLES

~200 real EVTX samples organized by ATT&CK tactic:

Tactic	Example Techniques with Samples
Execution	PowerShell, WMI, MSHTA, Regsvr32, CMSTP, Rundll32
Persistence	Registry Run keys, scheduled tasks, WMI subscriptions, services
Privilege Escalation	Token manipulation, DLL hijacking, UAC bypass
Defense Evasion	Process injection, timestomping, log clearing, AMSI bypass
Credential Access	LSASS dumping, Kerberoasting, DCSync, credential file access
Discovery	Network scanning, account enumeration, system info gathering
Lateral Movement	PsExec, WMI remote, RDP, SMB, DCOM
Command and Control	DNS tunneling, HTTP C2, named pipes

Use cases: testing Sigma rules, training detection analysts, validating SIEM parsers, building detection lab datasets.

---

11. Tool-to-Event-ID Mapping

Source: JPCERT/CC Tool Analysis Result Sheet

JPCERT/CC analyzed 49 attacker tools and documented which Windows Event IDs are generated in Security, Sysmon, and other logs. Key mappings:

Command Execution Tools

Tool	Security Event IDs	Sysmon Events	Other Logs
PsExec	4624 (Type 3), 4672, 5140, 7045	1 (Process Create), 17/18 (Named Pipes: \PSEXESVC)	System: 7045 (Service Install)
wmic	4624 (Type 3), 4672	1 (wmiprvse.exe child process)	WMI-Activity/Operational
WinRM/WinRS	4624 (Type 3), 4672	1 (wsmprovhost.exe child)	Microsoft-Windows-WinRM/Operational
schtasks	4698, 4702	1	TaskScheduler/Operational
BITS	—	1 (bitsadmin.exe)	Microsoft-Windows-Bits-Client/Operational

Credential Dumping Tools

Tool	Security Event IDs	Sysmon Events	Other Logs
Mimikatz	4624, 4672, 4648	1, 10 (lsass.exe access), 7 (sekurlsa DLL loads)	—
WCE	4624, 4648, 4672	1, 10 (lsass.exe access)	—
PWDump7	—	1, 10 (lsass.exe access)	—
gsecdump	—	1, 10 (lsass.exe access)	—
ntdsutil	4624, 4672	1 (ntdsutil.exe)	—

Lateral Movement Tools

Tool	Security Event IDs	Sysmon Events	Key Indicators
RDP	4624 (Type 10), 4778, 4779	1 (mstsc.exe), 3 (port 3389)	TerminalServices-LocalSessionManager
net use	4624 (Type 3), 5140, 5145	3 (port 445)	SMB share mapping
wmiexec	4624 (Type 3), 4672	1 (wmiprvse.exe spawning cmd)	WMI-Activity log

Reconnaissance Tools

Tool	Security Event IDs	Sysmon Events
nltest	—	1 (nltest.exe)
csvde/ldifde	4662 (AD object access)	1
dsquery	4662	1
net user/group	—	1
nmap	—	3 (network connections)

Evidence Removal Tools

Tool	Security Event IDs	Sysmon Events
wevtutil	1102 (log cleared)	1 (wevtutil.exe)
sdelete	—	1, 23/26 (file delete)
timestomp	—	2 (file creation time change)

---

12. Minimum Viable Logging Checklist

Tier 1: Absolute Minimum (Day 1)

Without these, you are effectively blind:

• Windows Security Event Log — 4624/4625 (logon), 4688 (process creation with command line), 4672 (special logon)
• Windows PowerShell Script Block Logging — Event ID 4104
• Sysmon installed — Event IDs 1 (process create), 3 (network), 11 (file create), 22 (DNS), 23 (file delete)
• Linux auth.log / secure — SSH authentication, sudo usage
• Linux auditd — at minimum: execve, identity files (/etc/passwd, /etc/shadow), sudoers, SSH config
• Cloud control plane logs — CloudTrail (management events), Azure Activity Log, GCP Admin Activity
• Firewall logs — allow and deny (deny-only misses exfiltration)
• DNS query logs — from DNS servers or Sysmon Event ID 22
• Time synchronization — NTP configured and verified across all sources

Tier 2: Detection Capable (Week 1-2)

Enables actual threat detection and hunting:

• Sysmon full config — add Event IDs 7 (image load, filtered), 8 (remote thread), 10 (process access), 12-14 (registry), 17-18 (named pipes)
• Windows advanced audit policy — all categories per section 4 above
• PowerShell Module Logging and Transcription
• Linux auditd full ruleset — per section 6 above
• Cloud data plane logs — S3 data events, sensitive API calls
• Web application logs — using OWASP vocabulary (section 3)
• Proxy/web gateway logs — URL, user agent, response codes
• Email gateway logs — sender, recipient, attachment hashes, URLs
• Endpoint Detection and Response (EDR) — telemetry to SIEM
• VPC Flow Logs / NSG Flow Logs — network traffic metadata
• Certificate/PKI logs — certificate issuance and usage
• Log centralization operational — all sources shipping to central SIEM/lake

Tier 3: Mature Detection (Month 1-3)

Comprehensive coverage with threat hunting capability:

• WEF infrastructure — Palantir/NSA subscription model deployed
• OSSEM normalization — common field names across all sources
• Log integrity monitoring — tamper detection on log stores
• Baseline established — normal behavior documented for anomaly detection
• Sigma rules deployed — minimum coverage for MITRE ATT&CK top techniques
• Retention policy defined — per regulatory requirements, hot/warm/cold tiers
• Alert triage runbooks — for every high/critical alert, documented response
• Threat hunting program — scheduled hunts using detection gaps as hypotheses
• Cloud-specific detections — GuardDuty/Defender/Security Command Center active
• Container/Kubernetes logs — API server audit logs, pod events, network policies
• Identity provider logs — Okta/Azure AD/Google Workspace sign-in and admin events
• Database audit logs — privileged queries, schema changes, data access

Detection Coverage Heat Map

Map your logging against ATT&CK tactics. Minimum viable coverage requires at least one reliable data source per tactic:

Tactic	Primary Log Source	Secondary Source
Initial Access	Web/email gateway, firewall	CloudTrail, proxy logs
Execution	Sysmon EID 1, Windows 4688, auditd execve	PowerShell 4104, WMI logs
Persistence	Sysmon EID 11-14, 17, 19-21, auditd file watches	Scheduled task logs, service install (7045)
Privilege Escalation	Windows 4672, 4673, Sysmon EID 10, auditd setuid	Token manipulation, UAC events
Defense Evasion	Sysmon EID 2, 8, 25, Windows 1102, 4719	Process injection, log clearing
Credential Access	Sysmon EID 10 (lsass), Windows 4768/4769	DPAPI events, NTLM relay
Discovery	Sysmon EID 1 (recon tools), EID 22 (DNS)	Network scanning (flow logs)
Lateral Movement	Windows 4624 (Type 3/10), 5140/5145	RDP, WinRM, PsExec indicators
Collection	File access audit (4663), Sysmon EID 24	Email access, database queries
Exfiltration	Firewall/proxy logs, DNS query volume	Cloud storage API calls, flow logs
Command and Control	Sysmon EID 3/22, proxy logs, DNS	TLS inspection, JA3/JA4 hashes
Impact	Windows 4720/4726, file deletion, service stop	Ransomware indicators (mass file rename)

---

References

• OWASP Logging Cheat Sheet
• OWASP Logging Vocabulary
• OSSEM Project
• OSSEM Common Data Model
• Sysmon Documentation
• sysmon-modular (Olaf Hartong)
• SwiftOnSecurity Sysmon Config
• Neo23x0 Sysmon Config (Florian Roth)
• Palantir Windows Event Forwarding
• NSA Event Forwarding Guidance
• JPCERT/CC Tool Analysis Result Sheet
• EVTX-ATTACK-SAMPLES
• python-evtx
• Fluent Bit Documentation
• Grafana Loki
• NIST SP 800-92: Guide to Computer Security Log Management
• NSA: Spotting the Adversary with Windows Event Log Monitoring
• ACSC: Windows Event Logging and Forwarding
• MalwareArchaeology: Windows Logging Cheat Sheets