Which Logs Should a SIEM Collect? A 2026 Priority Guide

The first time you are handed a SIEM project for a company with a few thousand endpoints, the instinct is always the same: turn everything on, point every device at the collector, and sort it out later. That plan dies in about three weeks, usually when someone from finance asks why the storage bill tripled. Log collection is not a technical problem, it is a prioritisation problem.

This guide is the decision list we wish we had on day one – what genuinely belongs in a corporate SIEM, what belongs in cheap archive storage, and what you should simply never send at all.

---

Why Can You Not Just Collect Every Log?

Run the arithmetic once and the answer becomes obvious. A mid-size corporate network with 5,000 endpoints, 300 servers, a pair of next-generation firewalls, a proxy, an email gateway and the usual network fabric will produce somewhere between 300 GB and 800 GB of raw log data every single day if nothing is filtered. Firewall traffic logs alone routinely account for half of that.

At 400 GB per day, one year of retention is roughly 146 TB of data. Commercial SIEM licensing is still commonly sold per GB per day, at figures often quoted between $1,500 and $2,500 per GB/day annually (approx. ₹1.3 lakh to ₹2.2 lakh). Even at the low end, 400 GB/day lands you in the region of ₹5 crore per year in licensing before you have paid for a single analyst. Self-hosting on OpenSearch removes the licence, but you then pay in nodes, disks and the engineer time to keep a 146 TB cluster healthy.

So the real question is never “can we collect this”. It is “what question will this log answer, and would we ever actually ask it”. If a log source cannot answer a detection question, an investigation question, or a compliance question, it has no business being indexed.

Log Source	Priority	Typical Daily Volume	Why
Domain Controllers (auth, Kerberos, account changes)	Tier 1 – Mandatory	Medium	Nearly every intrusion touches identity
EDR alerts (SentinelOne, Symantec, Defender)	Tier 1 – Mandatory	Low	Highest fidelity detections you own
Firewall deny and threat logs	Tier 1 – Mandatory	Medium	Perimeter attack visibility
VPN and SSO sign-in logs	Tier 1 – Mandatory	Low	Account takeover detection
Email gateway verdicts and URL clicks	Tier 1 – Mandatory	Medium	Phishing is still the top entry vector
Proxy blocked and threat transactions	Tier 1 – Mandatory	Medium	C2 and malicious download evidence
DNS query logs	Tier 2 – Collect filtered	Very High	Superb for hunting, brutal on volume
Firewall allowed traffic logs	Tier 2 – Collect filtered	Very High	Needed for scoping, not for alerting
Proxy allowed web traffic	Tier 3 – Archive only	Very High	Useful in IR, rarely in detection
Device health, heartbeats, SNMP polls	Never collect	High	Answers no security question

💬 “On our first build we enabled Windows event ID 5156 across the fleet because the documentation said it gives network connection visibility. It does. It also generated more data by itself than every other source combined, and we burned four days of cluster capacity before anyone noticed. We have never enabled it fleet-wide again.”

Which Logs Must You Always Collect?

Start from identity. If an attacker is inside your network, they are authenticating as somebody, and authentication logs are comparatively cheap. Domain Controller security logs are the single highest value source in a Windows enterprise, and there is no acceptable reason to skip them.

Event ID	What It Tells You	Collect From
4624 / 4625	Successful and failed logons, with logon type	DCs, servers, endpoints
4768 / 4769 / 4771	Kerberos ticket requests, useful for Kerberoasting	Domain Controllers only
4672	Special privileges assigned at logon	DCs and servers
4688	Process creation with full command line	Servers and endpoints
4720 / 4722 / 4728 / 4732 / 4756	Account created, enabled, added to privileged groups	Domain Controllers
7045	New service installed, classic persistence	All Windows hosts
1102	Security audit log was cleared	All Windows hosts
4104	PowerShell script block logging	Servers and endpoints

Note that 4688 requires command line auditing to be explicitly enabled through Group Policy. Without that setting the event records only the executable path, which strips out roughly ninety percent of its detection value. This is the most common misconfiguration we see in Indian enterprise deployments.

Beyond Windows, the non-negotiable list is short and consistent across almost every environment:

• VPN and remote access authentication – every session, success and failure, with source IP and geolocation.
• SSO and identity provider sign-in logs – Entra ID, Okta, or whatever fronts your SaaS estate. Impossible travel and MFA fatigue detections live here.
• Firewall threat logs and denied sessions at the perimeter, plus all administrative and configuration change events.
• EDR detections, quarantine actions, policy modifications and agent tamper or uninstall events.
• Email gateway verdicts – blocked, quarantined, released, plus rewritten URL click events.
• Cloud control plane audit logs such as AWS CloudTrail management events or Azure Activity logs.
• DHCP lease assignments – tiny volume, and without it you cannot map an IP address to a machine three weeks after an incident occured.

That last one surprises people. DHCP logs are boring and almost free, and they are the difference between an investigation that resolves in an hour and one that stalls permanently.

📌 Official Resource: Microsoft’s criticality-rated list of Windows security events worth monitoring, still the best starting point for an audit policy. Visit Official Website

Which Logs Are Safe to Drop?

Here is the list that saves you the most money for the least risk. None of these events have ever meaningfully contributed to a detection in any environment we have worked in.

• Windows 5156 and 5158 (Windows Filtering Platform connection allowed) – enormous volume, almost zero detection value when you already have firewall and EDR telemetry.
• Windows 4634 and 4647 (logoff events) – you can infer session end from other data. Logoffs are half your Windows volume and nobody alerts on them.
• Windows 4689 (process terminated) – process creation matters, termination almost never does.
• Windows 4658 and 4656 unfiltered handle events – only useful with a tightly scoped SACL on genuinely sensitive files.
• Syslog severity 6 and 7 (informational and debug) from switches and routers. Keep severity 0 to 4 and you lose nothing.
• SNMP polling responses, keepalives, heartbeats and agent check-ins from any product.
• Antivirus signature update and scan completed events. Operationally useful, security-irrelevant. Send them to your monitoring tool, not the SIEM.
• Spanning tree recalculations and access port link up/down flapping.
• Load balancer and health check probe traffic – predictable, constant, and pure noise.
• Print spooler, Windows Update and performance counter logs.

There is one nuance worth respecting. “Do not collect” and “do not retain” are different decisions. Some of these events have forensic value in a specific, rare investigation. The right answer for those is a cheap object storage archive, not an indexed SIEM tier, which we will come to shortly.

How Do You Handle Firewall and Proxy Traffic Logs?

These two sources are where the budget actually goes, and where blunt on/off thinking fails. You cannot drop them entirely because they are essential for scoping an incident. You also cannot index all of them.

For firewalls, apply the filtering at the collection layer – a Logstash filter, a Cribl-style pipeline, or the vendor’s own log forwarding profile:

1. Collect all denied sessions at the internet perimeter, and all denies crossing between internal security zones.
2. Collect allowed sessions that cross the perimeter, in either direction, at session end only. Do not log session start and session end separately.
3. Drop intra-zone permitted traffic entirely. Workstation to workstation chatter inside the same VLAN is not something you will ever query.
4. Drop broadcast, multicast, NetBIOS noise and traffic to internal NTP and DNS infrastructure.
5. Keep every threat, URL filtering, WildFire or sandbox verdict, and every configuration change regardless of volume. These are low volume and high value.
6. Consider NetFlow or IPFIX instead of full traffic logs for internal east-west visibility. You lose application detail and gain roughly an order of magnitude in volume.

Zscaler and other cloud proxies follow the same logic. Send blocked transactions, threat categories, DLP violations, sandbox verdicts and tunnel authentication events into the SIEM. Allowed web traffic goes to archive, and even there it is worth stripping static content requests – images, stylesheets, fonts and JavaScript typically account for over seventy percent of transaction count while contributing essentially nothing.

DNS deserves a special mention because it is simultaneously one of the best hunting datasets available and one of the largest. The compromise we usually recommend is to collect external resolutions and NXDOMAIN responses into the SIEM, drop queries to known internal zones, and deduplicate repeated identical lookups within a short window.

💬 “We once spent two weeks arguing to keep full proxy logs in hot storage for an entire year. Then we pulled the query history and found that no analyst had searched proxy data older than 34 days in eleven months. We moved everything past 90 days to object storage and the argument ended in about five minutes.”

What About EDR, Email Security and Zscaler Integrations?

Modern security tools have their own consoles, their own retention, and their own search. Duplicating all of that into your SIEM is the most expensive mistake available to you, and it is a mistake that vendors are happy to let you make.

For SentinelOne, Symantec Endpoint Protection or Defender, pull the alerts, threat detections, policy changes, device control events and anything indicating agent tampering. Do not pull the raw process, file and registry telemetry stream. That data is where the platform’s own value lives, it is enormous, and you can query it through the vendor API during an investigation when you actually need it. The volume of these raw telemetry feeds are enormous enough to double your entire ingestion on their own.

For Mimecast and Cisco ESA, the valuable records are message tracking with sender, recipient, subject, verdict and disposition, plus attachment sandbox results and URL rewrite click events. Click events in particular are the fastest way to answer “who else clicked it” during a phishing incident. What you can drop is the edge rejection stream – a gateway rejecting connections from known spam sources produces relentless volume and tells you nothing you would act on. Keep aggregate counts if you want the dashboard.

For IDS and IPS, collect medium severity and above after tuning, plus anything matching your own custom signatures. Informational and policy-level signatures should be filtered out at the sensor, not at the SIEM, otherwise you pay to transport noise you then pay again to discard.

How Do You Build a Log Tiering Plan for One Year of Retention?

The modern answer to the retention problem is not to collect less, it is to index less while storing more. Indexed, searchable storage is expensive. Compressed object storage is close to free by comparison – AWS S3 standard sits around ₹2 per GB per month, and deep archive classes are a fraction of that.

Tier	Retention	What Lives Here
Hot (fast search)	7 to 30 days	Tier 1 sources, active detections, dashboards
Warm (slower search)	30 to 90 days	Tier 1 plus filtered Tier 2, investigation window
Cold (searchable snapshot)	90 to 180 days	Filtered traffic and proxy data
Archive (object storage)	up to 12 months or more	Everything, compressed, for compliance and deep IR

A practical rollout looks like this:

1. Inventory every log source and record its actual daily volume for two weeks before deciding anything.
2. Map each source to a detection use case or a compliance requirement. Anything that maps to neither goes straight to archive or gets dropped.
3. Filter at the collection layer, never at the storage layer. Data you never transport is data you never pay for twice.
4. Normalise into a common schema early so that a seperate parser rewrite is not needed every time you add a source.
5. Configure index lifecycle policies to roll data automatically through the tiers.
6. Review quarterly. Query which indices analysts actually searched, and demote whatever nobody touched.

📌 Official Resource: OpenSearch Index State Management documentation, for automating hot, warm, cold and delete transitions. Visit Official Website

📌 Official Resource: MITRE ATT&CK Data Sources, which maps each attacker technique to the telemetry required to detect it. Use it to justify every log source you keep. Visit Official Website

That MITRE mapping is the argument that actually works in a budget meeting. When someone asks why you need Kerberos logs, “technique T1558, and here is the detection it powers” lands considerably better than “because it is important”.

Frequently Asked Questions

How much log data does a 1,000 user company generate per day?

Unfiltered, expect somewhere between 60 GB and 150 GB per day depending on how chatty your firewall and proxy configuration is. With sensible filtering applied at collection, most teams get that down to 15 GB to 30 GB of genuinely useful indexed data.

Should endpoint logs go directly into the SIEM or through the EDR?

Send EDR alerts and policy events to the SIEM, and keep raw endpoint telemetry inside the EDR platform where it is already indexed. You query it by API during an investigation rather than paying to store it twice.

Is one year of retention actually required by law in India?

CERT-In directions require organisations to maintain ICT system logs for a rolling period of 180 days within Indian jurisdiction, and various sectoral regulators impose longer windows. Compliance retention can sit in cheap archive storage, it does not need to be hot and searchable.

What is the single most under-collected log source?

DHCP lease logs, closely followed by DNS. Both are cheap relative to their investigative value, and their absence is usually discovered at the worst possible moment during an incident.

Can I filter logs after they reach the SIEM instead of before?

Technically yes, but you will already have paid the licensing or storage cost by then. Filtering belongs at the collection layer, in Logstash, a forwarder configuration, or the source device’s own logging profile.

Does dropping logs create a compliance risk?

It can, so document every filtering decision and the reasoning behind it. Auditors are generally satisfied by a defensible, written log management policy, and are far more concerned by undocumented gaps than by deliberate ones.

---

Found this helpful? Share it with a fellow maker or student. Subscribe to our RootSaid YouTube channel for hands-on hardware and AI project videos – and check out our RootSaid Robotics Malayalam channel for Malayalam tutorials.

Final Thoughts

The instinct to collect everything comes from a good place. Nobody wants to be the engineer who explains during an incident that the relevant log was never enabled. But an unfiltered SIEM fails in a quieter and more damaging way – queries time out, analysts stop trusting the platform, real alerts drown in volume, and eventually somebody proposes reducing retention to save cost, which is exactly the wrong lever.

A smaller, deliberately chosen dataset that analysts actually search beats a complete dataset that nobody can query. Start with identity, endpoint detections and perimeter denies. Add the rest only when you can name the detection or investigation it serves. These kind of decisions get easier once you have the MITRE mapping written down and can point at it.

If you are building your first SIEM, resist the urge to design the perfect pipeline before you have any real volume data. Collect Tier 1 for a month, measure what you actually have, then tune. Every environment is different, and the only log inventory that matters is the one you measured yourself.