{"id":7518,"date":"2021-10-28T12:26:06","date_gmt":"2021-10-28T19:26:06","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/insidetrack\/blog\/?p=7518"},"modified":"2023-06-25T14:06:44","modified_gmt":"2023-06-25T21:06:44","slug":"how-microsoft-narrows-the-threat-funnel-on-over-600-billion-monthly-security-events","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/insidetrack\/blog\/how-microsoft-narrows-the-threat-funnel-on-over-600-billion-monthly-security-events\/","title":{"rendered":"How Microsoft narrows the threat funnel on over 600 billion monthly security events"},"content":{"rendered":"

\"MicrosoftAt Microsoft, we typically see around 600 billion security events each month.<\/p>\n

Security events are innocuous actions that happen all the time, ranging from adding a user to a Microsoft SharePoint site, to creating a file, to deleting a folder, to opening an email. On their own, events are typically not notable, but in conjunction with other events, they can signal a threat, so we\u2019ve always got to be on the lookout.<\/p>\n

Across a large enterprise like ours, we use Microsoft Azure Sentinel<\/a> to see 20 billion security events each day, and that\u2019s a lot of data to collect and manage. Azure Sentinel is our cloud-native security information and events manager.<\/p>\n

Instead of sending our Security Operations Center (SOC) analysts on a wild hunt after every event in the SIEM, we use our own tools and solutions to funnel that big 12-digit number down to a more reasonable number. This focuses our efforts on the events that matter and how they were created\u2014this narrows us in on the ones that pose a real threat. The reason this is important is we can\u2019t\u2014and shouldn\u2019t\u2014respond to every event. Like anyone who has a security team, we want our security professionals spending their time on true threats to our company.<\/p>\n

\"Diagram
Combining solutions like machine learning and automation along with human expertise allows Microsoft to focus on the security events that require attention.<\/figcaption><\/figure>\n

But how can we be certain that the right events are being flagged for investigation?<\/p>\n

It all starts with Zero Trust<\/h2>\n

Zero Trust means verifying everything you can, including identity and device health. It means giving your users enough access to stay productive, but not enough to create unnecessary risk. Segment networks, encrypt your data from end to end, take advantage of telemetry, and assume there will be a breach.<\/p>\n

More specifically, Zero Trust architecture reduces risk across all environments by establishing strong identity verification, validating device compliance prior to granting access, and ensuring least privilege access to only explicitly authorized resources. These are the core principles of Microsoft\u2019s Zero Trust architecture.<\/a><\/p>\n

Adhering to it is the first step in protecting Microsoft and serves as a framework that encapsulates all our security controls. It\u2019s also how we empower self-service and let users access the resources they need to stay productive from anywhere.<\/p>\n

Zero Trust keeps a lot of threats outside of our ecosystem\u2014around 98 percent of attacks are subverted by these basic principles. Most data breaches, roughly 70 percent, come from phishing. In adhering to the values of Zero Trust, we assume that any device or user can be breached, which means carefully scrutinizing security events across our organization for concerning patterns.<\/p>\n

Going from several billion to several thousand<\/strong><\/h2>\n

You can find threats through use cases, scenarios that describe how unusual security events might be high risk.<\/p>\n

When you collect a lot of events data, you either create your own use cases or let the products do it for you. We rely heavily on the latter.<\/p>\n

Each of our extended detection and response tools (XDR) include a team dedicated to knowing and identifying if something is out of the ordinary. Inside Microsoft 365 Defender lives Threat Analytics, which consolidates findings\u2014like active threats and critical vulnerabilities\u2014from product groups and security researchers<\/a>. Then it tells us what the finding means, what the associated activity is, what the problem is, and which entities\u2014a device, a mailbox, an inbox, or something else\u2014are involved<\/a>.<\/p>\n

The Threat Analytics feature, along with the rich research-driven behavioral alerts<\/a> in Microsoft Defender for Endpoint, have a large impact in reducing the number of events we see. By utilizing our security tools that leverage machine learning, threat intelligence, data science, and more, we are able to filter an estimated 600 billion monthly events down to around 10,000 alerts. Whittling those 12 digits down to 4 gets us closer to real threats.<\/p>\n

But we still have to determine which items require\u00a0further investigation from our SOC<\/span>, and we only want to give our analysts cases that lead to an actionable response.<\/p>\n

Cutting that in half, and then some<\/h2>\n

An estimated 10,000 alerts are still a lot to handle. We utilize automation to help us understand what\u2019s going on and to reduce the number of cases we dig into.<\/p>\n

Microsoft Defender for Office 365\u2019s Automated Investigation and Response (AIR) helps our SOC prioritize security events that pose the greatest risk while reducing manual steps in the process. In part, it does this by triaging low-threat cases that have already been identified by our system.<\/p>\n

AIR works because it generates an investigation whenever a user flags an email as a phishing attempt. It doesn\u2019t matter if it\u2019s actually malicious or not, because our tools, like Microsoft Defender for Office 365, tell us if we need to investigate further. The tool also takes action<\/a>, like combining emails for deletion, blocking URLs, or providing additional steps to our SOC so that we can protect the company.<\/p>\n

And because AIR has taken care of the lower risk items, our SOC can worry about higher risk phishing.<\/p>\n

We also use Microsoft Defender for Endpoint Automated Investigation and Response (MDE AIR), which finds and fixes low-level malware<\/a> instances, stuff we regularly see. MDE AIR can clean up a device, remove any scheduled tasks, delete the service, erase the file, and simply tell us that the problem has been remediated. We can make choices about whether to even look at it, which reduces a lot of noise for our SOC.<\/p>\n

These are some of the ways we go from 10,000 monthly alerts down to a manageable 3,500 cases for investigation.<\/p>\n

But there are other ways to reduce the number of security alerts an organization must respond to.<\/p>\n

Sometimes several events are related and can be correlated into an incident. Microsoft 365 Defender uses threat intelligence to find these relationships<\/a>. Knowing the association between events, a SOC can be more efficient in its efforts, reducing the number of events along with duplicated efforts performed by multiple analysts.<\/p>\n

Machine learning and data science can show how impactful data can be. As event data grows, divergent datasets can be merged together to assist threat hunters. There is a function in Microsoft Azure Sentinel called Fusion that can spot ransomware in the background<\/a>. The solution can also automatically detect multistage attacks<\/a> and recognize patterns in data that would otherwise be too complex to see.<\/p>\n

Making it easy for the SOC<\/h2>\n

This is how we get from 3,500 cases to the actual security threats that need our attention. But it\u2019s not only about the volume of events\u2014it\u2019s also about how we spend our time.<\/p>\n

We want to be able to dig through the data, but when our analysts need to act, we want all that data to be curated and ready to go. Some of this is done through additional layers of automation, like bots that use the Microsoft Defender APIs<\/a> to pull important data for an analyst. Developers within the SOC team utilized the Microsoft Bot Framework<\/a> to make it easier for our analysts to get the data they need quickly and efficiently by connecting directly to these robust APIs. Here\u2019s an example:<\/p>\n

\"Sherlock
Sherlock Bot is one of the many automation tools created and used by SOC analysts to quickly investigate a security event.<\/figcaption><\/figure>\n

This saves time for the analyst having to go back and forth between screens to get data\u00a0or choose remediation actions<\/span>, all the while helping the analyst determine if the alert is a true positive that needs cleanup or a benign or false positive to add to the list for exclusion.<\/p>\n

A SOC can also leverage playbooks with automation rules<\/a> to further improve analysts\u2019 efficiency by assigning tasks to the right team, giving further resolution to incidents, and clearing out known false positives. Automation makes things easier, more efficient, and more accurate. It empowers our SOC to make strategic decisions instead of focusing on manual steps.<\/p>\n

The winnowing we do dramatically reduces the number of events that we need to respond to. For every 3,500 security events we investigate each month, only about 500 require remediation from Microsoft. This system allows us to quickly identify and respond to those real threats.<\/p>\n

Why quality, not quantity, matters<\/h2>\n

We collect a lot of events each month, but we don\u2019t have time to investigate everything. And not everything is worthy of our SOC\u2019s time. Most of our threats are dealt with by Zero Trust and good security hygiene, but we\u2019re still going to be cautious and perform due diligence.<\/p>\n

The solutions and practices we have in place help funnel these security events into a manageable number by eliminating innocuous events, resolving low-risk items, and remediating common problems on behalf of our SOC analysts. When the distilled batch of cases arrive for investigation, our SOC can leverage automation and other tools to work efficiently, quickly responding to items that really matter.<\/p>\n

That\u2019s how we get from 600 billion monthly events, a 12-digit number, down to 500 remediations, or 3 digits\u2019 worth of action items.<\/p>\n

\"Key<\/p>\n

Here are three things you can do to get started at your company:<\/p>\n