As originally posted on Security Boulevard, January 24, 2018
When it comes to anomalies, the sheer volume can be overwhelming. How do you separate the potentially malicious from the purely anomalous? One strategy is to use the adversary mindset as a framework for understanding what an adversary would do and why, and then examine how these anomalies line up with the framework of what a threat actor would do. This is the type of situation in which artificial intelligence and machine learning can be used effectively. So how can organizations apply this approach and use AI/ML to their advantage? Read on.
Anomaly Overload
When appropriately tooled (or engineered) to detect anomalies, enterprises often discover a high volume of them. There can be literally thousands of them generating alerts daily or weekly. These anomalies may be relevant for security, IT or policy reasons—or a combination thereof. At a high level, the problem is that some degree of analysis is required to determine the importance of each anomaly and what follow-on steps are appropriate.
The problem then is that much of the security analyst’s time is spent chasing down what often end up as IT or policy issues, not security ones. These security false positives take up valuable analytical time that could be used in more high-value projects and may give the actual security events more time to wreak havoc without detection. Thus, the network becomes even less secure.
The Adversary Mindset
How do you address this overwhelming number of anomalies in a more nuanced way? It’s important to understand what they are within the context of your network and which ones are actually malicious. Adopting an adversary mindset is key.
In terms of security, to extract value out of your anomaly detection solution, you need to think about the ways your adversaries would gain access to your network:
- Think about their objectives: Are they trying to steal and leak data? Disrupt business? Gather competitive intel?
- Consider how they achieve those objectives: What actions must they take to achieve their goals?
Use this framework to understand the anomalies you are seeing.
The Tactical Advantage of AI
Where does AI fit it? One of the core challenges here is how to find the anomalies in the first place. There are a couple of different ways to tackle this, and AI comes into play in both a tactical and strategic way.
At the tactical level, AI can be used to get a deep contextual understanding of the network and detect anomalies based on known and previously seen modes of behavior between hosts and between users, within the network.
With that, AI can help security professionals understand what “network normal” looks like. It’s critical that the network’s model of normal is constantly updated. Networks are incredibly dynamic and can change daily and hourly, at the minimum. This dynamic nature means you must have a way to continually update your understanding of what the network normally looks like to have a rolling baseline that is accurate. AI provides a way to address this problem.
The Strategic Advantage of AI
Though it’s important to consider how an adversary thinks and acts, putting these insights into practice can be extremely difficult given the volume of data coming out of a large enterprise network. In this machine-scale era, you are likely to be dealing with a scale of data that is far beyond what a human can really comprehend and link together.
Not all of this data is available in the same location, either. To effectively detect anomalies, you need the ability to access data from a variety of what likely are siloed data sources. AI really starts to lend its value here. Applied appropriately, AI can be used to link these previously siloed data sources together in a way that humans just cannot do by eyeing it.
Cutting Through the Noise
Today’s network security systems can become victims of their own success, so good at spotting anything out of the ordinary that IT security teams are deluged with more alerts than they can possible wade through. This creates alert fatigue rather than stronger security. Artificial intelligence is able to apply an understanding of how adversaries think and what methods they use to the anomaly landscape. This allows it to weed out the majority of harmless anomalies to focus on those that pose an actual threat to the network.
