The average corporate security organization spends $18 million annually but is largely ineffective at preventing breaches, IP theft and data loss. Why? The fragmented approach we’re currently using in the security operations center (SOC) doesn’t work.
Here’s an instant refresher on security operations and how we got where we are today: A decade ago, we protected our applications and websites by monitoring event logs — digital records of every activity that occurred in our cyber environment, which range from logins to emails to configuration changes. Logs were audited, flags were raised, suspicious activities were investigated, and data was stored for compliance purposes.
The security-driven data stored in a data lake can be in its native format, structured or unstructured, and therefore dimensional, dynamic and heterogeneous, which gives data lakes their distinction and advantage over data warehouses.
As malicious actors and adversaries became more active, and their tactics, techniques and procedures (or TTP’s, in security parlance) grew more sophisticated, simple logging evolved into an approach called “security information and event management” (SIEM), which involves using software to supply real-time analysis of security alerts generated by applications and network hardware. SIEM software uses rule-driven correlation and analytics to turn raw event data into potentially valuable intelligence.
Although it was no magic bullet (it’s challenging to implement and make everything work properly), the ability to find the so-called “needle in the haystack” and identify attacks beginning was a huge step forward.
Today, SIEMs remain, and the market is largely led by Splunk and IBM QRadar. Of course, the technology has advanced significantly because new use cases emerge constantly. Many companies have finally moved into cloud-native deployments and are leveraging machine learning and sophisticated behavioral analytics. However , new enterprise SIEM deployments are fewer, costs are greater, and — most importantly — the entire needs of the CISO and the hard-working team in the SOC have changed.
New security demands are asking too much of SIEM
First, data has exploded and SIEM is too narrowly focused. The mere number of security events is no longer sufficient because the aperture with this dataset is too narrow. While there is likely a massive amount of event data to capture and process from your events, you’re missing out on vast amounts of more information such as OSINT (open-source intelligence information), consumable external-threat feeds, and valuable information such as for instance malware and IP reputation databases, as well as reports from dark web activity. There are endless sources of intelligence, too many for the dated architecture of a SIEM.
Additionally , data exploded alongside costs. Data explosion + hardware + license costs = spiraling total cost of ownership. With so much infrastructure, both physical and virtual, the amount of information being captured has exploded. Machine-generated data has grown at 50x, while the average security budget grows 14% year on year.
The cost to store this information makes the SIEM cost-prohibitive. The average cost of a SIEM has skyrocketed to close to $1 million yearly , which is only for license and hardware costs. The economics force teams in the SOC to capture and/or retain less information so that they can keep costs in check. This causes the effectiveness of the SIEM to become even further paid down. I recently spoke with a SOC team who wished to query large datasets searching for evidence of fraud, but doing this in Splunk was cost-prohibitive and a slow, arduous process, leading the team to explore alternatives.
The shortcomings of the SIEM approach today are dangerous and terrifying. A recent survey by the Ponemon Institute surveyed almost 600 IT security leaders and found that, despite spending an average of $18. 4 million annually and having an average of 47 products and services, a whopping 53% of IT security leaders “did not know if their products were even working. ” It’s clearly time for change.
