Nebulock does a lot of work behind the scenes to make sure the agentic AI has access to the right information, in the right form, at the right time. We ingest data from many different sources for each of our customers, including EDR, SIEM, and more. The data within these sources is voluminous and dense, and we take great pains to find patterns in this data that could mean insights for our customers, no matter how subtle or hidden they might be. We use more than just one agent to process and manage data, for the best possible results. Sometimes, using classical machine learning is the best strategy for efficiency, quality, cost, and customer results.
What are the risks of shadow AI tools? For this discussion, I’m going to use our recently developed model that detects behavioral signs of AI automation or shadow AI usage. New AI tools are being developed all the time, by actors large and small, and they don’t all meet the security and liability requirements organizations have in place, such as:
Trustworthiness Inappropriate or missing data privacy policies Taking actions that are inappropriate to automate Possibility of malware, which can exfiltrate data without the user’s knowledge One of the biggest challenges to finding these shadow AI tools is that the newest and most untrustworthy tools may be unknown. We might not even know the name of the tool until it’s already been caught taking malicious or suspect actions.
How to find shadow AI tools acting on a network So, how do we solve this problem? A simple approach is to search processes and commands for key terms, such as the names of AI tools. This is a good start, with low overhead and requiring minimal technical expertise, but it doesn’t help to identify AI that is cutting edge and not yet known in the cybersecurity space.
An LLM? Maybe not the best approach Some people might just try to throw a large language model (LLM) at the problem, for example making an AI agent guess when a process is spawned by AI, based on whatever criteria it wants. However, this approach has some significant flaws.
Lack of Predictability/Reproducibility LLMs are nondeterministic, making results often unreliable and evaluation difficult to trust. For language generation the nondeterminism is a great feature. But for data analysis and classification modeling their inconsistent outputs are a real problem. When analyzing data, we want to ensure that each time we use the same input, we get the same correct output back. Even with temperature set to 0, LLMs can show subtle non-determinism, and their reasoning processes aren’t fixed mathematical functions.
Difficulty with Interpretability and Tuning We want to know why our model asserted that a process was AI, and to be able to explain that if customers need that detail. Interpretability is also important to direct our tuning of the model’s choices to perfectly align with our objectives: identifying what we define as “shadow AI,” and what types of processes to ignore for business needs. By knowing why and how the model arrived at an output, we can make appropriate adjustments where needed. But with an LLM, prompt engineering is really the only tuning option available, which limits the levers of control.
Potentially high costs Token costs are an important reality for AI users. Using an LLM agent to analyze whether processes were AI incurs varying token costs for each process examined. There’s no real way to predict the number of tokens that each agent call would use, making budgeting a huge challenge.
Why machine learning is a better approach We concluded that an LLM alone was not the right choice for this problem. Instead, we opted to use a classical machine learning (ML) technique known as gradient boosted machines (GBMs) , using a particular algorithm called Catboost.
Neural network-based models like LLMs learn by tuning huge numbers of internal parameters, allowing them to detect complex, layered patterns with minimal explicit guidance. This makes them well suited to messy, high-volume data like text and images, but what's happening inside the model is difficult or impossible to trace or explain. Classical ML techniques, like GBMs, logistic regression, or random forests, take a more structured approach, using explicit rules or decision trees rather than layered abstraction. When working with organized numeric data, ML approaches provide the following advantages over LLMs:
Efficiency Transparency Accuracy Affordability
What is a Gradient Boosted Machine (GBM) model? GBM techniques accept features, or attributes for different rows of data, and a target or ground truth label. For our example, the target is “AI or not AI,” and the features are assorted datapoints about each process and its context/surroundings, such as “number of child processes” or “time between spawn and parent process spawn.” Our model includes about 50 such features, all measuring different concepts. When a GBM is training, it builds a long chain of simple small decision trees, using features such as “number of child processes is > 5.” For each tree, the results are compared with the truth for each case, and the modeler identifies the mistakes. Each new tree focuses on fixing the mistakes the previous ones made, and hundreds are eventually constructed. At the end, the results of these trees are all combined together mathematically, and they create a final score between 0 and 1, like 0.92. That number represents the model's estimated probability that the process is AI, rather than a hard yes/no answer. You can then choose a threshold based on your risk tolerance.
Our AI classifier model doesn’t rely on matching text strings, such as “claude” or “gpt.” Instead we created a model by identifying chains of AI-spawned processes and labeling them all. This means, for example, that a process that is three direct hops downstream of a process that was spawned by “claude” can be identified as AI-related, even if it doesn’t individually have that “smoking gun” term anywhere in its metadata.
AI usage synthetic example: single decision tree from a gradient boosted machine model
This enabled us to train our model using features derived from the behavior patterns of these chains, such as how these processes are run, their shape/format, metadata, and their relationships to other processes - but not including any features that look for the names of AI tools. Our pure behavior model, then, identifies with high accuracy when AI tools are in use on an endpoint, even when we don’t know their names or when the names are obfuscated intentionally. We’re spotting cases when processes behave like AI.
Let's examine why this modeling strategy is the right choice.
True predictability/reproducibility Deterministic results are one of the great advantages of GBM models. When you pass the same input to it, you always receive the same output back. Because of this property, we can conduct audits of model activity, easily debug any problems, and measure the model’s performance with confidence. Using an LLM, we could try to measure performance on repeated model calls, but the behavior we see in testing won't always be comparable to what the LLM does in production. With our AI classifier model that’s not an issue.
Better interpretability and easier tuning Tree-based models, like those built using GBMs, have robust interpretation capabilities. We can map the model's trees and identify with precision why a model prediction is what it is and explain what inputs contributed most to the value. Also, the output of the model is not just a true or false binary. It's a calculated probability score based on empirical results of training. You can use this score to assess how confident you are with the case's classification. You could ask an LLM to assign probability to its output, but the number it generates will simply be a guess. It won’t be based on any empirical calculation.
Lower, more predictable costs A GBM model can run on resources as small as a local laptop, with zero outside services involved, calling no third parties such as Anthropic or OpenAI, and using no tokens at all. This cost savings is accompanied by latency improvements since there's no concern about third-party service uptime or I/O issues. We also have end-to-end control over the entire process, including the training and the inference, so data passed to this model or used in its training never leaves Nebulock’s servers.
Training your own classical ML model might be a little more work on the front end than just calling an LLM and asking it to complete the task, but once the training is done, the model object is:
Lightweight Easy to deploy Easy to retrain when new data or new business objectives appear How — and why — it works Because of this ML approach, the probability of a process being AI-run is easily accessible to our agentic AI. It can combine this information with other details about a process or event to inform its analysis of the threat posed by behavior on the network.
We chose to build this classifier and continue to build classical ML models to serve insights to our threat hunt agent because our intention is to be the best possible stewards of our customers’ trust. Customers expect Nebulock to provide the most accurate and precise results, keeping them proactive in the pursuit of cybersecurity threats, and they also expect us to understand their desire to be responsible with budgets. Anybody can throw more tokens at a problem, but there’s no guarantee it will produce good results. We choose to be conscientious about finding the best solution to the problem, and sometimes using established, reliable techniques instead of the newest shiny thing on the market is the correct move.
We need to acknowledge, of course, that all machine learning models have a risk of error, or making mistakes in their inferences. Our classification models are no exception. However, we proactively measure these mistakes and continually work to improve our model performance. With the explainability and interpretability they provide we can promise continuous improvement in a way that is incredibly difficult when a black box LLM is the only tool at your disposal. And the efficiency, transparency, accuracy, and affordability the models provide make them a real, viable alternative to an LLM.
Come build with us Integrating classical machine learning into agentic AI systems is only a small part of how we're building Nebulock. We're growing the team, and if you're interested in helping make security operations faster, safer, and more efficient for our customers we'd love to hear from you.
.webp)
