Trigger-based malware samples exhibit untrustworthy aims provided that certain environmental conditions
are satisfied. We have categorized trigger-based malware behaviours into two classes, i.e.,
invasive and evasive. Invasive behaviour is concerned with malicious activities, while malware uses evasive
behaviours for self-defence. In this article, we propose a greedy incremental approach for detecting invasive
trigger-based malicious behaviour. The method proceeds by identifying and supplying resources required by malign
samples. Trigger-based behaviours vary depending on environmental conditions. Such behaviours can be modelled as
directed graphs, where each node represents a state, and edges denote invocation of a specific event.
We define each state in the graph as atomic behaviour, represented using API calls, having legitimate
functionality. This atomic behaviour transforms into harmful action if augmented with additional APIs.
A well-known challenge with behavioural analysis is the proactive identification of malicious activities
before devices are compromised. Timely prediction of malicious behaviours without damaging the systems can be
fulfilled by defining them as atomic actions. To reckon a sub-behaviour as atomic, we propose a novel algorithm.
Besides, a new likelihood statistical significance test, H_test, is suggested to extract frequent subgraphs
of graphs, representing malware family signature. The main message underlying H_test is that the lesser the
number of subgraphs exhibiting benign behaviour, the higher the chances of the sub-graph being chosen as malware
family signature. Experiments conducted on 2320 real malware samples demonstrate a 42% reduction in the number
of signatures than the state-of-the-art methods, namely CDG and CMQDG. Also, 3% and 1.3% improvement in path
coverage compared to the state-of-the-art methods, namely Pytriger and GoldenEye.
KeyWords:
Malware, Behavior Analysis, Trigger-based Behavior, User-Based Behavior, User Activities.
