Abstract
A novel approach to malware classification is introduced based on analysis of instruction traces that are collected dynamically from the program in question. The method has been implemented online in a sandbox environment (i.e., a security mechanism for separating running programs) at Los Alamos National Laboratory, and is intended for eventual host-based use, provided the issue of sampling the instructions executed by a given process without disruption to the user can be satisfactorily addressed. The procedure represents an instruction trace with a Markov chain structure in which the transition matrix, P, has rows modeled as Dirichlet vectors. The malware class (malicious or benign) is modeled using a flexible spline logistic regression model with variable selection on the elements of P, which are observed with error. The utility of the method is illustrated on a sample of traces from malware and nonmalware programs, and the results are compared to other leading detection schemes (both signature and classification based). This article also has supplementary materials available online.
Original language | English (US) |
---|---|
Pages (from-to) | 1-18 |
Number of pages | 18 |
Journal | Annals of Applied Statistics |
Volume | 8 |
Issue number | 1 |
DOIs | |
State | Published - Mar 2014 |
Keywords
- Adaptive lasso
- Classification
- Elastic net
- Empirical bayes
- Logistic regression
- Malware detection
- Relaxed lasso
- Splines
ASJC Scopus subject areas
- Statistics and Probability
- Modeling and Simulation
- Statistics, Probability and Uncertainty