|
DETECTING MALICIOUS SOFTWARE BY DYNAMICEXECUTION
|
|
|
|
 |
| Title | DETECTING MALICIOUS SOFTWARE BY DYNAMICEXECUTION |
| Author | Dai, Jianyong
|
| Keywords | data mining
virus detection
malicious software
|
| Abstract | Traditional way to detect malicious software is based on signature matching. However, signature matching only detects known malicious software. In order to detect unknown malicious software, it is necessary to analyze the software for its impact on the system when the software is executed. In one approach, the software code can be statically analyzed for any malicious patterns. Another approach is to execute the program and determine the nature of the program dynamically. Since the execution of malicious code may have negative impact on the system, the code must be executed in a controlled environment. For that purpose, we have developed a sandbox to protect the system. Potential malicious behavior is intercepted by hooking Win32 system calls. Using the developed sandbox, we detect unknown virus using dynamic instruction sequences mining techniques. By collecting runtime instruction sequences in basic blocks, we extract instruction sequence patterns based on instruction associations. We build classification models with these patterns. By applying this classification model, we predict the nature of an unknown program. We compare our approach with several other approaches such as simple heuristics, NGram and static instruction sequences. We have also developed a method to identify a family of malicious software utilizing the system call trace. We construct a structural system call diagram from captured dynamic system call traces. We generate smart system call signature using profile hidden Markov model (PHMM) based on modularized system call block. Smart system call signature weakly identifies a family of malicious software. |
| Adviser | Guha, Ratan
|
| Publisher | University of Central Florida |
| Degree | Ph.D.
|
| Degree Discipline | School of Electrical Engineering and Computer Science
|
| Degree Grantor | Engineering and Computer Science
|
| Degree Program | Computer Science PhD
|
| Graduation Date | 2009-01-01 |
| Type | Doctoral dissertation
|
| Access Level | Public - Allow Worldwide Access
|
| Release Date | 2009-09-18 |
| Repository | University Archives
|
| Repository Collection | Electronic Theses and Dissertations
|
| Identifier | CFE0002798 |
| Access Link | http://purl.fcla.edu/fcla/etd/CFE0002798 |
|
|
|
|
|