PSI-NetVisor: Program semantic aware intrusion detection at network and hypervisor layer in cloud

Abstract

<p>Cloud Security is of paramount importance in the new era of virtualization technology. Tenant Virtual Machine (VM) level security solutions can be easily evaded by modern attack techniques. Out-VM monitoring allows cloud administrator (CA) to monitor and control a VM from a secure location outside the VM. In this paper, we propose an out-VM monitoring based approach named as ‘<b>P</b>rogram <b>S</b>emantic-Aware <b>I</b>ntrusion Detection at <b>Net</b>work and Hyper<b>visor</b> Layer’ (<i>PSI-NetVisor</i>) to detect attacks in both network and virtualization layer in cloud. <i>PSI-NetVisor</i> performs network monitoring by employing behavior based intrusion detection approach (BIDA) at the network layer of centralized Cloud Network Server (CNS); providing the first level of defense from attacks. It incorporates semantic awareness in the intrusion detection approach and enables it to provide network monitoring and process monitoring at the hypervisor layer of Cloud Compute Server (CCoS); providing the second level of defense from attacks. <i>PSI-NetVisor</i> employs Virtual Machine Introspection (VMI) libraries based on software break point injection to extract process execution traces from hypervisor. It further applies depth first search (DFS) to construct program semantics from control flow graph of execution traces. It applies dynamic analysis and machine learning approaches to learn the behavior of anomalies which makes it secure from obfuscation and encryption based attacks. <i>PSI-NetVisor</i> has been validated with latest intrusion datasets (UNSW-NB & Evasive Malware) collected from research centers and results seem to be promising.</p>