Cross-VM network filtering

• US 8,893,274 B2
• Filed: 08/03/2011
• Issued: 11/18/2014
• Est. Priority Date: 08/03/2011
• Status: Active Grant

• Current Assignee: Trend Micro Inc.
• Original Assignee: Trend Micro Inc.
• Inventors: Zhu, Minghang, Qian, Gongwei
• Primary Examiner(s): Kim, Jung
• Assistant Examiner(s): Wilcox, James J
• Application Number:
  US13/197,701
• Publication Number:
  US 20130036470A1
• Time in Patent Office: 1,203 Days
• US Class Current: 726/23
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms

Abstract

A security virtual machine inspects all data traffic between other virtual machines on a virtualization platform in order to prevent an inter-VM attack. Data traffic between the machines is intercepted at the privileged domain and directed to the security virtual machine via a hook mechanism and a shared memory location. The traffic is read by the security machine and analyzed for malicious software. After analysis, the security machine sends back a verdict for each data packet to the privileged machine which then drops each data packet or passes each data packet on to its intended destination. The privileged domain keeps a copy of each packet or relies upon the security machine to send back each packet. The security machine also substitutes legitimate or warning data packets into a malicious data package instead of blocking data packets. The shared memory location is a circular buffer for greater performance. Traffic is intercepted on a single host computer or between host computers.

Claims

1. A method of filtering data traffic in a virtualization environment, said method comprising:

receiving, at a privileged virtual machine executing a privileged domain responsible for managing communication between virtual machines, a data packet from a first virtual machine destined for a second virtual machine, said privileged virtual machine executing upon a virtualization platform on a host computer, wherein data packets of said virtualization environment pass through said privileged virtual machine;

intercepting said data packet by said privileged virtual machine;

sending said data packet from said privileged virtual machine to a memory location in said host computer shared between said privileged virtual machine and a security virtual machine executing a security domain on said virtualization platform, said sending not utilizing a virtual network of said virtualization environment, said security virtual machine being a different virtual machine than said privileged virtual machine;

receiving at said privileged virtual machine a verdict from said security virtual machine regarding said data packet; and

passing said data packet to said second virtual machine or dropping said data packet based upon said verdict.

2. The method as recited in claim 1 further comprising:

intercepting said data packet by using a hooking routine in the operating system of said privileged virtual machine, whereby said data packet is sent to said memory location instead of being sent to said second virtual machine.

3. The method as recited in claim 1 wherein said first and second virtual machines are both executing upon said virtualization platform on said host computer.

4. The method as recited in claim 1 wherein said memory location has been allocated by said security virtual machine in a memory of said host computer.

5. The method as recited in claim 1 wherein said verdict indicates passing said data packet, said method further comprising:

receiving said data packet from said security virtual machine along with said verdict.

6. The method as recited in claim 1 wherein said verdict indicates passing said data packet, said method further comprising:

holding a copy of said data packet at said privileged virtual machine after said sending, wherein said verdict received from said security virtual machine does not include said data packet.

7. The method as recited in claim 1 wherein said verdict indicates blocking said data packet, said method further comprising:

dropping said data packet by said security virtual machine, wherein a copy of said data packet is not retained by said privileged virtual machine after said sending.

8. The method as recited in claim 1 further comprising:

reading said data packet from said shared memory location by said security virtual machine; and

analyzing said data packet by said security virtual machine.

9. The method as recited in claim 1 further comprising:

granting access by said security virtual machine to said privileged virtual machine to write data packets into said shared memory location.

10. The method as recited in claim 1 further comprising:

intercepting said data packet by defining a rule in a pre-routing chain, in a forward chain, or in a post-routing chain of a bridge or a switch in said privileged virtual machine.

12. The method as recited in claim 1 wherein said verdict is a block verdict for said data packet, and wherein said indication of said data packet is a packet identifier.

11. A method of filtering data traffic in a virtualization environment, said method comprising:

reading, by a security virtual machine executing a security domain, a plurality of data packets from a memory location in a host computer shared between said security virtual machine and a privileged virtual machine executing a privileged domain responsible for managing communication between virtual machines, said security virtual machine executing upon a virtualization platform on said host computer, wherein data packets of said virtualization environment pass through said privileged virtual machine, and wherein said reading not utilizing a virtual network of said virtualization environment, said security virtual machine being a different virtual machine than said privileged virtual machine;

assembling said data packets into a data package;

analyzing said data package by said security virtual machine to determine if malicious software is present;

determining a pass or a block verdict for one of said data packets based upon said analyzing; and

sending said verdict to said privileged virtual machine along with an indication of said data packet.

13. The method as recited in claim 11 wherein said verdict is a pass verdict for said data packet, said method further comprising:

sending said data packet along with said verdict to said privileged virtual machine.

14. The method as recited in claim 11 wherein said verdict is a pass verdict for said data packet, said method further comprising:

sending said pass verdict to said privileged virtual machine along with a packet identifier for said data packet without sending said data packet.

15. The method as recited in claim 11 further comprising:

modifying at least one of said data packets to eliminate said malicious software; and

sending said at least one of said data packets to said privileged virtual machine.

16. The method as recited in claim 11 wherein said memory location is a buffer allocated for use by said security virtual machine, said method further comprising:

granting access by said security virtual machine to said privileged virtual machine to write data packets into said memory location.

17. The method as recited in claim 11 further comprising:

intercepting said data packets between a first virtual machine and a second virtual machine by defining a rule in a pre-routing chain, in a forward chain, or in a post-routing chain of a bridge or a switch in said privileged virtual machine.

18. A method of modifying data traffic in a virtualization environment, said method comprising:

reading, by a security virtual machine executing a security domain, a plurality of data packets from a memory location in a host computer shared between said security virtual machine and a privileged virtual machine executing a privileged domain responsible for managing communication between virtual machines, said security virtual machine executing upon a virtualization platform on said host computer, wherein data packets of said virtualization environment pass through said privileged virtual machine, and wherein said reading not utilizing a virtual network of said virtualization environment, said security virtual machine being a different virtual machine than said privileged virtual machine;

assembling said data packets into a data package;

detecting malicious software in said data package;

replacing, by said security virtual machine, a subset of said data packets to eliminate said malicious software in said data package; and

returning said modified data package to said privileged virtual machine.

19. The method as recited in claim 18 wherein said data packets originate with a first virtual machine and are destined for a second virtual machine, said method further comprising:

delivering said modified data package to said second virtual machine.

20. The method as recited in claim 18 further comprising:

writing said data packets into said memory location by said privileged virtual machine; and

said privileged virtual machine not retaining a copy of said data packets after said writing.

21. The method as recited in claim 18 wherein said data packets originate with a first virtual machine and are destined for a second virtual machine, said method further comprising:

not delivering said data package to said second virtual machine before said step of returning.

22. The method as recited in claim 18 further comprising:

returning only said subset of said data packets to said privileged domain; and

returning a packet identifier for each data packet in said data package that is not in said subset to said privileged domain.

23. The method as recited in claim 18 further comprising:

detecting said malicious software in said data package by a software application executing upon said security virtual machine.

24. The method as recited in claim 18 wherein said data packets originate with a first virtual machine and are destined for a second virtual machine, and wherein said first virtual machine and said second virtual machine are located upon different host computers.

25. The method as recited in claim 18 further comprising:

granting access by said security virtual machine to said privileged virtual machine to write data packets into said shared memory location.

26. The method as recited in claim 18 further comprising:

intercepting said data packets between a first virtual machine and a second virtual machine by defining a rule in a pre-routing chain, in a forward chain, or in a post-routing chain of a bridge or a switch in said privileged virtual machine.