System, method and article of manufacture for detecting collisions between video images generated by a camera and an object depicted on a display

• US 8,433,094 B1
• Filed: 12/30/2003
• Issued: 04/30/2013
• Est. Priority Date: 07/30/1999
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Nguyen, Katerina H.
• Primary Examiner(s): Strege, John
• Application Number:
  US10/749,797
• Time in Patent Office: 3,409 Days
• Field of Search: 382/107, 382/103, 382/203, 382/291, 382/224, 348/169
• US Class Current: 382/103
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/643 by determining the impact between objects, e.g. collision detection
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: Y10S 345/958 Collision avoidance

Abstract

A system, method and article of manufacture are provided for detecting collisions or any other type of interactions between video images and an object depicted on a display. First, video images generated by a camera are received. Upon receipt, a first collision detection operation is executed for generating a first confidence value representative of a confidence that the received video images have collided with an object depicted on a display. Also executed is a second collision detection operation for generating a second confidence value representative of a confidence that the received video images have collided with the object depicted on the display. The first confidence value and the second confidence value are then made available for various applications. Such applications may depict an interaction between the video images and the object depicted on the display based on the first confidence value and the second confidence value.

Claims

1. A computer-implemented method of detecting interactions between video images and an animated object depicted on a display, comprising:

generating a motion distribution of a person image in images captured as part of a video sequence by utilizing frame differencing;

estimating a location of a body part of the person image in the images;

estimating a size of a body in the person image in the images;

determining valid ranges of motion that can be caused by a person moving any parts of the person'"'"'s body based on the estimated location of the body part of the person image in the images and the estimated size of the body in the person image in the images;

eliminating motion outside of the ranges of motion determined to be valid;

depicting an animated object on a display, the animated object initially having at least one of a first speed and a first direction;

detecting an interaction of the body part of the person image in the images and the animated object depicted on the display; and

generating, by a computer, at least one of a second speed and a second direction of the animated object depicted on the display based on an interaction between the motion of the body part of the person image in the images and the animated object depicted on the display.

2. The computer-implemented method of claim 1 further including assuming a location of the body part is in a center of the motion distribution if there is no body part tracking.

3. The computer-implemented method of claim 1 further including determining a confidence that the interaction has been detected.

4. The computer-implemented method of claim 1, further comprising determining a bounding box about a head of the person in the images,

and wherein the determining valid ranges of motion is further based on the determined bonding box.

5. The computer-implemented method of claim 1, further comprising determining a bounding box about a torso of the person in the images,

and wherein the determining valid ranges of motion is further based on the determined bonding box.

6. A system for detecting interactions between video images and an animated object depicted on a display, comprising:

an input device configured to receive video images; and

logic configured to;

generate a motion distribution of a person image in video images captured as part of a video sequence by utilizing frame differencing,

estimate a location of a body part of the person image in the video images,

estimate a size of a body in the person image in the images;

determine valid ranges of motion that can be caused by a person moving any parts of the person'"'"'s body based on the estimated location of the body part of the person image in the video images and the estimated size of the body in the person image in the images,

eliminate motion outside of the valid ranges of motion determined to be valid,

depict an animated object on a display, the animated object initially having at least one of a first speed and a first direction, and

detect an interaction of the body part of the person image in the video images and the animated object depicted on the display; and

generate at least one of a second speed and a second direction of the animated object depicted on the display based on an interaction between the motion of the body part of the person image in the video images and the animated object depicted on the display.

7. The system of claim 6, further comprising logic configured to determine a bounding box about a head of the person in the images,

and wherein the logic configured to determine valid ranges of motion is configured to determine valid ranges of motion further based on the determined bounding box.

8. The system of claim 6, further comprising logic configured to determine a bounding box about a torso of the person in the images,

and wherein the logic configured to determine valid ranges of motion is configured to determine valid ranges of motion further based on the determined bounding box.

9. A computer readable storage device storing instructions that are executable by a computer to cause the computer to detect interactions between video images and an animated object depicted on a display, by:

generating a motion distribution of a person image in video images by utilizing frame differencing;

estimating a location of a body part of the person image in the video images;

estimating a size of a body in the person image in the images;

determining valid ranges of motion that can be caused by a person moving any parts of the person'"'"'s body based on the estimated location of the body part of the person image in the video images and the estimated size of the body in the person image in the images;

eliminating motion outside of the ranges of motion determined to be valid;

depicting an animated object on a display, the animated object initially having at least one of a first speed and a first direction; and

detecting an interaction of the body part of the person image in the video images and the animated object depicted on the display; and

generating at least one of a second speed and a second direction of the animated object depicted on the display based on an interaction between the motion of the body part of the person image in the video images and the animated object depicted on the display.

10. The computer readable storage device of claim 9 further storing computer instructions for assuming a location of the body part is in a center of the motion distribution if there is no body part tracking.

11. The computer readable storage device of claim 9 further storing computer instructions for determining a confidence that the interaction has been detected.

12. The computer readable storage device of claim 9 further storing computer instructions for determining a bounding box about a head of the person in the images,

and wherein the computer instructions for determining valid ranges of motion determine valid ranges of motion further based on the determined bounding box.

13. The computer readable storage device of claim 9 further storing computer instructions for determining a bounding box about a torso of the person in the images,

and wherein the computer instructions for determining valid ranges of motion determine valid ranges of motion further based on the determined bounding box.

14. A computer-implemented method of detecting interactions between a person image in a number of video images and an animated object depicted on a display, comprising:

analyzing two or more frames of video images captured as part of a video sequence and generating a motion distribution of a person image in the video images by utilizing frame differencing;

estimating motion of a body part of the person image in the video images;

estimating a size of a body in the person image in the images;

determining valid ranges of motion that can be caused by a person moving any parts of the person'"'"'s body based on the estimated motion of the body part of the person image in the video images and the estimated size of the body in the person image in the images;

eliminating motion of the body part of the person image outside of the ranges of motion determined to be valid;

detecting an interaction between an animated object depicted on a display and the estimated motion of the body part of the person image; and

generating, by a computer, at least one of a speed and a direction of the animated object depicted on the display based on the detected interaction between the estimated motion of the body part of the person image and the animated object depicted on the display.

15. The computer-implemented method of claim 14 further including assuming a location of the body part is in a center of the motion distribution if there is no body part tracking.

16. The computer-implemented method of claim 14 further including determining a confidence that the interaction has been detected.

17. The computer-implemented method of claim 14, further comprising determining a bounding box about a head of the person in the images,

and wherein the determining valid ranges of motion is further based on the determined bounding box.

18. The computer-implemented method of claim 14, further comprising determining a bounding box about a torso of the person in the images,

and wherein the determining valid ranges of motion is further based on the determined bounding box.

19. A system for detecting interactions between video images and an animated object depicted on a display, comprising:

an input device configured to receive video images; and

logic configured to;

analyze two or more frames of the video images and generate a motion distribution of a person image in the video images by utilizing frame differencing,

estimate a motion of a body part of the person image in the video images,

estimate a size of a body in the person image in the images;

determine valid ranges of motion that can be caused by a person moving any parts of the person'"'"'s body based on the estimated motion of the body part of the person image in the video images and the estimated size of the body in the person image in the images,

eliminate motion of the body part of the person image in the video images that is outside of the ranges of motion determined to be valid,

detect an interaction between an animated object and the estimated motion of the body part of the person image in the video images, and

generate at least one of a speed and a direction of the animated object depicted on the display based on the detected interaction between the motion of the body part of the person image in the video images and the animated object depicted on the display.

20. The system of claim 19, further comprising logic configured to determine a bounding box about a head of the person in the images,

and wherein the logic configured to determine valid ranges of motion is configured to determine valid ranges of motion further based on the determined bounding box.

21. The system of claim 19, further comprising logic configured to determine a bounding box about a torso of the person in the images,

and wherein the logic configured to determine valid ranges of motion is configured to determine valid ranges of motion further based on the determined bounding box.

22. A computer readable storage device storing instructions that are executable by a computer to cause the computer to detect interactions between video images and an animated object depicted on a display, by:

analyzing two or more frames of video images captured as part of a video sequence and generating a motion distribution of a person image in the video images by utilizing frame differencing;

estimating motion of a body part of the person image in the video images;

estimating a size of a body in the person image in the images;

determining valid ranges of motion that can be caused by a person moving any parts of the person'"'"'s body based on the estimated motion of the body part of the person image in the video images and the estimated size of the body in the person image in the images;

eliminating motion of the body part of the person image outside of the ranges of motion determined to be valid;

detecting an interaction between an animated object depicted on a display and the estimated motion of the body part of the person image in the video images; and

generating at least one of a speed and a direction of the animated object depicted on the display based on the interaction between the estimated motion of the body part of the person image in the video images and the animated object depicted on the display.

23. The computer readable storage device of claim 22 further storing computer instructions for assuming a location of the body part is in a center of the motion distribution if there is no body part tracking.

24. The computer readable storage device of claim 22 further storing computer instructions for determining a confidence that the interaction has been detected.

25. The computer readable storage device of claim 22 further storing computer instructions for determining a bounding box about a head of the person in the images,

and wherein the computer instructions for determining valid ranges of motion determine valid ranges of motion further based on the determined bounding box.

26. The computer readable storage device of claim 22 further storing computer instructions for determining a bounding box about a torso of the person in the images,

and wherein the computer instructions for determining valid ranges of motion determine valid ranges of motion further based on the determined bounding box.

27. A system for generating at least one of a speed and direction of an animated object based on interactions between video images and the animated object depicted on a display, comprising:

means for receiving video images;

means for determining motion of a body part of a person image in video images by generating a motion distribution of the person image in the video images by utilizing frame differencing, estimating a location of the body part of the person image in the video images, estimating a size of a body in the person image in the images, determining valid ranges of motion that can be caused by a person moving any parts of the person'"'"'s body based on the estimated location of the body part of the person image in the video images and the estimated size of the body in the person image in the images, and eliminating motion outside of the valid ranges of motion determined to be valid;

means for detecting an interaction of the body part of the person image in the video images and an animated object depicted on a display; and

means for generating at least one of a speed and a direction of the animated object depicted on the display based on the interaction between the motion of the body part of the person image in the video images and the animated object depicted on the display.

28. The system of claim 27 wherein the means for determining motion of a body part does so by further determining a bounding box about a head of the person in the images,

and the means for determining motion of a body part determines valid ranges of motion further based on the determined bounding box.

29. The system of claim 27 wherein the means for determining motion of a body part does so by further determining a bounding box about a torso of the person in the images,

and the means for determining motion of a body part determines valid ranges of motion further based on the determined bounding box.