Manipulation Of Virtual Objects Using Enhanced Interactive System

• US 20080166022A1
• Filed: 12/27/2007
• Published: 07/10/2008
• Est. Priority Date: 12/29/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: Hildreth, Evan
• Granted Patent: US 8,559,676 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The detection of motion of a user via a camera and the generation of a dynamic virtual representation of a user on a display, where the user'"'"'s detected motion causes the dynamic virtual representation to interact with virtual objects on the display. The magnitude and direction of the user'"'"'s detected motion is calculated to determine the magnitude and direction of a force applied by the dynamic virtual representation to the virtual object. Further arrangements include water or smoke fluid simulations, in order to enhance the user experience.

Claims

1. A computer-implemented method comprising:

receiving at least a first image of an object;

detecting the object in the first image;

generating an object data map based on detecting the object in the first image, the object data map indicating for each pixel in the first image, whether the object is detected at the each pixel;

generating a motion history indicating, for each pixel included in the object data map, an indication of time since the object has been detected at the each pixel; and

controlling an application based on the motion history.

2. The method of claim 1 wherein generating a motion history indicating, for each pixel included in the object data map, an indication of time since the object has been detected at the each pixel includes generating a motion history indicating, for each pixel included in the object data map, an opacity value corresponding to the time since the object has been detected at the each pixel.

3. The method of claim 1 wherein generating a motion history indicating, for each pixel included in the object data map, an indication of time since the object has been detected at the each pixel includes generating a motion history indicating, for each pixel included in the object data map, a number of frames since the object has been detected at the each pixel.

4. The method of claim 1 wherein generating a motion history indicating, for each pixel included in the object data map, an indication of time since the object has been detected at the each pixel includes generating a motion history indicating, for each pixel included in the object data map, a measurement of time since the object has been detected at the each pixel.

5. The method of claim 1 wherein:

generating a motion history indicating, for each pixel included in the object data map, an indication of time since the object has been detected at the each pixel includes filtering the motion history; and

controlling an application based on the motion history includes controlling an application based on the filtered motion history.

6. The method of claim 5 wherein filtering the motion history includes, for each pixel, computing a value for the each pixel based on a value of the each pixel and values of pixels neighboring the each pixel.

7. The method of claim 1 wherein receiving at least a first image of an object includes receiving at least a first image of a user.

8. The method of claim 1 wherein controlling an application based on the motion history includes displaying a representation of the object in a display image associated with the application.

9. The method of claim 8 wherein controlling an application based on the motion history includes controlling movement of the representation of the object in the display image associated with the application.

10. The method of claim 8 wherein displaying a representation of the object in a display image associated with the application includes displaying the representation of the object in the display image associated with the application as one or more particles of a particle system.

11. The method of claim 8 wherein displaying a representation of the object in a display image associated with the application includes:

generating an outline contour representing motion of the object based on the motion history; and

determining whether the representation of the object in the display image touches a virtual object rendered in the display image by the application based on the generated outline contour.

12. The method of claim 8 wherein displaying a representation of the object in a display image associated with the application includes displaying a representation of the object such that pixels corresponding to pixels in the motion history at which the indication of time indicates the object has been most recently detected are displayed with an opacity greater than pixels corresponding to pixels in the motion history at which the indication of time indicates the object has been detected further in the past.

13. The method of claim 12 wherein displaying a representation of the object such that pixels corresponding to pixels in the motion history at which the indication of time indicates the object has been most recently detected are displayed with an opacity greater than pixels corresponding to pixels in the motion history at which the indication of time indicates the object has been detected further in the past includes displaying a representation of the object such that pixels corresponding to pixels in the motion history are displayed with an opacity based on the indication of time associated with the pixel.

14. The method of claim 13 displaying a representation of the object such that pixels corresponding to pixels in the motion history are displayed with an opacity based on the indication of time associated with the pixel includes displaying a representation of the object such that a first pixel associated with an indication of time that indicates that the object has been detected more recently at the first pixel than a second pixel appears with an opacity that is greater than an opacity of the second pixel.

15. The method of claim 8 wherein displaying a representation of the object in a display image associated with the application includes:

generating an alpha channel based on the motion history; and

displaying a representation of the object in the display image based on the alpha channel.

16. The method of claim 8 wherein displaying a representation of the object in a display image associated with the application includes displaying an animated effect image based on an animation or a particle system.

17. The method of claim 8 wherein displaying a representation of the object in a display image associated with the application includes displaying a keyed image of the object.

18. The method of claim 8 wherein displaying a representation of the object in a display image associated with the application includes:

determining whether the representation of the object in the display image touches a virtual object rendered in the display image by the application based on the motion history; and

controlling the application based on determination results.

19. The method of claim 18 wherein controlling the application based on determination results includes:

calculating a velocity for the virtual object based on the motion history; and

controlling the application based on the determined velocity for the virtual object.

20. The method of claim 19 wherein:

calculating a velocity for the virtual object includes calculating a velocity and a direction of motion for the virtual object based on optical flow data associated with the motion history; and

controlling the application based on the determined velocity for the virtual object includes controlling the application based on the determined velocity and the determined direction of motion for the virtual object.

21. The method of claim 18, wherein the virtual object comprises a particle of a particle system.

22. The method of claim 1 wherein detecting the object in the first image includes comparing the first image with a background image.

23. A computer-implemented method comprising controlling an application based upon a motion history that indicates, for each pixel included in an object data map, an indication of time since an object has been detected at each pixel, the object data map indicating, for each pixel in a first image received of the object, whether the object is detected at the each pixel.

24. A system comprising:

a camera configured to capture at least a first image of an object;

a processor configured to;

receive the first image of the object from the camera,

detect the object in the first image,

generate an object data map in response to detecting the object in the first image, the object data map indicating, for each pixel in the first image, whether the object is detected at the each pixel,

generate a motion history indicating, for each pixel included in the object data map, an indication of time since the object has been detected at the each pixel, and

control an application based on the motion history; and

a display configured to render a display image associated with the application.

25. A computer readable storage medium having embodied thereon a computer program, the computer program comprising instructions for:

receiving at least a first image of an object;

detecting the object in the first image;

generating an object data map in response to detecting the object in the first image, the object data map indicating, for each pixel in the first image, whether the object is detected at the each pixel;

generating a motion history indicating, for each pixel included in the object data map, an indication of time since the object has been detected at the each pixel; and

controlling an application based on the motion history.

26. A system comprising:

means for receiving at least a first image of an object;

means for detecting the object in the first image;

means for generating an object data map in response to detecting the object in the first image, the object data map indicating, for each pixel in the first image, whether the object is detected at the each pixel;

means for generating a motion history indicating, for each pixel included in the object data map, an indication of time since the object has been detected at the each pixel; and

means for controlling an application based on the motion history.