Motion-based tracking

• US 8,717,288 B2
• Filed: 02/28/2012
• Issued: 05/06/2014
• Est. Priority Date: 01/21/2005
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hildreth, Evan
• Primary Examiner(s): CUTLER, ALBERT H
• Application Number:
  US13/406,971
• Publication Number:
  US 20120154272A1
• Time in Patent Office: 798 Days
• Field of Search: 463/2, 463/3, 463/4, 463/5, 463/34
• US Class Current: 345/156
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 2300/1018 Calibration; Key and button assignment
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 7/215 Motion-based segmentation

Abstract

Techniques are disclosed for determining a user'"'"'s motion in relation to displayed images. According to one general aspect, a first captured image is accessed. The first captured image includes (1) a first displayed image produced at a first point in time, and (2) a user. A second captured image is accessed. The second captured image includes (1) a second displayed image produced at a second point in time, and (2) the user. First information indicating motion associated with one or more objects in the first and second displayed images is accessed. Second information indicating both motion of the user and the motion associated with the one or more objects in the first and second displayed images is determined.

Claims

1. A method comprising:

accessing a first captured image of a first displayed image produced at a first point in time, and a user;

accessing a second captured image of a second displayed image produced at a second point in time, and the user;

accessing first information indicating motion of one or more objects in the first and second displayed images;

determining second information indicating both motion of the user and the motion of the one or more objects in the first and second displayed images; and

determining the motion of the user by isolating the motion of the user from the motion of the one or more objects in the first and second displayed images based on comparing the first information and the second information.

2. The method of claim 1 wherein the first information comprises a motion map describing motion of the one or more objects in the first and second displayed images.

3. The method of claim 1 wherein the second information comprises a motion map describing motion of both the user and the one or more objects in the first and second displayed images.

4. The method of claim 1 further comprising determining an input for a set of instructions based on the determined motion of the user.

5. The method of claim 1 wherein determining the second information includes comparing the first and second captured images.

6. The method of claim 5, wherein comparing the first and second captured images comprises performing an optical flow operation on the first captured image and the second captured image.

7. The method of claim 1 further comprising performing a geometric transform operation on the second information.

8. The method of claim 1 wherein accessing the first information comprises:

accessing the first and second displayed images; and

generating the first information by performing an absolute difference operation on the first displayed image and the second displayed image.

9. The method of claim 1 further comprising performing a latency compensation operation on the first information indicating the motion of the one or more objects in the first and second displayed images.

10. An apparatus comprising:

a captured image motion detection module configured to;

access a first captured image that includes a first displayed image produced at a first point in time, and a user;

access a second captured image that includes a second displayed image produced at a second point in time, and the user; and

generate first information representing motion between the first and second captured images by comparing the first captured image and the second captured image; and

a comparison module configured to determine the motion of the user by comparing the first information with second information representing motion of one or more objects in the first and second displayed images and isolating the motion of the user from the motion of the one or more objects in the first and second displayed images based on the comparing.

11. The apparatus of claim 10 wherein the captured image motion detection module is further configured to:

access the first displayed image;

access the second displayed image; and

generate the second information by comparing the first displayed image to the second displayed image.

12. The apparatus of claim 11 further comprising:

a camera configured to;

capture the first captured image and provide the first captured image to the captured image motion detection module, and

capture the second captured image and provide the second captured image to the captured image motion detection module;

a display device configured to display at least the first displayed image and the second displayed image; and

a computing device configured to execute a set of instructions to produce the first and second displayed images.

13. The apparatus of claim 10 wherein the captured image motion detection module is configured to generate the second information comprising a motion map that describes motion of the one or more objects in the first and second displayed images.

14. The apparatus of claim 10 wherein the comparison module is configured to compare the first information comprising a motion map that describes motion of both the user and the one or more objects in the first and second displayed images.

15. The apparatus of claim 10 wherein the captured image motion detection module is further configured to determine an input for a set of instructions based on the determined motion of the user.

16. The apparatus of claim 10 wherein the captured image motion detection module is configured to determine the first information by comparing the first and second captured images.

17. The apparatus of claim 16, wherein the captured image motion detection module is configured to compare the first and second captured images by performing an optical flow operation on the first captured image and the second captured image.

18. The apparatus of claim 10 wherein the comparison module is further configured to perform a geometric transform operation on the first information.

19. The apparatus of claim 10 wherein the comparison module is configured to:

access the first and second displayed images; and

generate the second information by performing an absolute difference operation on the first displayed image and the second displayed image.

20. The apparatus of claim 10 wherein the captured image motion detection module is further configured to perform a latency compensation operation on the second information indicating the motion of the one or more objects in the first and second displayed images.

21. A system comprising:

means for accessing a first captured image that includes a first displayed image produced at a first point in time, and a user;

means for accessing a second captured image that includes a second displayed image produced at a second point in time, and the user;

means for generating first information representing motion of one or more objects between the first and second captured images by comparing the first captured image and the second captured image;

means for generating second information representing both motion of the user and the motion of the one or more objects between the first and second captured images by comparing the first captured image and the second captured image; and

means for determining the motion of the user by isolating the motion of the user from the motion of the one or more objects in the first and second displayed images based on comparing the first information and the second information.

22. The system of claim 21 further comprising a motion detection module configured to:

access the first displayed image;

access the second displayed image; and

generate the second information by comparing the first displayed image to the second displayed image.

23. The system of claim 22 further comprising:

means for capturing the first captured image and providing the first captured image to the motion detection module, and

means for capturing the second captured image and providing the second captured image to the motion detection module;

means for displaying at least the first displayed image and the second displayed image; and

means for executing a set of instructions to produce the first and second displayed images.

24. The system of claim 21, wherein the means for accessing the first captured image, means for accessing the second captured image, and means for generating comprise a motion detection module, or wherein the means for determining comprises a comparison module.

25. A non-transitory computer-readable medium including instructions that, when executed by one or more processing devices, cause the one or more processing devices to:

access a first captured image of a first displayed image produced at a first point in time, and a user;

access a second captured image of a second displayed image produced at a second point in time, and the user;

access first information indicating motion of one or more objects in the first and second displayed images;

determine second information indicating both motion of the user and the motion of the one or more objects in the first and second displayed images; and

determine the motion of the user by isolating the motion of the user from the motion of the one or more objects in the first and second displayed images based on comparing the first information and the second information.

26. The non-transitory computer-readable medium of claim 25, wherein the first information comprises a motion map describing motion of the one or more objects in the first and second displayed images.

27. The non-transitory computer-readable medium of claim 25, wherein the second information comprises a motion map describing motion of both the user and the one or more objects in the first and second displayed images.

28. The non-transitory computer-readable medium of claim 25, wherein the instructions for accessing the first information comprise instructions for:

accessing the first and second displayed images; and

generating the first information by performing an absolute difference operation on the first displayed image and the second displayed image.

29. A method comprising:

accessing a first captured image of a first displayed image produced at a first point in time, and a user;

accessing a second captured image of a second displayed image produced at a second point in time, and the user;

accessing first information indicating motion of one or more objects in the first and second displayed images, wherein the first information comprises a motion map describing motion of the one or more objects in the first and second displayed images;

determining second information indicating both motion of the user and the motion of the one or more objects in the first and second displayed images; and

determining the motion of the user by comparing the first information and the second information.

30. A method comprising:

accessing a first captured image of a first displayed image produced at a first point in time, and a user;

accessing a second captured image of a second displayed image produced at a second point in time, and the user;

accessing first information indicating motion of one or more objects in the first and second displayed images;

determining second information indicating both motion of the user and the motion of the one or more objects in the first and second displayed images, wherein the second information comprises a motion map describing motion of both the user and the one or more objects in the first and second displayed images; and

determining the motion of the user by comparing the first information and the second information.