METHOD AND APPARATUS FOR TRACKING THREE-DIMENSIONAL MOVEMENTS OF AN OBJECT USING A DEPTH SENSING CAMERA

• US 20090122146A1
• Filed: 10/30/2008
• Published: 05/14/2009
• Est. Priority Date: 07/27/2002
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Zalewski, Gary M., Mao, Xiadong, Marks, Richard
• Granted Patent: US 8,570,378 B2
• Time in Patent Office: Days
• US Class Current: 348/169
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/248 involving reference images or patches

Abstract

A controller (110) includes a body (111), one or more buttons disposed on the body, and one or more detectable elements (e.g. 122, 124, 126, and/or 128) disposed on the body that are arranged in a pattern. A tracking device for use in obtaining information for controlling an execution of a game program by a processor for enabling an interactive game to be played by a user includes a body (111) having a section to be oriented towards a screen when a progress of a game provided via execution of the game apparatus is displayed upon the screen, and at least one photonically detectable (“PD”) element (e.g. 122, 124, 126, and/or 128) assembled with the body, a position of the photonically detectable element within an image being recordable by an image capture device (112) when the section is oriented at least partly towards the screen, wherein positions of the PD element at different points in time are quantifiable to quantify movement of the body in space.

Claims

1. A method for use in obtaining input for an application, comprising:

establishing communications with an image capture device that is capable of providing output data that includes depth measurements;

using the image capture device to capture images of an object as the object moves in a three-dimensional manner;

receiving the output data provided by the image capture device as the image capture device captures images of the object; and

analyzing the output data provided by the image capture device to detect and track the three-dimensional movements of the object.

2. The method of claim 1, wherein the image capture device comprises a three-dimensional camera.

3. The method of claim 1, wherein the object comprises one or more light sources.

4. The method of claim 3, wherein the one or more light sources are arranged in an arcuate pattern.

5. The method of claim 3, wherein the one or more light sources are arranged in a geometric shape.

6. The method of claim 1, wherein the object comprises a controller.

7. The method of claim 1, further comprising:

generating input for the application based on the tracked three-dimensional movements of the object.

8. The method of claim 1, wherein the analyzing the output data provided by the image capture device comprises:

determining position information for the object based on the output data provided by the image capture device.

9. The method of claim 1, wherein the analyzing the output data provided by the image capture device comprises:

determining orientation information for the object based on the output data provided by the image capture device.

10. The method of claim 1, wherein the analyzing the output data provided by the image capture device comprises:

determining acceleration information for the object based on the output data provided by the image capture device.

11. The method of claim 1, wherein the analyzing the output data provided by the image capture device comprises:

determining velocity information for the object based on the output data provided by the image capture device.

12. The method of claim 1, wherein the analyzing the output data provided by the image capture device comprises:

determining at least one of a tilt, pitch, yaw, and roll for the object based on the output data provided by the image capture device.

13. A computer program product comprising a medium for embodying a computer program for input to a computer and a computer program embodied in the medium for causing the computer to perform steps comprising:

establishing communications with an image capture device that is capable of providing output data that includes depth measurements;

using the image capture device to capture images of an object as the object moves in a three-dimensional manner;

receiving the output data provided by the image capture device as the image capture device captures images of the object; and

analyzing the output data provided by the image capture device to detect and track the three-dimensional movements of the object.

14. The computer program product of claim 13, wherein the image capture device comprises a three-dimensional camera.

15. The computer program product of claim 13, wherein the object comprises one or more light sources.

16. The computer program product of claim 15, wherein the one or more light sources are arranged in an arcuate pattern.

17. The computer program product of claim 15, wherein the one or more light sources are arranged in a geometric shape.

18. The computer program product of claim 13, wherein the object comprises a controller.

19. The computer program product of claim 13, wherein the computer program further causes the computer to perform a step comprising:

generating input for an application based on the tracked three-dimensional movements of the object.

20. The computer program product of claim 13, wherein the analyzing the output data provided by the image capture device comprises:

determining position information for the object based on the output data provided by the image capture device.

21. The computer program product of claim 13, wherein the analyzing the output data provided by the image capture device comprises:

determining orientation information for the object based on the output data provided by the image capture device.

22. The computer program product of claim 13, wherein the analyzing the output data provided by the image capture device comprises:

determining acceleration information for the object based on the output data provided by the image capture device.

23. The computer program product of claim 13, wherein the analyzing the output data provided by the image capture device comprises:

determining velocity information for the object based on the output data provided by the image capture device.

24. The computer program product of claim 13, wherein the analyzing the output data provided by the image capture device comprises:

determining at least one of a tilt, pitch, yaw, and roll for the object based on the output data provided by the image capture device.

25. An apparatus, comprising:

an image capture device that is capable of providing output data that includes depth measurements; and

a processing system that is communicatively coupled to the image capture device, wherein the processing system is configured to execute steps comprising using the image capture device to capture images of an object as the object moves in a three-dimensional manner, receiving the output data provided by the image capture device as the image capture device captures images of the object, and analyzing the output data provided by the image capture device to detect and track the three-dimensional movements of the object.

26. The apparatus of claim 25, wherein the image capture device comprises a three-dimensional camera.

27. The apparatus of claim 25, wherein the object comprises one or more light sources.

28. The apparatus of claim 27, wherein the one or more light sources are arranged in an arcuate pattern.

29. The apparatus of claim 27, wherein the one or more light sources are arranged in a geometric shape.

30. The apparatus of claim 25, wherein the object comprises a controller.

31. The apparatus of claim 25, wherein the processing system is further configured to execute a step comprising:

generating input for an application based on the tracked three-dimensional movements of the object.

32. The apparatus of claim 25, wherein the analyzing the output data provided by the image capture device comprises:

determining position information for the object based on the output data provided by the image capture device.

33. The apparatus of claim 25, wherein the analyzing the output data provided by the image capture device comprises:

determining orientation information for the object based on the output data provided by the image capture device.

34. The apparatus of claim 25, wherein the analyzing the output data provided by the image capture device comprises:

determining acceleration information for the object based on the output data provided by the image capture device.

35. The apparatus of claim 25, wherein the analyzing the output data provided by the image capture device comprises:

determining velocity information for the object based on the output data provided by the image capture device.

36. The apparatus of claim 25, wherein the analyzing the output data provided by the image capture device comprises:

determining at least one of a tilt, pitch, yaw, and roll for the object based on the output data provided by the image capture device.