METHODS AND SYSTEMS FOR ENABLING DEPTH AND DIRECTION DETECTION WHEN INTERFACING WITH A COMPUTER PROGRAM

• US 20120319950A1
• Filed: 08/27/2012
• Published: 12/20/2012
• Est. Priority Date: 09/15/2003
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Marks, Richard L., Deshpande, Hrishikesh R.
• Granted Patent: US 8,758,132 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/5372 for tagging characters, objects or locations in the game scene, e.g. displaying a circle under the character controlled by the player
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/306 for displaying a marker associated to an object or location in the game field
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

One or more images can be captured with a depth camera having a capture location in a coordinate space. First and second objects in the one or more images can be identified and assigned corresponding first and second object locations in the coordinate space. A relative position can be identified in the coordinate space between the first object location and the second object location when viewed from the capture location by computing an azimuth angle and an altitude angle between the first object location and the object location in relation to the capture location. The relative position includes a dimension of depth with respect to the coordinate space. The dimension of depth is determined from analysis of the one or more images. A state of a computer program is changed based on the relative position.

Claims

1. A computer-implemented method for detecting depth and direction when interfacing with a computer program, comprising:

(a) capturing one or more images with one or more depth camera that utilizes controlled lighting to obtain distance information, wherein the one or more depth camera has a capture location in a coordinate space;

(b) identifying a first object in the image and assigning the first object a first object location in the coordinate space;

(c) identifying a second object in the image and assigning the second object a second object location in coordinate space;

(d) identifying a relative position in coordinate space between the first object location and the second object location when viewed from the capture location, wherein identifying the relative position includes computing an azimuth angle and an altitude angle between the first object location and the second object location in relation to the capture location, wherein the relative position includes a dimension of depth with respect to the coordinate space, wherein the dimension of depth is determined from analysis of the one or more images; and

(e) changing a state of the program based on the relative position.

2. The method of claim 1 wherein the relative position defines a pointing direction of the first or second object.

3. The method of claim 1 wherein (a) includes capturing two or more images with first and second image capture devices and wherein the dimension of depth is determined by taking first and second images with first and second image capture devices located at spaced-apart positions and measuring distances of an object in each image relative to a reference in each of the first and second images.

4. The method of claim 3, further comprising synchronizing the first and second image capture devices using a strobe signal that is visible to each of the tow or more image capture devices.

5. The method of claim 1, wherein the capture location is at a proximate location of a display screen and the display screen is capable of rendering interactive graphics.

6. The method of claim 1, wherein the relative position in coordinate space is determined from relative times of arrival of a sound signal from the first or second at object at two or more different microphones

7. The method of claim 1 wherein the relative position defines a pointing direction of the second body part when viewed by the image capture device at the capture location that is proximate to the display screen.

8. The method of claim 1, wherein the computer program is a video game.

9. The method of claim 1, wherein the one or more depth camera utilizes controlled infrared lighting to obtain distance information.

10. A system for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program, comprising:

a processor;

a memory coupled to the processor, the memory having embodied therein one or more computer executable instructions configured to implement, upon execution, a method for detecting depth and direction when interfacing with a computer program, the method comprising;

(a) capturing one or more images with one or more depth camera that utilizes controlled lighting to obtain distance information, wherein the one or more depth camera has a capture location in a coordinate space;

(b) identifying a first object in the image and assigning the first object a first object location in the coordinate space;

(c) identifying a second object in the image and assigning the second object a second object location in coordinate space;

(d) identifying a relative position in coordinate space between the first object location and the second object location when viewed from the capture location, wherein identifying the relative position includes computing an azimuth angle and an altitude angle between the first object location and the second object location in relation to the capture location, wherein the relative position includes a dimension of depth with respect to the coordinate space, wherein the dimension of depth is determined from analysis of the one or more images; and

(e) changing a state of the program based on the relative position.

11. The system of claim 10, further comprising the one or more depth camera that utilizes controlled lighting to obtain distance information, wherein the one or more depth camera is coupled to the processor.

12. The system of claim 10, wherein the one or more depth camera utilizes controlled infrared lighting to obtain distance information

13. A non-transitory computer-readable storage medium having embodied therein one or more computer executable instructions configured to implement, upon execution, a method for detecting depth and direction when interfacing with a computer program, the method comprising:

(a) capturing one or more images with one or more depth camera that utilizes controlled lighting to obtain distance information, wherein the one or more depth camera has a capture location in a coordinate space;

(b) identifying a first object in the image and assigning the first object a first object location in the coordinate space;

(c) identifying a second object in the image and assigning the second object a second object location in coordinate space;

(d) identifying a relative position in coordinate space between the first object location and the second object location when viewed from the capture location, wherein identifying the relative position includes computing an azimuth angle and an altitude angle between the first object location and the second object location in relation to the capture location, wherein the relative position includes a dimension of depth with respect to the coordinate space, wherein the dimension of depth is determined from analysis of the one or more images; and

(e) changing a state of the program based on the relative position.