Methods and systems for enabling direction detection when interfacing with a computer program

• US 20070298882A1
• Filed: 12/12/2005
• Published: 12/27/2007
• Est. Priority Date: 09/15/2003
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Marks, Richard, Deshpande, Hrishikesh
• Granted Patent: US 7,646,372 B2
• Time in Patent Office: Days
• US Class Current: 463/36
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/219 for aiming at specific areas on the display, e.g. light-guns
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method for detecting direction when interfacing with a computer program is provided. The method includes capturing an image presented in front of an image capture device. The image capture device has a capture location in a coordinate space. When a person is captured in the image, the method includes identifying a human head in the image and assigning the human head a head location in the coordinate space. The method also includes identifying an object held by the person in the image and assigning the object an object location in coordinate space. The method further includes identifying a relative position in coordinate space between the head location and the object location when viewed from the capture location. The relative position defines a pointing direction of the object when viewed by the image capture device. The method may be practiced on a computer system, such as one used in the gaming field.

Claims

1. A method for detecting direction when interfacing with a computer program, comprising:

(a) capturing an image presented in front of an image capture device, the image capture device having a capture location in a coordinate space;

when a person is captured in the image,

(b) identifying a human head in the image and assigning the human head a head location in the coordinate space;

(c) identifying an object held by the person in the image and assigning the object an object location in coordinate space; and

(d) identifying a relative position in coordinate space between the head location and the object location when viewed from the capture location, wherein the relative position defines a pointing direction of the object when viewed by the image capture device.

2. A method for detecting direction when interfacing with a computer program as recited in claim 1, wherein the relative position is identified by computing an azimuth angle and an altitude angle between the head location and the object location in relation to the capture location.

3. A method for detecting direction when interfacing with a computer program as recited in claim 1, wherein the capture location is at a proximate location of a display screen and the display screen is capable of rendering interactive graphics.

4. A method for detecting direction when interfacing with a computer program as recited in claim 3, wherein the pointing direction is toward the display screen.

5. A method for detecting direction when interfacing with a computer program as recited in claim 3, further comprising:

repeating (a)-(d) continually to update the pointing direction; and

displaying the continually updated pointing direction on the display screen.

6. A method for detecting direction when interfacing with a computer program as recited in claim 5, further comprising:

enabling selection of particular interactive graphics using the displayed pointing direction.

7. A method for detecting direction when interfacing with a computer program as recited in claim 6, wherein the selection is in response to a detected trigger event.

8. A method for detecting direction when interfacing with a computer program as recited in claim 7, wherein the detected trigger event is identified in the image, the identification comprising,

identifying a first characteristic of the object held by the person at a first point in time;

and

identifying a second characteristic of the object held by the person at a second point in time,

wherein the trigger event is activated when a degree of difference is determined to have existed between first characteristic and the second characteristic when examined between the first point in time and the second point in time.

9. A method for detecting direction when interfacing with a computer program as recited in claim 8, wherein the trigger even being activated is indicative of interactivity with the interactive graphics.

10. A method for detecting direction when interfacing with a computer program as recited in claim 9, wherein the interactivity can include one or more of selection of a graphic, shooting of a graphic, touching a graphic, moving of a graphic, activation of a graphic, triggering of a graphic and acting upon or with a graphic.

11. A method for detecting direction when interfacing with a computer program as recited in claim 1, wherein identifying the human head is processed using template matching in combination with face detection code.

12. A method for detecting direction when interfacing with a computer program as recited in claim 1, wherein identifying the object held by the person is facilitated by color tracking of a portion of the object.

13. A method for detecting direction when interfacing with a computer program as recited in claim 12, wherein color tracking includes one or a combination of identifying differences in colors and identifying on/off states of colors.

14. A method for detecting direction when interfacing with a computer program as recited in claim 5, wherein identifying the object held by the person is facilitated by identification of changes in positions of the object when repeating (a)-(d).

15. A method for detecting direction when interfacing with a computer program as recited in claim, wherein the computer program is a video game.

16. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program, comprising:

(a) capturing an image presented in front of an image capture device, the image capture device having a capture location in a coordinate space that is proximate to the display screen;

when a person is captured in the image,

(b) identifying a first body part of the person in the image and assigning the first body part a first location in the coordinate space;

(c) identifying a second body part of the person in the image and assigning the second body part a second location in coordinate space; and

(d) identifying a relative position in coordinate space between the first location and the second location when viewed from the capture location, wherein the relative position defines a pointing direction of the second body part when viewed by the image capture device at the capture location that is proximate to the display screen.

17. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 16, wherein the first body part is a human head and the second body part is a human hand.

18. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 16, wherein (a)-(d) is repeated continually during execution of the computer program, and

examining a shape of the human hand during the repeating of (a)-(d) to determine particular shape changes.

19. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 16, wherein particular shape changes trigger interactivity with interactive graphics of the computer program.

20. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 19, wherein the interactivity can include one or more of selection of a graphic, shooting of a graphic, touching a graphic, moving of a graphic, activation of a graphic, triggering of a graphic and acting upon or with a graphic.

21. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 16, wherein the second body part is identified by way of an object held by the human hand.

22. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 16, wherein the object includes color.

23. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 22, wherein the color is capable of switching between states to trigger interactivity with interactive graphics of the computer program.

24. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 23, wherein additional colors are present on the object, the colors capable of being switched to trigger interactivity with interactive graphics of the computer program.

25. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 22, wherein the color can switch from on/off states to trigger interactivity with interactive graphics of the computer program

26. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 16, wherein the computer program is a video game.

27. A method for detecting pointing direction of an object directed toward a display screen that can render graphics of a computer program as recited in claim 16, wherein the relative position is identified by computing an azimuth angle and an altitude angle between the head location and the object location in relation to the capture location.