METHODS FOR DIRECTING POINTING DETECTION CONVEYED BY USER WHEN INTERFACING WITH A COMPUTER PROGRAM

• US 20100056277A1
• Filed: 11/10/2009
• Published: 03/04/2010
• Est. Priority Date: 09/15/2003
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Marks, Richard L., Deshpande, Hrishikesh R.
• Granted Patent: US 8,568,230 B2
• Time in Patent Office: Days
• US Class Current: 463/36
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/219 for aiming at specific areas on the display, e.g. light-guns
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device. The computing device being interfaced with an image capture device. The method includes detecting a human head of a person in the image taken with the image capture device and assigning the human head a head location. And, detecting an object held by the person in the image and assigning the object an object location. The method determines a relative position in space between the head location and the object location from the capture location, such that the relative position is used to identify a pointing direction of the object.

Claims

1. A method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device, the computing device being interfaced with an image capture device, comprising:

(a) detecting a human head of a person in the image taken with the image capture device and assigning the human head a head location;

(b) detecting an object held by the person in the image and assigning the object an object location; and

(c) determining a relative position in space between the head location and the object location from the capture location, the relative position used to identify a pointing direction of the object.

2. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 1, further comprising:

capturing a plurality of frames through the image capture device, each of the frames being analyzed to process (a)-(c); and

determining changes in the pointing direction of the object over the plurality of analyzed frames.

3. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 2, wherein the determined changes in the pointing direction are translated to detected motion of the pointing direction over the plurality of frames captured through the image capture device.

4. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 1, wherein the pointing direction is translated as a pointing location on a display screen, the display screen being located at about the location of the image capture device.

5. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 1, wherein the relative position is identified by computing a first angle and a second angle between the head location and the object location in relation to the capture location.

6. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 4, wherein the display screen is configured to render interactive graphics.

7. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 1, further comprising:

enabling interactivity with particular interactive graphics presented on a display screen using a graphic indicator of the pointing direction.

8. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 7, wherein the interactivity includes one or more of selection of a graphic, shooting of a graphic, touching a graphic, moving of a graphic, activation of a graphic, triggering of a graphic and acting upon or with a graphic.

9. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 1, wherein detecting the human head is processed using face detection.

10. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 1, wherein detecting the object is facilitated using one or more of colors, lights, or a combination of identifying differences in colors, lights, shapes, or identifying on/off states.

11. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 1, wherein the computer program is a video game.

12. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 1, wherein the computer program is either local to the image capture device or networked over the internet.

13. A method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device, the computing device being interfaced with an image capture device, comprising:

(a) detecting a human head of a person in the image taken with the image capture device and assigning the human head a head location;

(b) detecting a body part of the person, other than the human, in the image and assigning the body part a body part location; and

(c) determining a relative position in space between the head location and the body part location from the capture location, the relative position used to identify a pointing direction of the body part.

14. The method for detecting direction when interfacing with a computer program executed on a computing device as recited in claim 13, further comprising:

capturing a plurality of frames through the image capture device, each of the frames being analyzed to process (a)-(c); and

determining changes in the pointing direction of the body part over the plurality of analyzed frames.

15. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 13, wherein the determined changes in the pointing direction are translated to detected motion of the pointing direction over the plurality of frames captured through the image capture device.

16. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 13, wherein the pointing direction is translated as a pointing location on a display screen, the display screen being located at about the location of the image capture device.

17. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 13, wherein the relative position is identified by computing a first angle and a second angle between the head location and the body part in relation to the capture location.

18. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 16, wherein the display screen is configured to render interactive graphics, and enabling interactivity with particular interactive graphics presented on a display screen using a graphic indicator of the pointing direction.

19. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 18, wherein the interactivity includes one or more of selection of a graphic, shooting of a graphic, touching a graphic, moving of a graphic, activation of a graphic, movement of a graphic, triggering of a graphic and acting upon or with a graphic.

20. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 13, wherein detecting the human head is processed using face detection and detecting the body part is facilitated by identifying shapes.

21. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 13, wherein the computer program is a video game or an interactive communication program.

22. The method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device as recited in claim 13, wherein the computer program is either local to the image capture device or networked over the internet.