Methods for Capturing Images of a Control Object and Tracking to Control Interfacing with Video Game Objects

• US 20160027188A1
• Filed: 10/07/2015
• Published: 01/28/2016
• Est. Priority Date: 02/11/2003
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Marks, Richard L.
• Granted Patent: US 9,881,382 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

Methods for real time motion capture for controlling an object in a video game are provided. One method includes defining a model of a control object and identifying a marker on the control object. The method also includes capturing movement associated with the control object with a video capture device. Then, interpreting the movement associated with the control object to change a position of the model based on data captured through the video capture device, wherein the data captured includes the marker. The method includes moving the video game object presented on the display screen in substantial real-time according to the change of position of the model.

Claims

1. A method for real-time motion capture for control of a video game object during game play, comprising:

defining a model of a control object;

identifying a marker on the control object;

capturing movement associated with the control object with a video capture device; and

interpreting the movement associated with the control object to change a position of the model based on data captured through the video capture device, the data captured including the marker; and

moving the video game object presented on the display screen in substantial real-time according to the change of position of the model.

2. The method of claim 1, wherein the control object is a hand of a person, the hand being tracked over time to capture changes, such that changes in the hand being tracked enable manipulations of the video game object, wherein the manipulations include the moving.

3. The method of claim 1, wherein the method operation of capturing movement associated with the control object includes,

capturing movement associated with an object being controlled by the control object.

4. The method of claim 1, further comprising:

continuing to capture movement associated with the control object, interpret the movement associated with the control object to change a position of the model and control movement of the video game object being presented on the display screen according to the change of position of the model.

5. The method of claim 1, wherein the method is executed by a computing device, the computing device is interfaced with the video capture device.

6. The method of claim 1, wherein the control object is a head of a person and the model is a skeleton image of the head.

7. A method for interfacing with a video game, comprising:

detecting a person in a field of view of a video capture device;

identifying skeleton image of the person, the skeleton image identifying one or more limbs of the person, wherein one of the limbs is a hand of the person and one of the limbs is a head of the person;

detecting a marker in a location of the hand of the person, the marker being associated with an object being held in the hand of the person; and

tracking the skeleton image and the detected object being held in the hand of the person by analyzing video frames obtained by the video capture device, during the tracking, identifying motion of the person and of the object being held in the hand of the person, the tracking being used to control and interface with one or more objects of the video game rendered on a display while the video game is executed by a processor.

8. The method for interfacing with a video game as recited in claim 7, wherein the identified motion of the person is compared to a database of motion, and based on the comparisons, at least part of the identified motion of the person is rendered on the display using a motion from the database of motions, the database of motions being predefined.

9. The method for interfacing with a video game as recited in claim 7, wherein the processor is part of a computing device, the computing device being in communication with the video capture device and the display device.

10. The method for interfacing with a video game as recited in claim 7, wherein the skeleton image represents a model of at least part of the person.

11. The method for interfacing with a video game as recited in claim 7, wherein the object includes a light, the light being one of non-visible or visible.

12. The method for interfacing with a video game as recited in claim 7, further comprising:

determining depth parameters of at least part of the skeleton image, the depth parameters used to fill in the skeleton image and define a three dimensional image of at least part of the person.

13. A method for interfacing with a video game, comprising:

detecting a person in a field of view of a video capture device;

identifying skeleton image of the person, the skeleton image identifying one or more limbs of the person, wherein one of the limbs is a hand of the person and one of the limbs is a head of the person;

detecting a marker in a location of the hand of the person, the marker being associated with an object being held in the hand of the person; and

tracking the skeleton image and the detected object being held in the hand of the person by analyzing video frames obtained by the video capture device, during the tracking, identifying motion of the person and of the object being held in the hand of the person, the tracking being used to control and interface with one or more objects of the video game rendered on a display while the video game is executed by a processor,

wherein before rendering on the display, at least part of the identified motion of the person is replaced with a motion from a database of motions for rendering on the display, the database of motions being predefined.

14. The method for interfacing with a video game as recited in claim 13, wherein the processor is part of a computing device, the computing device being in communication with the video capture device and the display device.

15. The method for interfacing with a video game as recited in claim 13, wherein the skeleton image represents a model of at least part of the person.

16. The method for interfacing with a video game as recited in claim 13, wherein the object includes a light, the light being one of non-visible or visible.

17. The method for interfacing with a video game as recited in claim 13, further comprising:

determining depth parameters of at least part of the skeleton image, the depth parameters used to fill in the skeleton image and define a three dimensional image of at least part of the person.