Methods for capturing images of a control object and tracking to control interfacing with video game objects

• US 9,881,382 B2
• Filed: 10/07/2015
• Issued: 01/30/2018
• Est. Priority Date: 02/11/2003
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Marks, Richard L.
• Primary Examiner(s): Earles, Bryan
• Application Number:
  US14/877,844
• Publication Number:
  US 20160027188A1
• Time in Patent Office: 846 Days
• Field of Search: 345156
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

Methods for real time motion capture for controlling an object in a video game are provided. One method includes defining a model of a control object and identifying a marker on the control object. The method also includes capturing movement associated with the control object with a video capture device. Then, interpreting the movement associated with the control object to change a position of the model based on data captured through the video capture device, wherein the data captured includes the marker. The method includes moving the video game object presented on the display screen in substantial real-time according to the change of position of the model.

Claims

1. A method for interfacing with a video game, comprising:

detecting a person in a field of view of a video capture device;

identifying skeleton image of the person, the skeleton image identifying one or more limbs of the person, wherein one of the limbs is a hand of the person and one of the limbs is a head of the person;

detecting a marker in a location of the hand of the person, the marker being associated with an object being held in the hand of the person; and

tracking the skeleton image and the detected object being held in the hand of the person by analyzing video frames obtained by the video capture device, during the tracking, identifying motion of the person and of the object being held in the hand of the person, the tracking being used to control and interface with one or more objects of the video game rendered on a display while the video game is executed by a processor.

2. The method for interfacing with a video game as recited in claim 1, wherein the identified motion of the person is compared to a database of motion, and based on the comparisons, at least part of the identified motion of the person is rendered on the display using a motion from the database of motions, the database of motions being predefined.

3. The method for interfacing with a video game as recited in claim 1, wherein the processor is part of a computing device, the computing device being in communication with the video capture device and the display device.

4. The method for interfacing with a video game as recited in claim 1, wherein the skeleton image represents a model of at least part of the person.

5. The method for interfacing with a video game as recited in claim 1, wherein the object includes a light, the light being one of non-visible or visible.

6. The method for interfacing with a video game as recited in claim 1, further comprising:

determining depth parameters of at least part of the skeleton image, the depth parameters used to fill in the skeleton image and define a three dimensional image of at least part of the person.

7. A method for interfacing with a video game, comprising:

detecting a person in a field of view of a video capture device;

identifying skeleton image of the person, the skeleton image identifying one or more limbs of the person, wherein one of the limbs is a hand of the person and one of the limbs is a head of the person;

detecting a marker in a location of the hand of the person, the marker being associated with an object being held in the hand of the person; and

tracking the skeleton image and the detected object being held in the hand of the person by analyzing video frames obtained by the video capture device, during the tracking, identifying motion of the person and of the object being held in the hand of the person, the tracking being used to control and interface with one or more objects of the video game rendered on a display while the video game is executed by a processor,

wherein before rendering on the display, at least part of the identified motion of the person is replaced with a motion from a database of motions for rendering on the display, the database of motions being predefined.

8. The method for interfacing with a video game as recited in claim 7, wherein the processor is part of a computing device, the computing device being in communication with the video capture device and the display device.

9. The method for interfacing with a video game as recited in claim 7, wherein the skeleton image represents a model of at least part of the person.

10. The method for interfacing with a video game as recited in claim 7, wherein the object includes a light, the light being one of non-visible or visible.

11. The method for interfacing with a video game as recited in claim 7, further comprising:

determining depth parameters of at least part of the skeleton image, the depth parameters used to fill in the skeleton image and define a three dimensional image of at least part of the person.