Methods for capturing images of markers of a person to control interfacing with an application

• US 10,410,359 B2
• Filed: 01/30/2018
• Issued: 09/10/2019
• Est. Priority Date: 02/11/2003
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Marks, Richard L.
• Primary Examiner(s): Earles, Bryan
• Application Number:
  US15/884,326
• Publication Number:
  US 20180158196A1
• Time in Patent Office: 588 Days
• Field of Search: 348168
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

A method for real-time motion capture for a person when interfacing with an application executed by a computer is provided. The method includes capturing a space using a camera interfaced with the computer. The space includes at least part of the person. Identifying a plurality of markers disposed on the person. The plurality of markers is associated with at least a torso, arms and legs of the person. Capturing movement of the person by tracking the plurality of markers from images taken by the camera. The method is configured to interpret one or more movements of the person as an input for the application, while the movement of the person is tracked using the plurality of markers.

Claims

1. A method for real-time motion capture for a person when interfacing with an application executed by a computer, comprising:

capturing a space using a camera interfaced with the computer, the space including at least part of the person;

identifying a plurality of markers disposed on the person, the plurality of markers being associated with at least a torso, arms and legs of the person, wherein each of said plurality of markers is configured to blink together at a first frequency to enable distinguishing said person from another person having a second plurality of markers blinking together at a second frequency;

capturing movement of the person by tracking the plurality of markers from images taken by the camera;

interpreting one or more movements of the person as an input for the application while the movement of the person is tracked using the plurality of markers, wherein the one or more movements for the input are based on movements obtained from storage; and

at least one or more of said plurality of markers is associated with a headband worn by a head of the person, the headband is configured to include a device for providing sound to an ear of the person.

2. The method of claim 1, wherein the one or more movements include changes in positions of said arms of the person.

3. The method of claim 1, wherein the one or more movements include changes in positions of said legs of the person.

4. The method of claim 1, wherein the one or more movements include changes in positions of said arms, legs and torso of the person.

5. The method of claim 1, wherein the plurality of markers are disposed on clothing of the person.

6. The method of claim 1, wherein the plurality of markers are disposed on a body suit worn by the user, the body suit being a shirt or pants and said shirt.

7. The method of claim 1, further comprising,

determining depth of one or more of the plurality of markers, the depth assisting in tracking the person and interpreting the one or more movements of the person as the input.

8. The method of claim 1, wherein wrists of the arms of the parson are associated with markers.

9. Computer readable media, being non-transitory, having program instructions for real-time motion capture for a person when interfacing with an application executed by a computer, the computer readable media comprising:

program instructions for capturing a space using a camera interfaced with the computer, the space including at least part of the person;

program instructions for identifying a plurality of markers disposed on the person, the plurality of markers being associated with at least a torso, arms and legs of the person, and each of said plurality of markers being of a retroreflective material;

program instructions for capturing movement of the person by tracking the plurality of markers from images taken by the camera, wherein a light is directed toward the space that includes said plurality of markers and is configured to blink to differentiate said plurality of markers from lights in the space not associated with the person, and wherein at least two of said plurality of markers is configured to reflect a different color to differentiate said at least two of said plurality of markers; and

program instructions for interpreting one or more movements of the person as an input for the application while the movement of the person is tracked using the plurality of markers, wherein the one or more movements for the input are based on movements obtained from storage, wherein at least one or more of said plurality of markers is associated with a headband worn by a head of the person, the headband is configured to include a device for providing sound to an ear of the person.

10. The computer readable media of claim 9, wherein the one or more movements include changes in positions of said arms of the person.

11. The computer readable media of claim 9, wherein the one or more movements include changes in positions of said legs of the person.

12. The computer readable media of claim 9, wherein the one or more movements include changes in positions of said arms, legs and torso of the person.

13. The computer readable media of claim 9, wherein the plurality of markers are disposed on clothing of the person.

14. The computer readable media of claim 9, wherein the plurality of markers are disposed on a body suit worn by the person, the body suit being a shirt or pants and said shirt.

15. The computer readable media of claim 9, further comprising,

determining depth of one or more of the plurality of markers, the depth assisting in tracking the person and interpreting the one or more movements of the person as the input.

16. The computer readable media of claim 9, wherein wrists of the arms of the parson are associated with markers.

17. A method for real-time motion capture for a person when interfacing with an application executed by a computer, comprising:

capturing a space using a camera interfaced with the computer, the space including at least part of the person;

identifying a plurality of markers disposed on a headband worn by the person, and a device for providing sound related to the application to an ear of the user;

capturing movement of the person by tracking the plurality of markers from images taken by the camera; and

interpreting one or more movements of the person as an input for the application while the movement of the person is tracked using the plurality of markers.