METHODS FOR CAPTURING DEPTH DATA OF A SCENE AND APPLYING COMPUTER ACTIONS

• US 20090215533A1
• Filed: 02/24/2009
• Published: 08/27/2009
• Est. Priority Date: 02/27/2008
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Haigh, Mike, Zalewski, Gary
• Granted Patent: US 8,840,470 B2
• Time in Patent Office: Days
• US Class Current: 463/32
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/201 Playing authorisation given at platform level
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4751 for defining user accounts, e.g. accounts for children
• CPC Class Codes: H04N 21/4755 for defining user preferences, e.g. favourite actors or genre retrieval personalisation and generation of user profiles for the retrieval of video data G06F16/739; user profiles in network data switching protocols H04L67/306
• CPC Class Codes: H04N 21/4781 Games

Abstract

A computer-implemented method is provided to automatically apply predefined privileges for identified and tracked users in a space having one or more media sources. The method includes an operation to define and save to memory, a user profile. The user profile may include data that identifies and tracks a user with a depth-sensing camera. In another operation privileges that define levels of access to particular media for the user profile are defined and saved. The method also includes an operation to capture image and depth data from the depth-sensing camera of a scene within the space. In yet another operation, the user is tracked and identified within the scene from the image and depth data. In still another operation the defined privileges are automatically applied to one or more media sources, so that the user is granted access to selected content from the one or more media sources when identified and tracked within the scene.

Claims

9. A computer-implemented method for identifying and tracking real-world objects to automatically apply predefined computer-generated effects to virtual world representations of the real world objects, comprising:

(a) defining and saving to a memory, a user profile, the user profile including data for identifying and tracking the user with a depth sensing camera;

(b) defining and saving to the memory, animations to be integrated into a virtual world scene associated with the user profile;

(c) capturing a scene using the depth sensing camera;

(d) identifying the user within the scene using the depth sensing camera;

(e) automatically applying the defined animations based on the identified and tracked user, so that a display screen shows the integrated animations.

10. The method of claim 9, wherein capturing the scene includes filtering out stationary objects so as to focus on moving objects, and focusing on moving objects includes,

focusing on a moving object is in the scene;

analyzing features of the moving object using the image and depth data; and

determining if the moving object correspond to the user.

11. The method of claim 10, wherein the user is one of a human or a pet.

12. The method of claim 11, wherein tracking the user further includes,

displaying a history path of the user, the history path of the user identifying the movement over time and the animations associated with the movements.

13. The method of claim 12, further comprising:

saving the history path to storage; and

enabling replay of the history path.

14. The method of claim 9, wherein the animations are applied to contours of objects found in the scene, based on the captured depth data.

15. The method of claim 9, further comprising:

pre-selecting the animations by a user.

16. The method of claim 9, wherein multimedia content is presented on a display screen along with the animations, based on the identified user.

17. The method of claim 9, wherein capturing a scene uses point tracking to define stationary objects within the scene.

18. The method of claim 9, wherein tracking the user includes identification of relative position, direction and velocity.

19. The method of claim 9, wherein the depth data is used to identify points on stationary objects, the points being used to draw outlines of objects found in the scene.

20. A computer implemented method for identifying and tracking a real-world users within a real-world space, comprising:

(a) defining a user profile, the user profile including image and depth data related to physical characteristics of the real-world user, the image and depth data captured by a depth-sensing camera;

(b) capturing image and depth data for a scene using the depth-sensing camera;

(c) identifying moving objects within the scene;

(d) locking onto a head within the moving objects using image and depth data; and

(e) analyzing the image and depth data for the head in real-time, the analysis including comparing image and depth data for the head to user profile image and depth data related to physical characteristic, wherein a user is identified when image and depth data within the user profile substantially matches image and depth data for the head.

1. A computer-implemented method to automatically apply predefined privileges for identified and tracked users in a space having one or more media sources, comprising:

(a) defining and saving to a memory, a user profile, the user profile including data for identifying and tracking a user with a depth sensing camera;

(b) defining and saving to the memory, privileges that define levels of access to particular media for the user profile;

(c) capturing a scene within the space using the depth sensing camera to produce image and depth data;

(d) identifying the user within the scene using the image and depth data;

(e) automatically applying the defined privileges to one or more media sources, so that the user is granted access to selected content from the one or more media sources when identified and tracked within the scene.

2. The method of claim 1, wherein capturing the scene includes filtering out stationary objects so as to focus on moving objects, and focusing on moving objects includes,

determining if a head object is found in the scene;

analyzing face features of the head using the image and depth data;

determining if the identified face features correspond to the user; and

applying the privileges that define levels of access to particular media for the user profile.

3. The method of claim 2, wherein tracking includes tracing movement of the user in the scene, the tracing assists in determining if the identified face features correspond to the user.

4. The method of claim 2, wherein the depth data is used to identify relative three-dimensional positioning of physical characteristics of the user.

5. The method of claim 2, wherein the depth data of the stationary objects is subtracted out from the scene when determining if the head object is found in the scene.

6. The method of claim 1, further comprising:

determining if more than one user is found in the scene;

if more than one user is found, applying privileges associated with a least restrictive user profile.

7. The method of claim 1, wherein the one or more media sources include devices that provide multimedia content.

8. The method of claim 7, wherein the multimedia content is presented in part on a display screen that is present in the space.

21. The method of claim 20, wherein defining a user profile includes:

(a) initiating a scan using a depth-sensing camera;

(b) focusing the scan to a particular portion of the human body;

(c) collecting image and depth data for the particular portion of the human body;

(d) processing the collected image and depth data go generate a three-dimensional model of the particular portion of the human body; and

(e) saving the three-dimensional model to a memory, the three-dimensional model also being associated with a user profile.

22. The method of claim 20, further comprising:

applying user animations associated with the user profile when the user is identified.

21-1. The method of claim 20, further comprising:

applying user permissions associated with the user profile when the user is identified.