Methods for capturing depth data of a scene and applying computer actions

• US 8,840,470 B2
• Filed: 02/24/2009
• Issued: 09/23/2014
• Est. Priority Date: 02/27/2008
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.), Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Zalewski, Gary, Haigh, Mike
• Primary Examiner(s): Matthews, Colleen
• Assistant Examiner(s): HARRISTON, WILLIAM A
• Application Number:
  US12/392,044
• Publication Number:
  US 20090215533A1
• Time in Patent Office: 2,037 Days
• Field of Search: 463/32
• US Class Current: 463/32
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/201 Playing authorisation given at platform level
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4751 for defining user accounts, e.g. accounts for children
• CPC Class Codes: H04N 21/4755 for defining user preferences, e.g. favourite actors or genre retrieval personalisation and generation of user profiles for the retrieval of video data G06F16/739; user profiles in network data switching protocols H04L67/306
• CPC Class Codes: H04N 21/4781 Games

Abstract

A computer-implemented method is provided to automatically apply predefined privileges for identified and tracked users in a space having one or more media sources. The method includes an operation to define and save to memory, a user profile. The user profile may include data that identifies and tracks a user with a depth-sensing camera. In another operation privileges that define levels of access to particular media for the user profile are defined and saved. The method also includes an operation to capture image and depth data from the depth-sensing camera of a scene within the space. In yet another operation, the user is tracked and identified within the scene from the image and depth data. In still another operation the defined privileges are automatically applied to one or more media sources, so that the user is granted access to selected content from the one or more media sources.

Claims

1. A computer-implemented method, comprising:

(a) defining and saving to a memory, a user profile, the user profile including data for identifying and tracking the user with a depth sensing camera;

(b) defining and saving to the memory, animations to be integrated into a virtual world scene based on the user profile;

(c) capturing a scene using the depth sensing camera;

(d) identifying the user within the scene using the depth sensing camera, the identifying further configured to identify stationary objects in the scene, wherein points located on the stationary objects are used to at least partially outline the identified stationary objects; and

(e) automatically applying the defined animations onto at least one identified stationary object in the scene to be displayed on a screen, such that the defined animations are selected for the identified and tracked user.

2. The method of claim 1, wherein capturing the scene includes filtering out stationary objects so as to focus on moving objects, and focusing on moving objects includes,

focusing on a moving object is in the scene;

analyzing features of the moving object using the image and depth data; and

determining if the moving object correspond to the user.

3. The method of claim 2, wherein the user is one of a human or a pet.

4. The method of claim 3, wherein tracking the user further includes,

displaying a history path of the user, the history path of the user identifying the movement over time and the animations associated with the movements.

5. The method of claim 4, further comprising:

saving the history path to storage; and

enabling replay of the history path.

6. The method of claim 1, wherein the animations are applied to contours of the at least one stationary object found in the scene, based on the captured depth data.

7. The method of claim 1, further comprising:

pre-selecting the animations for the user, and pre-selecting other animations for other users.

8. The method of claim 1, wherein multimedia content is presented on the display screen along with the animations, based on the identified user.

9. A computer-implemented method, comprising:

(a) defining and saving to a memory, a user profile, the user profile including data for identifying and tracking the user with a depth sensing camera;

(b) defining and saving to the memory, animations to be applied into a virtual world scene associated with the user profile;

(c) capturing a scene using the depth sensing camera;

(d) identifying the user within the scene using the depth sensing camera;

(e) automatically applying the defined animations onto objects or stationary objects found in the captured scene using point tracking, the defined animations being pre-defined for the identified and tracked user, so that a display screen shows the applied animations.

10. A computer implemented method, comprising:

(a) defining a user profile, the user profile including image and depth data related to physical characteristics of a real-world user, the image and depth data captured by a depth-sensing camera;

(b) capturing image and depth data for a scene using the depth-sensing camera, wherein point tracking is used to identify stationary objects in the scene, the points being used to draw outlines of stationary objects found in the scene;

(c) identifying moving objects within the scene;

(d) locking the depth-sensing camera onto a human head within the scene;

(e) analyzing the image and depth data for the human head in real-time, the analysis including comparing image and depth data for the human head to user profile image and depth data related to physical characteristic, wherein a user is identified when image and depth data within the user profile substantially matches image and depth data for the head, and identifying animations pre-selected for the user profile when the user is identified; and

(f) applying the identified animations onto selected ones of the stationary objects identified in the scene.

11. The method of claim 10, wherein defining a user profile includes:

i. initiating a scan using a depth-sensing camera;

ii. focusing the scan to a particular portion of the human body;

iii. collecting image and depth data for the particular portion of the human body;

iv. processing the collected image and depth data to generate a three-dimensional model of the particular portion of the human body; and

v. saving the three-dimensional model to a memory, the three-dimensional model also being associated with a user profile.

12. The method of claim 10, further comprising:

applying user permissions associated with the user profile when the user is identified.