VISUAL BASED IDENTITIY TRACKING

• US 20120165097A1
• Filed: 03/02/2012
• Published: 06/28/2012
• Est. Priority Date: 01/29/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Leyvand, Tommer, Dernis, Mitchell Stephen, Li, Jinyu, Sun, Jian, Meekhof, Casey Leon, Keosababian, Timothy Milton, Wei, Yichen
• Granted Patent: US 8,926,431 B2
• Time in Patent Office: Days
• US Class Current: 463/36
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/40 Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 9/24 Electric games; Games using electronic circuits not otherwise provided for video games A63F13/00; computerized gaming systems G07F17/32
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

A video game system (or other data processing system) can visually identify a person entering a field of view of the system and determine whether the person has been previously interacting with the system. In one embodiment, the system establishes thresholds, enrolls players, performs the video game (or other application) including interacting with a subset of the players based on the enrolling, determines that a person has become detectable in the field of view of the system, automatically determines whether the person is one of the enrolled players, maps the person to an enrolled player and interacts with the person based on the mapping if it is determined that the person is one of the enrolled players, and assigns a new identification to the person and interacts with the person based on the new identification if it is determined that the person is not one of the enrolled players.

Claims

1. A method of configuring interaction between a set of enrolled players and a computer based application, comprising:

performing the computer based application including interacting with a subset of the set of enrolled players;

determining that a person has become detectable in the play space, the person is not one of the subset of the set of enrolled players currently interacting with the computer based application;

automatically identifying the person as a specific enrolled player of the set of enrolled players; and

mapping the person to the specific enrolled player and interacting with the person based on the mapping.

2. The method of claim 1, wherein:

the automatically identifying the person as a specific enrolled player of the set includes visually identifying the person as the specific enrolled player.

3. The method of claim 1, further comprising:

sensing visual data of the person; and

sensing depth data of the person, the automatically identifying the person as a specific enrolled player is based on the visual data and the depth data.

4. The method of claim 1, further comprising:

establishing thresholds, the thresholds are used when automatically identifying the person as the specific enrolled player; and

dynamically updating the thresholds while and based on performing the computer based application.

5. The method of claim 1, wherein the automatically identifying comprises:

receiving visual data and depth data from one or more sensors;

creating a set of visual signatures indicative of appearance of the person based on the visual data and depth data; and

determining whether the visual signatures correspond to stored signatures for any of the set of enrolled players.

6. The method of claim 5, wherein:

the creating visual signatures includes creating a signature for an appearance of the person'"'"'s face, creating a signature for an appearance of the person'"'"'s clothing and creating a signature for the person'"'"'s height.

7. The method of claim 6, wherein the determining whether the visual signatures correspond to stored signatures for any of the set of enrolled players comprises:

(a) determining whether the signature for the appearance of the person'"'"'s face corresponds to stored signatures for faces of any of the set of enrolled players;

(b) determining whether the signature for the appearance of the person'"'"'s clothing corresponds to stored signatures for clothing of any of the set of enrolled players;

(c) determining whether the signature for the person'"'"'s height corresponds to stored signatures for height of any of the set of enrolled players; and

making an aggregate decision as to whether the person is the specific enrolled player based on steps (a), (b) and (c).

8. The method of claim 1, wherein the automatically identifying comprises:

receiving a visual image and a depth image from one or more sensors;

creating a of one or more visual signatures indicative of clothing of the person based on the visual image and depth image; and

determining whether the one or more visual signatures correspond to stored signatures for any of the set of enrolled players.

9. The method of claim 1, further comprising:

the interacting with the person based on the mapping includes controlling an avatar in the video game based on motion of the person and outputting display signals indicating the video game with the avatar.

10. The method of claim 1, wherein:

the computer based application is a video game;

determining that insufficient data exists for identifying the person;

requesting the person to perform a specific act as part of the video game in response to determining that insufficient data exists for identifying the person; and

obtaining sufficient data response to the specific act.

11. A video game apparatus, comprising:

an interface to a monitor;

a sensor system; and

one or more processors in communication with the interface and the sensor system;

the one or more processors establish thresholds, enroll players in a video game, perform the video game including interacting with a subset of the players based on the enrolling, determine that a person has become detectable in the play space of the video game, automatically determine whether the person is one of the enrolled players, map the person to an enrolled player and interact with the person in the video game based on the mapping if it is determined that the person is one of the enrolled players, and assign a new identification to the person and interact with the person in the video game based on the new identification if it is determined that the person is not one of the enrolled players.

12. The video game apparatus of claim 11, wherein:

the one or more processors automatically determine whether the person is one of the enrolled players by visually identifying the person.

13. The video game apparatus of claim 11, wherein:

the sensor system senses visual data of the person and depth data of the person, the one or more processors automatically determine whether the person is one of the enrolled players based on the visual data and the depth data.

14. The video game apparatus of claim 11, wherein:

the one or more processors create visual signatures for the person and use thresholds to compare the visual thresholds for the person to stored visual signatures for the enrolled players.

15. The video game apparatus of claim 14, wherein:

the one or more processors dynamically update the thresholds while performing the video game and based on performing the video game.

16. The video game apparatus of claim 14, wherein:

at least one of the visual signatures is created using depth data from the sensor system.

17. The video game apparatus of claim 16, wherein:

the visual signatures includes a visual signature for an appearance of the person'"'"'s face, a visual signature for an appearance of the person'"'"'s clothing and a visual signature for the person'"'"'s height.

18. The video game apparatus of claim 11, wherein:

the one or more processors interact with the person in the video game based on the new identification by controlling an avatar in the video game based on motion of the person.

19. One or more processor readable storage devices having processor readable code embodied on said one or more processor readable storage devices, said processor readable code for programming one or more processors to perform a method comprising:

receiving visual data and depth data from one or more sensors pertaining to a first entity not currently interacting with an application;

creating a set of visual signatures indicative of appearance of the first entity based on the visual data and depth data;

comparing the set of visual signatures to stored signatures for entities enrolled in the application;

determining whether the visual signatures correspond to stored signatures for any of the entities enrolled in the application; and

mapping the first entity to a particular entity enrolled in the application if it is determined that the visual signatures correspond to stored signatures for the particular entity, the mapping allows the first entity to interact with the application.

20. One or more processor readable storage devices according to claim 19, wherein:

the set of visual signatures includes a signature for an appearance of the first entity'"'"'s face, a signature for an appearance of the first entity'"'"'s clothing and a signature for the first entity'"'"'s height.