Method and device for identifying and extracting images of multiple users, and for recognizing user gestures

• US 8,411,149 B2
• Filed: 08/03/2006
• Issued: 04/02/2013
• Est. Priority Date: 08/03/2006
• Status: Expired due to Fees

• Current Assignee: Alterface S.A.
• Original Assignee: Alterface S.A.
• Inventors: Sohy, Xavier, Maison, Benot
• Primary Examiner(s): Tran, Sinh
• Assistant Examiner(s): FLOHRE, JASON A
• Application Number:
  US12/309,800
• Publication Number:
  US 20090244309A1
• Time in Patent Office: 2,434 Days
• Field of Search: None
• US Class Current: 348/207.1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/27 characterised by a large display in a public venue, e.g. in a movie theatre, stadium or game arena
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/6623 for animating a group of characters
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

The invention relates to a method for identifying and extracting images of one or more users in an interactive environment comprising the steps of: —obtaining a depth map (7) of a scene in the form of an array of depth values, and an image (8) of said scene in the form of a corresponding array of pixel values, said depth map (7) and said image (8) being registered; applying a coordinate transformation to said depth map (7) and said image (8) for obtaining a corresponding array (15) containing the 3D positions in a real-world coordinates system and pixel values points; —grouping said points according to their relative positions, by using a clustering process (18) so that each group contains points that are in the same region of space and correspond to a user location (19); —defining individual volumes of interest (20) each corresponding to one of said user locations (19); —selecting, from said array (15) containing the 3D positions and pixel values, the points located in said volumes of interest for obtaining segmentation masks (35) for each user; —applying said segmentation masks (35) to said image (8) for extracting images of said users. The invention also relates to a method for recognizing gestures of said users.

Claims

1. A device for identifying and extracting images of multiple users in an interactive environment scene comprising:

a video camera for capturing an image from the scene;

a depth perception device for providing depth information about said scene;

at least one computer processor for processing said depth information and said image information, wherein;

said device comprises means for using individual volume of interest from said scene for each user;

said video camera comprises means for obtaining an image of said scene in a form of a corresponding array of pixel values;

said depth perception device comprises means for obtaining a depth map of said scene, said depth map and said image being registered to one another;

said at least one computer processor comprises means for coordinate transforming said depth map and said image to obtain a corresponding array of points containing 3D positions in a real-world coordinates system and pixel values at said 3D positions;

said at least one computer processor also comprises means for grouping said points according to their relative positions, by using a clustering process so that each group contains points that are in a same region of space and correspond to a respective user location;

said at least one computer processor further comprises means for defining individual volumes of interest, each volume of interest corresponding to one of said user locations;

said means for using said individual volumes of interest comprises means for selecting, from said array of points containing the 3D positions and pixel values, the points located in said volumes of interest for obtaining segmentation masks for each user; and

said means for using said individual volumes of interest further comprises means for applying said segmentation masks to said image for extracting images of said users.

2. A device for identifying and extracting images of multiple users in an interactive environment scene comprising:

means for obtaining a depth map of a scene in a form of an array of depth values, and an image of said scene in a form of a corresponding array of pixel values, said depth map and said image being registered to one another,

means for coordinate transforming said depth map and said image to obtain a corresponding array of points containing 3D positions in a real-world coordinates system and pixel values at said 3D positions;

means for grouping said points according to their relative positions, by using a clustering process so that each group contains points that are in a same region of space and correspond to a respective user location;

means for defining individual volumes of interest, each volume of interest corresponding to one of said user locations;

means for selecting, from said array of points containing the 3D positions and pixel values, the points located in said volumes of interest for obtaining segmentation masks for each user, and

means for applying said segmentation masks to said image for extracting images of said users.