Information processor and information processing method

• US 10,210,629 B2
• Filed: 12/01/2014
• Issued: 02/19/2019
• Est. Priority Date: 02/14/2014
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Ohba, Akio, Segawa, Hiroyuki, Inada, Tetsugo, Ogasawara, Hidehiko, Okamoto, Hirofumi
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US15/115,521
• Publication Number:
  US 20170011519A1
• Time in Patent Office: 1,541 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

A detection area is set in a three-dimensional space in which a subject exists. When an actual hand enters the detection area, coordinate points (white and black dots) represented by pixels making up a silhouette of the hand in a depth image enter the detection area. In the detection area, a reference vector is set that shows the direction which the hand should face relative to the shoulder as a reference point. Then, an inner product between two vectors, a vector from the reference point to each of coordinate points and the reference vector, is calculated, followed by comparison between the inner products. Positions of coordinate points whose inner products are ranked high are acquired as the position of tips of the hand.

Claims

1. An information processor for detecting a position of a given target of a subject in a three-dimensional space shot with a camera, the information processor comprising:

a depth image acquisition portion adapted to acquire a depth image representing, on an image plane, a distance of the subject from the camera in a depth direction as a pixel value; and

a coordinate point analysis portion adapted to identify a tip position of the target and generate and output position information thereof by performing a given calculation on each of coordinate points included in a detection area set in the three-dimensional space and comparing these coordinate points if a given number or more of coordinate points representing pixels of the depth image in the three-dimensional space are included in the detection area,

wherein the coordinate point analysis portion determines a reference vector representing a direction which the target should face when the target exists in the detection area on the basis of a relative position of the detection area relative to a given reference point in a three-dimensional space and a motion characteristic of the target relative to the reference point and identifies the tip position of the target in a direction of the reference vector by calculating and comparing inner products, each between a coordinate point vector from the reference point to one of the coordinate points included in the detection area and the reference vector.

2. The information processor of claim 1, further comprising:

a reference part detection section adapted to detect, in a three-dimensional space, a position of another part serving as a reference for motion range of a target to be detected, wherein the coordinate point analysis portion identifies a position of the reference point on the basis of a position of the other part, determines the reference vector on the basis of the position of the reference point, and calculates inner products between the coordinate point vectors and the reference vector.

3. The information processor of claim 1, wherein after identifying a tip position of the target, the coordinate point analysis portion further identifies a tip of another target using the tip position of the target as a reference point.

4. The information processor of claim 1, further comprising:

a reference part detection section adapted to detect, in a three-dimensional space, another part serving as a reference for motion range of a target to be detected,

wherein the coordinate point analysis portion determines bounds within which to set detection areas on the basis of the motion range identified based on the position of the other part.

5. The information processor of claim 1, wherein

the depth image acquisition portion acquires the depth image for an image frame of a movie shot with a camera, and

the coordinate point analysis portion determines the bounds within which to set the detection areas on the basis of the position of the target detected in a previous time step.

6. The information processor of claim 1, wherein the coordinate point analysis portion includes, as coordinate points to be compared against, those coordinate points existing in areas adjacent to the detection area that includes the given number or more of coordinate points.

7. The information processor of claim 1, wherein the coordinate point analysis portion further sets, in the three-dimensional space, a dead area that defines bounds within which the coordinate points, included in the detection area, are not subject to processing.

8. The information processor of claim 1, wherein the coordinate point analysis portion changes a shape of the detection area on the basis of at least either a shape or motion characteristic of the target.

9. An information processing method for an information processor to detect a position of a given target of a subject in a three-dimensional space shot with a camera, the information processing method comprising:

acquiring a depth image representing, on an image plane, a distance of the subject from the camera in a depth direction as a pixel value and storing the depth image in a memory; and

identifying a tip position of the target and generating and outputting position information thereof by performing a given calculation on each of coordinate points included in a detection area set in the three-dimensional space and comparing these coordinate points if a given number or more of coordinate points representing pixels of the depth image read from the memory are included in the detection area,

wherein the identifying and generating includes determining a reference vector representing a direction which the target should face when the target exists in the detection area on the basis of a relative position of the detection area relative to a given reference point in a three-dimensional space and a motion characteristic of the target relative to the reference point and identifies the tip position of the target in a direction of the reference vector by calculating and comparing inner products, each between a coordinate point vector from the reference point to one of the coordinate points included in the detection area and the reference vector.

10. A non-transitory, computer-readable storage medium containing a computer program, which when executed by a computer, causes the computer to perform actions in connection with detecting a position of a given target of a subject in a three-dimensional space shot with a camera, the actions, comprising:

acquiring a depth image representing, on an image plane, a distance of the subject from the camera in a depth direction as a pixel value; and

identifying a tip position of the target and generating and outputting position information thereof by performing a given calculation on each of coordinate points included in a detection area set in the three-dimensional space and comparing these coordinate points if a given number or more of coordinate points representing pixels of the depth image in the three-dimensional space are included in the detection area,

wherein the identifying and generating includes determining a reference vector representing a direction which the target should face when the target exists in the detection area on the basis of a relative position of the detection area relative to a given reference point in a three-dimensional space and a motion characteristic of the target relative to the reference point and identifies the tip position of the target in a direction of the reference vector by calculating and comparing inner products, each between a coordinate point vector from the reference point to one of the coordinate points included in the detection area and the reference vector.