Image processor, image processing method, and computer program

• US 9,767,568 B2
• Filed: 08/17/2015
• Issued: 09/19/2017
• Est. Priority Date: 10/15/2014
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Ikenoue, Shoichi
• Primary Examiner(s): Lee, Jonathan S
• Application Number:
  US14/827,697
• Publication Number:
  US 20160110610A1
• Time in Patent Office: 764 Days
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Disclosed herein is an image processor including: a depth image acquisition portion adapted to acquire a depth image for an image frame making up the movie, the depth image representing, as a pixel value on an image plane, a subject distance from an imaging device; an edge extraction portion adapted to generate an edge image for the image frame and identify a picture area of a tracked target in the depth image on the basis of an outline of the tracked target represented by tracking results for the image frame at a previous time so as to extract an edge in an area limited on the basis of the picture area; and a tracking section adapted to compare the extracted edge against an outline candidate for the tracked target to find a likelihood so as to estimate the outline of the tracked target and output the outline as tracking results.

Claims

1. An image processing method for an image processor to track a first tracked target and a second tracked target in a movie, the first tracked target located at a first z-distance z1 and the second tracked target located at a second z-distance z2 at least a z-distance z3 from z1, the image processing method comprising:

a) calculating a depth image from an image frame making up a movie at time t, the depth image representing, as a pixel value on an image plane, a unique subject distance from an imaging device,

wherein the depth image is calculated for an entirety of the image frame;

b) acquiring a first tracking result for the first tracked target and a second tracking result for the second tracked target at time t−

1,

c) determining a first processing area for the first tracked target and a second processing area for the second tracked target,

wherein the first processing area and the second processing area are determined based on a basis of movable ranges of the first tracked target and the second tracked target, respectively,

wherein the first z-distance and the second z-distance are used to adjust the movable ranges,

d) extracting a first picture area for the first tracked target from the image frame at time t by;

fitting the first tracking result at time t−

1 to the depth image;

calculating an average z-distance Zt for all pixels within an area bounded by the first tracking result; and

extracting all pixels in the image frame which have values within a range of Zt−

Δ

Z<

z1<

Zt+Δ

Z in the depth image to form the first picture area,

wherein Δ

Z<

z3;

e) extracting a second picture area for the second tracked target from the image frame at time t by;

fitting the second tracking result at time t−

1 to the depth image;

calculating an average z-distance Zw for all pixels within an area bounded by the second tracking result; and

extracting all pixels in the image frame which have values within a range of Zw−

Δ

Z<

z2<

Zw+Δ

Z in the depth image to form the second picture area;

f) generating a third tracking result for the first tracked target at time t using the first picture area;

g) generating a fourth tracking result for the second tracked target at time t using the second picture area; and

h) repeating steps a)-g) for an image frame at time t+1 to track the first tracked target and the second tracked target in the movie.

2. The method of claim 1, wherein the wherein step c) further includes extracting a foreground area from the image frame using a background image prepared in advance.

3. The method of claim 1, wherein the image frame is subjected to gray level reduction prior to steps d) and e).

4. An image processing method for an image processor to track a tracked target in a movie, the image processing method comprising:

a) calculating a depth image from an image frame making up a movie at time t, the depth image representing, as a pixel value on an image plane, a unique subject distance from an imaging device,

wherein the depth image is calculated for an entirety of the image frame;

b) acquiring a tracking result for the tracked target at time t−

1,

c) determining a processing area for the first tracked target,

wherein the processing area is determined based on a movable range of the tracked target,

wherein a z-distance z1 of the tracked target is used to adjust the movable range,

d) extracting a picture area for the tracked target from the image frame at time t by;

fitting the tracking result at time t−

1 to the depth image;

calculating an average z-distance Zt for all pixels within an area bounded by the tracking result;

and extracting all pixels in the image frame which have values within a range of Zt−

Δ

Z<

z1<

Zt+Δ

Z in the depth image to form the picture area;

e) generating a tracking result for time using the picture area;

f) repeating steps a)-e) for an image frame at time t+1 to track the tracked target in the movie.