IMAGE PROCESSOR, IMAGE PROCESSING METHOD, AND COMPUTER PROGRAM

• US 20160110610A1
• Filed: 08/17/2015
• Published: 04/21/2016
• Est. Priority Date: 10/15/2014
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: IKENOUE, Shoichi
• Granted Patent: US 9,767,568 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Disclosed herein is an image processor including: a depth image acquisition portion adapted to acquire a depth image for an image frame making up the movie, the depth image representing, as a pixel value on an image plane, a subject distance from an imaging device; an edge extraction portion adapted to generate an edge image for the image frame and identify a picture area of a tracked target in the depth image on the basis of an outline of the tracked target represented by tracking results for the image frame at a previous time so as to extract an edge in an area limited on the basis of the picture area; and a tracking section adapted to compare the extracted edge against an outline candidate for the tracked target to find a likelihood so as to estimate the outline of the tracked target and output the outline as tracking results.

Claims

1. An image processor for tracking a picture of a target in a movie, the image processor comprising:

a depth image acquisition portion adapted to acquire a depth image for an image frame making up the movie, the depth image representing, as a pixel value on an image plane, a subject distance from an imaging device;

an edge extraction portion adapted to generate an edge image for the image frame and identify a picture area of a tracked target in the depth image on a basis of an outline of the tracked target represented by tracking results for the image frame at a previous time so as to extract an edge in an area limited on a basis of the picture area from the edge image; and

a tracking section adapted to compare the extracted edge against an outline candidate for the tracked target to find a likelihood so as to estimate the outline of the tracked target and output the outline as tracking results.

2. The image processor of claim 1,

wherein the edge extraction portion sets a processing area for the depth image on the basis of the outline of the tracked target represented by the tracking results for the image frame at the previous time and identifies the picture area of the tracked target by limiting the processing area.

3. The image processor of claim 1,

wherein the edge extraction portion further extracts a foreground area from the image frame using a background image prepared in advance and limits the area from which to extract edges using a tracked target picture area identified in the depth image and the foreground image.

4. The image processor of claim 1,

wherein the edge extraction portion performs edge extraction on an image with a reduced number of gray levels, obtained by subjecting the image frame to given gray level reduction, so as to generate an edge image for the image frame.

5. The image processor of claim 1,

wherein the edge extraction portion finds the distance of the tracked target from the imaging device from the depth image on the basis of the outline so as to identify, as the tracked target picture area, an area made up of pixels having values that fall within a given range from the distance.

6. The image processor of claim 1,

wherein the tracking section generates the outline candidate by distributing, in a parameter space, particles that represent a shape space vector that defines an outline and finds a presence probability distribution of particles representing the outline of the tracked target in the parameter space on a basis of a likelihood of the outline candidate represented by each particle for edges extracted by the edge extraction portion so as to estimate the outline.

7. An image processing method for an image processor to track a picture of a target in a movie, the image processing method comprising:

acquiring an image frame making up the movie from an imaging device and storing the image frame in a memory;

acquiring a depth image for the image frame making up the movie, the depth image representing, as a pixel value on an image plane, a subject distance from the imaging device;

reading the image frame from the memory, generating an edge image for the image frame, and identifying a picture area of a tracked target in the depth image on a basis of an outline of the tracked target represented by tracking results for the image frame at a previous time so as to extract an edge in an area limited on a basis of the picture area from the edge image; and

comparing the extracted edges against an outline candidate for the tracked target to find a likelihood so as to estimate the outline of the tracked target and store the outline as tracking results in the memory.

8. A computer program causing a computer for tracking a picture of a target in a movie to achieve, the computer program comprising:

acquiring a depth image for an image frame making up the movie, the depth image representing, as a pixel value on an image plane, a subject distance from an imaging device;

generating an edge image for the image frame and identifying a picture area of a tracked target in the depth image on a basis of an outline of the tracked target represented by tracking results for the image frame at a previous time so as to extract an edge in an area limited on a basis of the picture area from the edge image; and

comparing the extracted edge against an outline candidate for the tracked target to find a likelihood so as to estimate the outline of the tracked target and output the outline as tracking results.