Apparatus for and method of processing image based on object region

• US 10,163,027 B2
• Filed: 06/24/2016
• Issued: 12/25/2018
• Est. Priority Date: 06/29/2015
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Yano, Kotaro
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US15/192,633
• Publication Number:
  US 20160379078A1
• Time in Patent Office: 914 Days
• Field of Search: 382106, 382173, 382190, 382107, 382199, 382284, 345633, 345634, 3482221, 348239, 600424
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

An image processing apparatus includes an image data acquisition unit configured to acquire image data, a distance information acquisition unit configured to acquire distance information corresponding to the image data, a detection unit configured to detect one or more object regions, each representing an object, from the image data based on the distance information, an extraction unit configured to extract an object region representing an object which does not overlap a specific three-dimensional region, from among detected object regions, based on the distance information, and an image processing unit configured to perform image processing for the image data based on the extracted object region.

Claims

1. An image processing apparatus comprising:

one or more processors; and

a memory coupled to the one or more processors, the memory having stored thereon instructions which, when executed by the one or more processors, cause the image processing apparatus to;

acquire image data;

acquire distance information corresponding to the image data;

detect one or more object regions, each representing an object, from the image data based on the distance information;

set a specific three-dimensional region;

extract an object region representing an object which does not overlap the specific three-dimensional region, from among detected object regions, based on the distance information; and

perform image processing for synthesizing the extracted object region in the image data and a corresponding region in the background image at a predetermined ratio or replacing the extracted object region in the image data with the corresponding region in the background image.

2. The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:

detect the object region from the image data based on the distance information and an image feature amount of the image data.

3. The image processing apparatus according to claim 2,

wherein the image feature amount is an image feature amount representing a contour of an object, and

wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:

extract an identification feature amount for identifying the contour of the object from the distance information, and detect the object region from the image data based on the identification feature amount and the image feature amount.

4. The image processing apparatus according to claim 3,

wherein the distance information is a range image, and

wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:

extract edge information from the range image as the identification feature amount.

5. The image processing apparatus according to claim 3, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:

acquire image data including no object as a background image.

6. The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the image processing apparatus to:

output a processing result of the image processing performed.

7. The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:

arrange an object corresponding to the specific three-dimensional region, acquire distance information corresponding to the object, and set the specific three-dimensional region based on the distance information.

8. The image processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:

distinguish an object region of an object existing inside the specific three-dimensional region and an object region of an object existing outside the specific three-dimensional region, among objects whose object positions have been detected, and perform the image processing.

9. The image processing apparatus according to claim 8, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:

perform image processing for making one of the object region of the object existing inside the specific three-dimensional region and the object region of the object existing outside the specific three-dimensional region translucent.

10. The image processing apparatus according to claim 8, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:

perform image processing for blurring one of the object region of the object existing inside the specific three-dimensional region and the object region of the object existing outside the specific three-dimensional region.

11. An image processing method performed by a computer comprising:

acquiring image data;

acquiring distance information corresponding to the image data;

detecting one or more object regions, each representing an object, from the image data based on the distance information;

setting a specific three-dimensional region;

extracting an object region representing an object which does not overlap the specific three-dimensional region, from among detected object regions, based on the distance information; and

performing image processing for synthesizing the extracted object region in the acquired image data and a corresponding region in the background image at a predetermined ratio or replacing the extracted object region in the image data with the corresponding region in the background image.

12. A non-transitory computer-readable storage medium storing a program for causing a computer to:

acquire image data;

acquire distance information corresponding to the image data;

detect one or more object regions, each representing an object, from the image data based on the distance information;

set a specific three-dimensional region;

extract an object region representing an object which does not overlap the specific three-dimensional region, from among detected object regions, based on the distance information; and

perform image processing for synthesizing the extracted object region in the image data and a corresponding region in the background image at a predetermined ratio or replacing the extracted object region in the image data with the corresponding region in the background image.