Method of image processing and device therefore

• US 9,031,282 B2
• Filed: 10/19/2012
• Issued: 05/12/2015
• Est. Priority Date: 10/21/2011
• Status: Active Grant

• Current Assignee: Industry-University Cooperation Foundation Hanyang University, LG Innotek Company Limited (LG Corporation)
• Original Assignee: LG Innotek Company Limited (LG Corporation)
• Inventors: Park, Jung Ah, Jhee, Ho Jin, Kim, Whoi Yul, Kwak, Jae Ho, Ra, Moon Soo, Hyun, Jong Min
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US13/655,985
• Publication Number:
  US 20130101170A1
• Time in Patent Office: 935 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

An image processing method and an image processing apparatus are provided. The image processing method includes dividing the image into a plurality of regions; setting a portion of the divided regions to a first region of interest; detecting a candidate region for a target from the first region of interest; determining if the detected candidate region corresponds to the target; detecting a target region by using the candidate region if the candidate region corresponds to the target; estimating a pose of the target by using the detected target region; and performing modeling with respect to the target.

Claims

1. An image processing method comprising:

acquiring depth information of an image;

dividing the image into a plurality of regions based on the acquired depth information;

setting a portion of the divided regions to a first region of interest;

setting a plurality of reference depth values with respect to an entire range of the depth information of the image;

extracting pixel sets existing in a preset unit section formed about the plurality of reference depth values to decompose the image of the first region of interest into the pixel sets;

creating at least one pixel object by performing a labeling process with respect to each pixel set;

removing a pixel object determined as noise by analyzing the pixel object;

creating a candidate region for a region corresponding to the target in relation to at least one pixel object which is not determined as noise;

determining if the detected candidate region corresponds to the target;

detecting a target region by using the candidate region if the candidate region corresponds to the target;

estimating a pose of the target by using the detected target region; and

performing modeling with respect to the target;

wherein each section formed about each reference depth value is overlapped with an adjacent section.

2. The image processing method of claim 1, wherein an interval between the plurality of reference depth values is equal to or less than a size of the unit section.

3. The image processing method of claim 1, wherein the removed pixel object corresponds to one of a first case in which a number of pixels constituting the at least one pixel object is less than a preset threshold value;

a second case in which a width of a rectangle surrounding an outer portion of the at least one pixel object is less than a preset threshold value;

a third case in which a ratio of a number of pixels contained in the rectangle surrounding the outer portion of the at least one pixel object exists beyond a preset threshold range; and

a fourth case in which the at least one pixel object is spaced apart from a portion of a boundary line to divide the image into the regions.

4. The image processing method of claim 1, wherein the determining if the detected candidate region corresponds to the target comprises:

determining if a portion of the target exists beyond the first region of interest;

recovering the portion of the target existing beyond the first region of interest if the portion of the target exists beyond the first region of interest; and

determining similarity between the candidate region and a sample image by comparing the candidate region with the sample image;

wherein the portion of the target is recovered based on the unit section including the candidate region.

5. The image processing method of claim 1, wherein the detecting of the target region by using the candidate region if the candidate region corresponds to the target comprises:

separating a second region of interest by using the detected candidate region;

performing a binarization process with respect to the second region of interest by using average depth information of the candidate region;

performing a labeling process with respect to the second region of interest subject to the binarization process; and

removing noise from the second region of interest subject to the labeling process.

6. The image processing method of claim 1, wherein the estimating of the pose of the target by using the detected target region comprises:

smoothing an outline of the target region;

extracting a skeleton of the target region; and

estimating parts constituting the target.

7. The image processing method of claim 6, wherein the estimating of the parts constituting the target comprises:

estimating a central part by using a center of gravity of the target region; and

estimating a sub-part by using the central part.

8. An image processing apparatus comprising:

a memory; and

a processor for performing the following operations;

acquiring depth information of an image;

module separating a first region of interest for a region in which a target exists, from the image;

detecting a candidate region for the target from the first region of interest;

determining if the candidate region corresponds to the target;

estimating a pose of the target by using the target region which is detected by using the candidate region as the candidate region corresponds to the target; and

modeling the target;

wherein separating a first region of interest comprises dividing the image into a plurality of regions, and setting a part of the divided regions to the first region of interest; and

wherein detecting a candidate region comprises;

setting a plurality of reference depth values with respect to an entire range of the depth information of the image;

extracting pixel sets existing in a preset unit section formed about the plurality of reference depth values to decompose the image of the first region of interest into the pixel sets;

creating at least one pixel object by performing a labeling process with respect to each pixel set;

removing a pixel object determined as noise by analyzing the pixel object; and

detecting the candidate region for a region corresponding to the target in relation to at least one pixel object which is not determined as noise;

wherein each section formed about each reference depth value is overlapped with an adjacent section.

9. The image processing apparatus of claim 8, wherein an interval between the reference depth values is equal to or less than a size of the unit section.

10. The image processing apparatus of claim 8, wherein detecting a candidate region comprises removing the pixel object determined as noise by analyzing the pixel object, and wherein a criterion to remove the pixel object includes one of a first case in which a number of pixels constituting the at least one pixel object is less than a preset threshold value, a second case in which a width of a rectangle surrounding an outer portion of the at least one pixel object is less than a preset threshold value, a third case in which a ratio of a number of pixels contained in the rectangle surrounding the outer portion of the at least one pixel object exits beyond a preset threshold range, and a fourth case in which the at least one pixel object is spaced apart from a portion of a boundary line to divide the image into the regions.

11. The image processing apparatus of claim 8, wherein determining if the candidate region corresponds to the target comprises determining if a portion of the target exists beyond the first region of interest, recovering the portion of the target existing beyond the first region of interest if the portion of the target exists beyond the first region of interest, determining similarity between the candidate region and a sample image by comparing the candidate region with the sample image, and determining if the candidate region corresponds to the target.

12. The image processing apparatus of claim 8, wherein detecting candidate region comprises detecting the target region by separating a second region of interest by using the detected candidate region, performing a binarization process with respect to the second region of interest by using average depth information of the candidate region, performing a labeling process with respect to the second region of interest subject to the binarization process, and removing noise from the second region of interest subject to the labeling process.

13. The image processing apparatus of claim 8, wherein estimating a pose of the target comprises estimating a pose of the target region by performing a smoothing process with respect to an outline of the target region, extracting a skeleton of the target region, and estimating parts constituting the target.

14. The image processing apparatus of claim 13, wherein estimating a pose of the target comprises estimating the pose of the target region by estimating a central part by using a center of gravity of the target region, and estimating a sub-part by using the central part.