Image processing device and method

• US 9,449,222 B2
• Filed: 03/31/2015
• Issued: 09/20/2016
• Est. Priority Date: 04/24/2014
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Xie, Shufu, He, Yuan, Sun, Jun
• Primary Examiner(s): Niu, Feng
• Application Number:
  US14/674,882
• Publication Number:
  US 20150310302A1
• Time in Patent Office: 539 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

The present disclosure relates to an image processing device and method. The device includes: an acquiring unit for acquiring an image containing an object image; a calculating unit for calculating one or more candidate object image areas based on color and edge characteristics of the image; a selecting unit for selecting a candidate object image area with the maximal similarity to a predetermined color model as a first object image area from the one or more candidate object image areas; a determining unit for determining other of the one or more candidate object image areas as a non-object image area or a second object image area with the first object image area; a creating unit for creating an adaptive color model; and a detecting unit for detecting, based on the adaptive color model, an area containing the object image in the each of the first and second object image areas.

Claims

1. An image processing device, comprising:

an acquiring unit for acquiring an image containing an object image;

a calculating unit for calculating one or more candidate object image areas based on color and edge characteristics of the image;

a selecting unit for selecting, based on a predetermined color model, a candidate object image area with a maximal similarity to the predetermined color model as a first object image area from the one or more candidate object image areas, where the maximal similarity is larger than or equal to a similarity threshold;

a determining unit for determining, based on color characteristics of the one or more candidate object image areas, another of the one or more candidate object image areas as one of a non-object image area and a second object image area with the first object image area;

a creating unit for creating, for each of the first and second object image areas, an adaptive color model; and

a detecting unit for detecting, based on the respective adaptive color model, an area containing the object image in the each of the first and second object image areas.

2. The device according to claim 1, wherein the calculating unit comprises:

a page color calculating unit for calculating a page color based on a color of each pixel in the image;

a distance calculating unit for calculating a distance between the color of each pixel in the image and the page color to obtain a distance map;

a color image calculating unit for calculating a color binary image based on the distance map;

an edge image calculating unit for calculating an edge binary image based on the image;

a fusing unit for fusing the color binary image and the edge binary image to obtain a binary fused image; and

a connected component analyzing unit for performing a connected component analysis algorithm on the binary fused image, to obtain one or more connected components in the binary fused image as the one or more candidate object image areas.

3. The device according to claim 2, wherein the fusing unit comprises:

a classifying unit for classifying the pixel that is both a foreground pixel and a non-edge pixel as an object pixel, and classifying the pixel that is one of a background pixel and an edge pixel as a non-object pixel.

4. The device according to claim 2, wherein the color image calculating unit comprises:

a classifying unit for classifying the pixel with a distance between a color value and an average color value larger than or equal to a distance threshold as a foreground pixel, and classifying the pixel with a distance between the color value and the average color value smaller than the distance threshold as a non-object pixel, to obtain the color binary image.

5. The device according to claim 2, wherein the edge image calculating unit comprises:

a gradient calculating unit for applying, based on one of the distance map, color information and intensity information of the image, a gradient operator to the image to obtain a gradient image; and

a classifying unit for classifying, in the gradient image, the pixel with a gradient larger than or equal to a gradient threshold as an edge pixel, and classifying the pixel with a gradient smaller than the gradient threshold as a non-edge pixel, obtain the edge binary image.

6. The device according to claim 2, wherein the determining unit converts all pixels in a connected component with a distance to a predetermined edge of the image larger than a predetermined distance among the one or more connected components into non-object pixels, to make the connected component no longer a candidate object image area.

7. The device according to claim 2, wherein the calculating unit further comprises:

a counting unit for counting, for each row in the binary fused image, a sum of pixels which are object pixels;

a converting unit for converting all object pixels on a row for which the sum is smaller than a sum threshold into non-object pixels; and

an updating unit for updating labeling information of each connected component.

8. The device according to claim 2, wherein the determining unit converts all pixels in a connected component with an area smaller than one of a lower limit area threshold and larger than an upper limit area threshold among the one or more connected components into non-object pixels, to make the connected component no longer a candidate object image area.

9. The device according to claim 1, wherein the determining unit comprises:

an average color calculating unit for calculating an average color value of pixels in the first object image area;

a distance calculating unit for calculating, in one of other candidate object image areas, a distance between a color value of each pixel and the average color value;

a classifying unit for classifying the pixel with a distance between the color value and the average color value smaller than or equal to a distance threshold as an object pixel, and classifying the pixel with a distance between the color value and the average color value larger than the distance threshold as a non-object pixel;

a ratio calculating unit for calculating a ratio of a number of object pixels to a number of pixels in the one of the other candidate object image areas; and

an object image determining unit for determining a one of the other candidate object image areas with the ratio larger than or equal to a ratio threshold as a second object image area, and determining the one of the other candidate object image areas with the ratio smaller than the ratio threshold as a non-object image area.

10. The device according to claim 1, further comprising:

an object image obtaining unit for obtaining, based on color characteristics of other areas of the image except for the one or more candidate object image areas, a third object image area from the other areas with the first object image area,

wherein the creating unit creates a further adaptive color model for the third object image area, and

the detecting unit detects an area containing the object image in the third object image area based on the further adaptive color model.

11. The device according to claim 10, wherein the object image obtaining unit comprises:

an average color calculating unit for calculating an average color value of pixels in the first object image area;

a distance calculating unit for calculating, in other areas of the image except for the one or more candidate object image areas, a distance between a color value of each pixel and the average color value;

a classifying unit for classifying the pixel with a distance between the color value and the average color value smaller than or equal to a distance threshold as an object pixel, and classifying the pixel with a distance between the color value and the average color value larger than the distance threshold as a non-object pixel, to obtain an other area binary image; and

a connected component analyzing unit for performing a connected component analysis algorithm on the other area binary image, to obtain a connected component in the other area binary image as the third object image area.

12. An image processing method, comprising:

acquiring an image containing an object image;

calculating one or more candidate object image areas based on color and edge characteristics of the image;

selecting, based a predetermined color model, a candidate object image area with a maximal similarity to the predetermined color model as a first object image area from the one or more candidate object image areas, wherein the maximal similarity is larger than or equal to a similarity threshold;

determining, based on color characteristics of the one or more candidate object image areas, another of the one or more candidate object image areas as one of a non-object image area and a second object image area with the first object image area;

creating, for each of the first and second object image areas, an adaptive color model; and

detecting, based on the respective adaptive color model, an area containing the object image in the each of the first and second object image areas.

13. The method according to claim 12, wherein the calculating the one or more candidate object image areas based on the color and edge characteristics of the image comprises:

calculating a page color based on the color of each pixel in the image;

calculating a distance between the color of each pixel in the image and the page color to obtain a distance map;

calculating a color binary image based on the distance map;

calculating an edge binary image based on the image;

fusing the color binary image and the edge binary image to obtain a binary fused image; and

performing a connected component analysis algorithm on the binary fused image, to obtain one or more connected components in the binary fused image as the one or more candidate object image areas.

14. The method according to claim 13, wherein the calculating the one or more candidate object image areas based on the color and edge characteristics of the image further comprises:

counting, for each row in the binary fused image, a sum of pixels which are object pixels;

converting all object pixels on a row for which the sum is smaller than a sum threshold into non-object pixels; and

updating labeling information of each connected component.

15. The method according to claim 13, wherein all pixels in a connected component with one of an area smaller than a lower limit area threshold and larger than an upper limit area threshold among the one or more connected components are converted into non-object pixels, to make the connected component no longer a candidate object image area.

16. The method according to claim 12, wherein the determining, based on color characteristics of the one or more candidate object image areas, other of the one or more candidate object image areas as a non-object image area or a second object image area with the first object image area comprises:

calculating an average color value of pixels in the first object image area; and

performing, in one of the other candidate object image areas, the following steps;

calculating a distance between a color value of each pixel and the average color value;

classifying the pixel with a distance between the color value and the average color value smaller than or equal to a distance threshold as an object pixel, and classifying the pixel with a distance between the color value and the average color value larger than the distance threshold as a non-object pixel;

calculating a ratio of a number of object pixels to a number of pixels in a whole area; and

determining the one of the other candidate object image areas with the ratio larger than or equal to a ratio threshold as a second object image area, and determining the one of the other candidate object image areas with the ratio smaller than the ratio threshold as a non-object image area.

17. The method according to claim 12, further comprising:

obtaining, based on color characteristics of another area of the image except for the one or more candidate object image areas, a third object image area from the another area with the first object image area,

creating a further adaptive color model for the third object image area, and

detecting an area containing the object image in the third object image area based on the further adaptive color model.

18. The method according to claim 17, wherein the obtaining, based on color characteristics of other area of the image except for the one or more candidate object image areas, a third object image area from the other area with the first object image area comprises:

calculating an average color value of pixels in the first object image area;

in the another area of the image except for the one or more candidate object image areas, calculating, a distance between a color value of each pixel and the average color value;

classifying the pixel with a distance between the color value and the average color value smaller than or equal to a distance threshold as an object pixel, and classifying the pixel with a distance between the color value and the average color value larger than the distance threshold as a non-object pixel, to obtain an other area binary image; and

performing connected component analysis algorithm on the other area binary image, to obtain a connected component in the other area binary image as the third object image area.

19. A non-transitory machine-readable storage medium carrying a program product comprising machine-readable instruction codes stored therein, wherein the instruction codes, when read and executed by a computer, cause the computer to execute the method according to claim 12.