Image processing device

• US 6,332,038 B1
• Filed: 04/06/1999
• Issued: 12/18/2001
• Est. Priority Date: 04/13/1998
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Konya, Minehiro, Funayama, Ryuji, Takezawa, Hajime
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US09/286,475
• Time in Patent Office: 987 Days
• Field of Search: 382/103, 382/107, 382/115, 382/118, 382/181, 382/190, 382/203-206, 382/286, 382/288, 382/293, 382/298, 348/77, 348/78, 348/135, 348/152, 348/169, 348/171
• US Class Current: 382/190
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

The present image processing device includes an image obtaining device which obtains an electronic image (still image) or a series of electronic images (moving image), an image extraction processing device which performs image extraction processing for extracting from the image obtained by the image obtaining device a specified object domain an operator wishes to cut out, and a result output device which outputs an image extracted by the image extraction processing device. The image extraction processing device sets two mutually exclusive provisional domains in the obtained image, and, based on separability of the provisional domains, extracts a partial image containing the specified object domain.

Claims

1. An image processing device which extracts from an image obtained from outside said image processing device a partial image containing a specified object domain for extraction, comprising:

image extraction processing means, which set two mutually exclusive provisional domains in the obtained image, and extract the partial image based on separability of the two provisional domains,

wherein said image extraction processing means extract the partial image based on a separability series obtained by incrementally changing the sizes of the two provisional domains, and

wherein the sizes of the two provisional domains are changed by setting the two provisional domains in the obtained image to initially have sizes smaller than a size of the obtained image, and then successively increasing the sizes of the two provisional domains, while keeping their centers fixed, until they exceed limits of a screen of the obtained image.

2. The image processing device set forth in claim 1, wherein:

the two mutually exclusive provisional domains are a first provisional domain and a second provisional domain surrounding the first provisional domain, an inner boundary of the second provisional domain being in contact with an outer boundary of the first provisional domain.

3. The image processing device set forth in claim 2, wherein:

the outer boundary of the first provisional domain has a shape which is substantially the same as that of the specified object domain for extraction.

4. The image processing device set forth in claim 2, wherein:

respective outer boundaries of the first and second provisional domains have mutually similar shapes.

5. The image processing device set forth in claim 4, wherein:

respective outer boundaries of the first and second provisional domains have shapes which are substantially the same as that of the specified object domain for extraction.

6. The image processing device set forth in claim 2, wherein:

said image extraction processing means extract the partial image based on a separability series obtained by incrementally changing the sizes of the first and second provisional domains.

7. The image processing device set forth in claim 2, wherein:

said image extraction processing means set first provisional domains centered on each of the pixels of the obtained image.

8. The image processing device set forth in claim 1, wherein said image extraction processing means include:

domain setting means, which set the two mutually exclusive provisional domains in the obtained image;

image information obtaining means, which obtain image information of each of the provisional domains;

calculating means, which, based on the image information, calculate the separability of the two provisional domains; and

image extracting means, which, based on the calculated separability, extract the partial image from the obtained image such that the specified object domain present in the obtained image is contained in the partial image.

9. The image processing device set forth in claim 8, wherein:

said image extracting means set a further provisional domain in part of the partial image extracted so as to contain the specified object domain, obtain image information of the further provisional domain, and, based on the image information, extract a domain, smaller than the partial image, which contains the specified object domain.

10. The image processing device set forth in claim 9, wherein:

when the image information obtained is pixel distribution information, said image extracting means calculate a probability density function from the distribution information, and, by applying the probability density function to the partial image, extract a domain, smaller than the partial image, which contains the specified object domain.

11. The image processing device set forth in claim 1, wherein:

the specified object domain is a human face domain.

12. The image processing device set forth in claim 11, wherein said image extracting means include:

means for obtaining image information of an extracted human face domain; and

means for detecting, as numerical information, facial characteristics from the image information;

wherein said image extracting means, based on the numerical information, extract the partial image from the obtained image such that a facial portion contained in the human face domain has a predetermined size and position.

13. The image processing device set forth in claim 12, wherein:

said numerical information is at least one of facial width, vertical axis of symmetry, vertical position of eyes, vertical position of nose, and vertical position of mouth.

14. An image processing device which extracts from an image obtained from outside said image processing device a partial image containing a specified object domain for extraction, comprising:

image extraction processing means, which set two mutually exclusive provisional domains in the obtained image, and extract the partial image based on separability of the two provisional domains,

point indicating means which indicate a given point in the obtained image, the point indicated by the point indicating means in the obtained image being indicated by a user, wherein;

said image extraction processing means include;

domain setting means, which set the two mutually exclusive provisional domains in the obtained image;

image information obtaining means, which obtain image information of each of the provisional domains;

calculating means, which, based on the image information, calculate separability of the two provisional domains; and

image extracting means, which, based on the calculated separability, extract the partial image from the obtained image such that the specified object domain present in the obtained image is contained in the partial image, and wherein

said domain setting means set a first provisional domain containing the point indicated by said point indicating means, and a second provisional domain exclusive of the first provisional domain; and

said image extracting means extract the partial image from the obtained image such that the partial image contains the specified object domain, which in turn contains the point indicated by said point indicating means.

15. The image processing device set forth in claim 14, wherein:

the second provisional domain surrounds the first provisional domain, an inner boundary of the second provisional domain being in contact with an outer boundary of the first provisional domain.

16. The image processing device set forth in claim 15, wherein:

at least one of an outer boundary of the first provisional domain and an outer boundary of the second provisional domain has a shape which is substantially the same as that of the specified object domain for extraction.

17. The image processing device set forth in claim 14, wherein:

when said point indicating means indicate a point in the center of the specified object domain present in the obtained image, said domain setting means set a first provisional domain containing the point indicated by said point indicating means, and a second provisional domain exclusive of the first provisional domain, and incrementally change sizes of the first and second provisional domains; and

said image extracting means, based on a separability series obtained by incrementally changing the sizes of the first and second provisional windows, extract the partial image from the obtained image such that the specified object domain, which is present in the obtained image and which contains at its own center the point indicated by said point indicating means, falls in the center of the partial image.

18. The image processing device set forth in claim 17, wherein:

said image extracting means set a further provisional domain in part of the partial image extracted so as to contain the specified object domain, obtain image information of the further provisional domain, and, based on the image information, extract a domain, smaller than the partial image, which contains the specified object domain.

19. The image processing device set forth in claim 18, wherein:

when the image information obtained is pixel distribution information, said image extracting means calculate a probability density function from the distribution information, and, by applying the probability density function to the partial image, extract a domain, smaller than the partial image, which contains the specified object domain.

20. The image processing device set forth in claim 17, wherein:

the specified object domain is a human face domain.

21. The image processing device set forth in claim 20, wherein said image extracting means include:

means for obtaining image information of an extracted human face domain; and

means for detecting, as numerical information, facial characteristics from the image information;

wherein said image extracting means, based on the numerical information, extract the partial image from the obtained image such that a facial portion contained in the human face domain has a predetermined size and position.

22. The image processing device set forth in claim 21, wherein:

said numerical information is at least one of facial width, vertical axis of symmetry, vertical position of eyes, vertical position of nose, and vertical position of mouth.

23. The image processing device set forth in claim 14, wherein:

when said point indicating means indicate a point in a position other than the center of the specified object domain present in the obtained image, said domain setting means set a first provisional domain containing the point indicated by said point indicating means, and a second provisional domain exclusive of the first provisional domain, and incrementally change sizes of the first and second provisional domains; and

said image extracting means, based on a separability series obtained by incrementally changing the sizes and positions of the first and second provisional windows, extract the partial image from the obtained image so as to contain the specified object domain, which is present in the obtained image and which contains the point indicated by said point indicating means.

24. The image processing device set forth in claim 23, wherein:

said image extracting means sets a further provisional domain in part of the partial image extracted so as to contain the specified object domain, obtain image information of the further provisional domain, and, based on the image information, extract a domain, smaller than the partial image, which contains the specified object domain.

25. The image processing device set forth in claim 24, wherein:

when the image information obtained is pixel distribution information, said image extracting means calculate a probability density function from the distribution information, and, by applying the probability density function to the partial image, extract a domain, smaller than the partial image, which contains the specified object domain.

26. The image processing device set forth in claim 23, wherein:

the specified object domain is a human face domain.

27. The image processing device set forth in claim 26, wherein said image extracting means include:

means for obtaining image information of an extracted human face domain; and

means for detecting, as numerical information, facial characteristics from the image information;

wherein said image extracting means, based on the numerical information, extract the partial image from the obtained image such that a facial portion contained in the human face domain has a predetermined size and position.

28. The image processing device set forth in claim 27, wherein:

said numerical information is at least one of facial width, vertical axis of symmetry, vertical position of eyes, vertical position of nose, and vertical position of mouth.

29. The image processing device set forth in claim 14, wherein:

said domain setting means incrementally change not only the sizes and positions, but also the shapes of the first and second provisional domains; and

said image extracting means, based on a separability series obtained by incrementally changing the sizes, positions, and shapes of the first and second provisional windows, extract the partial image from the obtained image so as to contain the specified object domain, which is present in the obtained image and which contains the point indicated by said point indicating means.

30. The image processing device set forth in claim 29, wherein:

the second provisional domain surrounds the first provisional domain, an inner boundary of the second provisional domain being in contact with an outer boundary of the first provisional domain; and

said domain setting means include means for changing the shape of the second provisional domain by modifying the outer boundary of the second provisional domain with the first provisional domain in a fixed state.

31. An image processing device which extracts from an image obtained from outside said image processing device a partial image containing a specified object domain for extraction, comprising:

image extraction processing means, which set two mutually exclusive provisional domains in the obtained image, and extract the partial image based on separability of the two provisional domains, wherein

said image extraction processing means further include memory means which store the obtained image; and

when the obtained image is a moving image made up of a series of electronic images, said image extraction processing means read out, as an image in question, a given image of the series of images stored in the memory means;

extract a partial image from the image in question such that a specified object domain present in the image in question is contained in the partial image;

set in the image in question a pair of windows as initial windows for the next image in the series of obtained images, change the sizes and positions of the provisional domains to obtain various image information; and

use the image information to determine the most suitable pair of windows.

32. The image processing device set forth in claim 31, wherein:

said image extracting means set a further provisional domain in part of the partial image extracted so as to contain the specified object domain, obtain information of the further provisional domain, and, based on the image information, extract a domain, smaller than the partial image, which contains the specified object domain.

33. The image processing device set forth in claim 31, wherein:

the specified object domain in the series of electronic images stored in said memory means is a human face domain.

34. The image processing device set forth in claim 31, further comprising:

point indicating means which indicate a given point in the obtained image;

wherein said image extraction processing means include;

means which, when the obtained image is a moving image made up of a series of electronic images, and when the image extraction processing means read out, as an image in question, a given image of the series of images stored in the memory means and said point indicating means indicate a point in the center of the specified object domain present in the image in question, set a first provisional domain containing the indicated point and a second provisional domain exclusive of the first provisional domain;

means for calculating separability of the specified object domain and the first and second provisional domains based on image information of the first and second provisional domains; and

means which, based on a separability series obtained by incrementally changing the sizes of the first and second provisional domains, extract the partial image from the image in question such that the specified object domain, containing at its own center the point indicated by said point indicating means, falls in the center of the partial image;

wherein said image extraction processing means, using a pair of windows made up of the two provisional domains as initial windows for the next image in the series of obtained images, change size and position of the provisional domains to obtain various image information, and use the image information to determine the most suitable pair of windows.

35. The image processing device set forth in claim 34, wherein:

said image extracting means sets a further provisional domain in part of the partial image extracted so as to contain the specified object domain, obtain image information of the further provisional domain, and, based on the image information, extract a domain, smaller than the partial image, which contains the specified object domain.

36. The image processing device set forth in claim 34, wherein:

the specified object domain in the series of electronic images stored in said memory means is a human face domain.

37. The image processing device set forth in claim 31, further comprising:

point indicating means which indicate a given point in the obtained image;

wherein said image extraction processing means include;

memory means which store the obtained image;

means which, when the obtained image is a moving image made up of a series of electronic images, and when said image extraction processing means read out, as an image in question, a given image of the series of images stored in the memory means and said point indicating means indicate a given point in the specified object domain present in the image in question, set a first provisional domain containing the indicated point and a second provisional domain exclusive of the first provisional domain;

means for calculating separability of the specified object domain and the first and second provisional domains based on image information of the first and second provisional domains; and

image extracting means, which, based on a separability series obtained by incrementally changing the sizes of the first and second provisional domains, extract the partial image from the image in question such that the partial image contains the specified object domain, which is present in the obtained image and contains the point indicated by said point indicating means;

wherein said image extraction processing means, using a pair of windows made up of the two provisional domains as initial windows for the next image in the series of obtained images, change the sizes and positions of the provisional domains to obtain various image information, and use the image information to determine the most suitable pair of windows.

38. The image processing device set forth in claim 37, wherein:

when the image information obtained is pixel distribution information, said image extracting means calculate a probability density function from the distribution information, and, by applying the probability density function to the partial image, extract a domain, smaller than the partial image, which contains the specified object domain.

39. The image processing device set forth in claim 38, wherein:

said image extracting means set a further provisional domain in part of the partial image extracted so as to contain the specified object domain, obtain image information of the further provisional domain, and, based on the image information, extract a domain, smaller than the partial image, which contains the specified object domain.

40. The image processing device set forth in claim 37, wherein:

the specified object domain in the series of electronic images stored in said memory means is a human face domain.

41. The image processing device set forth in claim 40, wherein said image extracting means include:

means for obtaining image information of an extracted human face domain; and

means for detecting, as numerical information, facial characteristics from the image information;

wherein said image extracting means, based on the numerical information, extract the partial image from the obtained image such that a facial portion contained in the human face domain has a predetermined size and position.

42. The image processing device set forth in claim 41, wherein:

said numerical information is at least one of facial width, vertical axis of symmetry, vertical position of eyes, vertical position of nose, and vertical position of mouth.

43. A computer-readable recording medium which records an image extraction processing program for causing a computer to execute processing including the steps of:

setting two mutually exclusive provisional domains in an obtained image;

extracting a specified object domain based on image information of the provisional domains,

wherein the step of extracting a specified object domain comprises;

obtaining a separability series by incrementally changing the sizes of the two provisional domains, the sizes of the two provisional domains being changed by setting the two provisional domains in the obtained image to initially have sizes smaller than a size of the obtained image and then successively increasing the sizes of the two provisional domains, while keeping their centers fixed, until they exceed limits of a screen of the obtained image.

44. A computer-readable medium which records an image extraction processing program for causing a computer to execute processing including the steps of:

setting two mutually exclusive provisional domains in an obtained image;

obtaining image information of the respective provisional domains;

calculating separability of the provisional domains based on the image information;

extracting a partial image from the obtained image such that a specified object domain present in the obtained image is contained in the partial image,

wherein the sizes of the two provisional domains are incrementally changed to obtain a separability series and the partial image is extracted based on the separability series, and

wherein the sizes of the two provisional domains are changed by setting the two provisional domains in the obtained image to initially have sizes smaller than a size of the obtained image and then successively increasing the sizes of the two provisional domains, while keeping their centers fixed, until they exceed limits of a screen of the obtained image.

45. A computer-readable recording medium which records an image extraction processing program for causing a computer to execute processing including the steps of:

when a point at a center of a specified object domain present in an obtained image is indicated by a user, setting a first provisional domain containing the indicated point and a second provisional domain exclusive of the first provisional domain;

calculating separability of the first and second provisional domains based on image information of the first and second provisional domains; and

based on a separability series obtained by incrementally changing the sizes of the first and second provisional domains, extracting a partial image from the obtained image such that the specified object domain, which is present in the obtained image and which contains the indicated point at its own center, falls in the center of the partial image.

46. A computer-readable recording medium which records an image extraction processing program for causing a computer to execute processing including the steps of:

when a point other than a center of a specified object domain present in an obtained image is indicated by a user, setting a first provisional domain containing the indicated point and a second provisional domain exclusive of the first provisional domain, calculating separability of the first and second provisional domains based on image information of the first and second provisional domains, and based on a separability series obtained by incrementally changing the sizes and positions, or the sizes, positions, and shapes of provisional domains, extracting a partial image from the obtained image so as to contain the specified image domain, which is present in the obtained image, and which contains the indicated point.

47. A computer-readable recording medium which records an image extraction processing program for causing a computer to execute processing including the steps of:

when a stored obtained image is a moving image made up of a series of electronic images, reading out a given image of the series of stored images as an image in question;

extracting a partial image from the image in question so as to contain a specified object domain present in the image in question;

setting in the image in question a pair of windows made up of two provisional domains; and

using the pair of windows made up of the two provisional domains as initial windows for the next image in the series of obtained images, changing the sizes and positions of the provisional domains to obtain various image information, and using the image information to determine the most suitable pair of windows.

48. A computer-readable recording which records an image extraction processing program for causing a computer to execute processing including the steps of:

when an obtained image stored in memory means is a moving image made up of a series of electronic images, and when a given image of the stored series of images is read out as an image in question and a point in the center of a specified object domain present in the image in question is indicated, setting a first provisional domain containing the indicated point and a second provisional domain exclusive of the first provisional domain;

calculating separability of the specified object domain and the first and second provisional domains based on image information of a pair of windows made up of the first and second provisional domains;

based on a separability series obtained by incrementally changing the sizes of the first and second provisional domains, extracting a partial image from the image in question such that the specified object domain, containing the indicated point at its own center, falls in the center of the partial image;

setting in the image in question a pair of windows made up of two provisional domains; and

using the pair of windows made up of the two provisional domains as initial windows for the next image in the series of obtained images, changing size and position of the provisional domains to obtain various image information, and using the image information to determine the most suitable pair of windows.

49. A computer-readable recording medium which records an image extraction processing program for causing a computer to execute processing including the steps of:

when a stored obtained image is a moving image made up of a series of electronic images, and when a given image of the stored series of images is read out as an image in question and a given point in a specified object domain present in the image in question is indicated, setting a first provisional domain containing the indicated point and a second provisional domain exclusive of the first provisional domain;

calculating separability of the specified object domain and the first and second provisional domains based on image information of the first and second provisional domains;

based on a separability series obtained by incrementally changing the sizes and positions of the first and second provisional domains, extracting a partial image from the image in question such that the specified object domain, which is present in the obtained image and which contains the indicated point, falls in the center of the partial image;

setting in the image in question a pair of windows made up of two provisional domains; and

using the pair of windows made up of the two provisional domains as initial windows for the next image in the series of obtained images, changing the sizes of the provisional domains to obtain various image information, and using the image information to determine the most suitable pair of windows.

50. An image processing method comprising:

(a) setting a window comprising first and second mutually exclusive provisional domains in a base image;

(b) calculating separability of the first provisional domain and the second provisional domain based on image information contained in the respective domains;

(c) increasing the size of the window;

(d) repeating steps (b) and (c) until the size of the window exceeds a predetermined window size;

(e) determining whether the window contains a specified object domain based on the calculated separabilities; and

(f) extracting a partial image containing the specified object domain from the base image if step (e) determines that the window contains a specified object domain.

51. The image processing method set forth in claim 50, wherein the predetermined window size is based on the extent of the base image.

52. The image processing method as set forth in claim 50, wherein the second provisional domain surrounds the first provisional domain, an inner boundary of the second provisional domain contacting an outer boundary of the first provisional domain.

53. The image processing method as set forth in claim 50, wherein steps (a)-(f) are repeated for each pixel of the base image.

54. The image processing method as set forth in claim 50, wherein steps (a)-(f) are performed for pixels of the base image specified using a coordinate indicating device.

55. The image processing method as set forth in claim 50, further comprising:

(g) setting a further provisional domain in part of the partial image extracted in step (f);

(h) obtaining image information of the further provisional domain; and

(i) based on the image information, extracting a domain, smaller than the partial image, which contains the specified object domain.

56. The image processing method as set forth in claim 55, wherein the image information is pixel distribution information.

57. A computer-readable medium having computer-executable instructions for performing the steps recited in claim 55.

58. A computer-readable medium having computer-executable instructions for performing the steps recited in claim 50.