Method and apparatus for detecting and/or tracking one or more colour regions in an image or sequence of images

• US 7,236,629 B2
• Filed: 04/08/2003
• Issued: 06/26/2007
• Est. Priority Date: 04/08/2002
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Bober, Miroslaw, Paschalakis, Stavros, Cooper, James
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): Bayat; Ali
• Application Number:
  US10/408,316
• Publication Number:
  US 20040017938A1
• Time in Patent Office: 1,540 Days
• Field of Search: 382162-168, 382171-172, 382/190, 382/272, 382/274, 382/276, 358518-523, 358/1.9, 348/453
• US Class Current: 382/171
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 1/56 Processing of colour picture signals H04N1/52 takes precedence
• CPC Class Codes: H04N 9/643 Hue control means, e.g. flesh tone control

Abstract

A method of detecting a region having predetermined colour characteristics in an image comprises transforming colour values of pixels in the image from a first colour space to a second colour space, using the colour values in the second colour space to determine probability values expressing a match between pixels and the predetermined colour characteristics, where the probability values range over a multiplicity of values, using said probability values to identify pixels at least approximating to said predetermined colour characteristics, grouping pixels which at least approximate to said predetermined colour characteristics, and extracting information about each group, wherein pixels are weighted according to the respective multiplicity of probability values, and the weightings are used when grouping the pixels and/or when extracting information about a group.

Claims

1. A method of detecting at least one region having predetermined colour characteristics in an image, the method comprising:

transforming colour values of pixels in the image from a first colour space to a second colour space;

using the colour values in the second colour space to determine probability values expressing a match between pixels and the predetermined colour characteristics, where the probability values ranges over a multiplicity of values;

using said probability values to identify pixels at least approximating to said predetermined colour characteristics;

grouping pixels which at least approximate to said predetermined colour characteristics; and

extracting information about each group;

wherein pixels are weighted according to the respective multiplicity of probability values, and the weightings are used when grouping the pixels and/or when extracting information about a group.

2. A method as claimed in claim 1, wherein the extracted information includes one or more of the following:

power based on the sum of the probabilities of pixels in a group, group area, group size, group location, group centroid, group width, group height, group boundary, group bounding box width, and group bounding box height.

3. A method as claimed in claim 1, further comprising:

downsampling the image at any stage during its processing.

4. A method as claimed in claim 1, wherein the second colour space enhances said predetermined colour characteristics and achieves robustness to illumination changes.

5. A method as claimed in claim 1, wherein said first colour space is a linear colour space and the second colour space is a non-linear colour space.

6. A method as claimed in claim 1, wherein the second colour space is a logarithmic colour space.

7. A method as claimed in claim 1, using the colour values in the second colour space to determine probability values comprises:

determining the probability that a pixel has predetermined colour characteristics by projecting its colour value onto a predetermined probability model, said model including an entire histogram or one or more regions of a histogram of the sought colour characteristics.

8. A method as claimed in claim 7, wherein the probability model is calibrated any number of times, either by replacing or merging with the existing probability model, using a sample colour region selected by a user.

9. A method as claimed in claim 7, wherein the probability model is calibrated by extracting a region of an image of a subject.

10. A method as claimed in claim 1, wherein the group of pixels which is most central in the image or has the highest sum of probabilities is selected for display.

11. A method as claimed in claim 1, wherein the image is in a sequence of images, and at least one group of pixels having the predetermined colour characteristics is determined in several images in the sequence.

12. A method as claimed in claim 11, wherein said at least one group of pixels is tracked or displayed though the sequence of images.

13. A method as claimed in claim 12, wherein movements of tracked groups of pixels are averaged over a predetermined number of frames to produce average positions.

14. A method as claimed in claim 13, wherein the number of frames used for the averaging of the movement can be determined by a user within predefined limits.

15. A method as claimed in claim 13, wherein a group movement larger than a predefined limit resets the averaging for that group.

16. A method as claimed in claim 12, wherein each tracked group is assigned an identification tag to enable the tracked groups to be cycled or selected.

17. A method as claimed in claim 12, wherein a user selects the groups for tracking or display.

18. A method as claimed in claim 12, wherein the group which is most central in the image or has the highest sum of probabilities is selected for tracking or display.

19. A method as claimed in claim 18, wherein the tracked or displayed group switches to the group having the highest sum of probabilities when the tracked or displayed group does not have the highest sum of probabilities for a predetermined number of successive images.

20. A method as claimed in claim 11, wherein groups of pixels in different images are matched using a distance or size function.

21. A method as claimed in claim 1, wherein said colour characteristics correspond to human skin colour tones.

22. A method as claimed in claim 1, wherein the at least one region corresponds to a human face.

23. A method as claimed in claim 1, wherein the at least one region corresponds to a human hand.

24. A method as claimed in claim 1, wherein zooming is performed according to the area of a group of pixels in the image.

25. A method as claimed in claim 24, wherein there is a set of discrete zoom levels.

26. A method as claimed in claim 25, further comprising:

using a temporal hysteresis function to smooth changes between discrete zoom levels.

27. A method of operating a device including an image capture unit, the method comprising:

identifying groups of pixels in a captured image having predetermined colour characteristics using a method as claimed in claim 1.

28. A method as claimed in claim 27, further, comprising:

extracting one or more regions containing said groups of pixels for further processing.

29. A method as claimed in claim 28, wherein the further processing includes displaying the extracted regions on an image display unit.

30. A method as claimed in claim 27, wherein the device is utilized in an image transmission system.

31. A computer program stored on a computer readable media for executing a method as claimed in claim 1.

32. An apparatus for detecting at least one region having predetermined colour characteristics in an image, the apparatus comprising:

a signal processor configured to

transform colour values of pixels in the image from a first colour space to a second colour space;

use the colour values in the second colour space to determine probability values expressing a match between pixels and the predetermined colour characteristics, where the probability values ranges over a multiplicity of values;

use said probability values to identify pixels at least approximating to said predetermined colour characteristics;

group pixels which at least approximate to said predetermined colour characteristics; and

extract information about each group;

wherein pixels are weighted according to the respective multiplicity of probability values, and the weightings are used when grouping the pixels or when extracting information about a group.

33. An apparatus as claimed in claim 32, further comprising:

a data storage unit; and

an image data processing unit.

34. A transmitter for a communication system comprising an apparatus as claimed in claim 32 or claim 33.

35. An apparatus as claimed in claim 32, wherein the apparatus is a mobile videophone.

36. An apparatus for detecting at least one region having predetermined colour characteristics in an image, the apparatus comprising:

means for transforming colour values of pixels in the image from a first colour space to a second colour space;

means for using the colour values in the second colour space to determine probability values expressing a match between pixels and the predetermined colour characteristics, where the probability values ranges over a multiplicity of values;

means for using said probability values to identify pixels at least approximating to said predetermined colour characteristics;

means for grouping pixels which at least approximate to said predetermined colour characteristics; and

means for extracting information about each group;

wherein pixels are weighted according to the respective multiplicity of probability values, and the weightings are used when grouping the pixels or when extracting information about a group.