Method and apparatus for detecting and/or tracking one or more colour regions in an image or sequence of images

• US 20040017938A1
• Filed: 04/08/2003
• Published: 01/29/2004
• Est. Priority Date: 04/08/2002
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Bober, Miroslaw, Paschalakis, Stavros, Cooper, James
• Granted Patent: US 7,236,629 B2
• Time in Patent Office: Days
• US Class Current: 382/171
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 1/56 Processing of colour picture signals H04N1/52 takes precedence
• CPC Class Codes: H04N 9/643 Hue control means, e.g. flesh tone control

Abstract

A method and apparatus for detecting and/or tracking one or more colour regions in an image or sequence of images A method of detecting a region having predetermined colour characteristics in an image comprises transforming colour values of pixels in the image from a first colour space to a second colour space, using the colour values in the second colour space to determine probability values expressing a match between pixels and the predetermined colour characteristics, where the probability values range over a multiplicity of values, using said probability values to identify pixels at least approximating to said predetermined colour characteristics, grouping pixels which at least approximate to said predetermined colour characteristics, and extracting information about each group, wherein pixels are weighted according to the respective multiplicity of probability values, and the weightings are used when grouping the pixels and/or when extracting information about a group.

Claims

1. A method of detecting at least one region having predetermined colour characteristics in an image, the method comprising

transforming colour values of pixels in the image from a first colour space to a second colour space;

using the colour values in the second colour space to determine probability values expressing a match between pixels and the predetermined colour characteristics, where the probability values range over a multiplicity of values;

using said probability values to identify pixels at least approximating to said predetermined colour characteristics;

grouping pixels which at least approximate to said predetermined colour characteristics;

and extracting information about each group;

wherein pixels are weighted according to the respective multiplicity of probability values, and the weightings are used when grouping the pixels and/or when extracting information about a group.

2. A method as claimed in claim 1 wherein the extracted information includes one or more of the following:

power based on the sum of the probabilities of pixels in a group, group area, group size, group location, group centroid, group width, group height, group boundary, group bounding box width, group bounding box height.

3. A method as claimed in any preceding claim comprising downsampling the image at any stage during its processing.

4. A method as claimed in any preceding claim wherein the second colour space enhances said predetermined colour characteristics and achieves robustness to illumination changes.

5. A method as claimed in any preceding claim wherein said first colour space is a linear colour space and the second colour space is a non-linear colour space.

6. A method as claimed in any preceding claim wherein the second colour space is a logarithmic colour space.

7. A method as claimed in any preceding claim comprising determining the probability that a pixel has predetermined colour characteristics by projecting its colour value onto a predetermined probability model, said model comprising an entire histogram or one or more regions of a histogram of the sought colour characteristics.

8. A method as claimed in claim 7 wherein the probability model is calibrated any number of times, either by replacing or merging with the existing probability model, using a sample colour region selected by the user.

9. A method as claimed in any preceding claim wherein the group which is most central in the image and/or has the highest sum of probabilities is selected for display.

10. A method as claimed in any preceding claim wherein the image is in a sequence of images, and at least one group of pixels having the predetermined characteristics is determined in several images in the sequence.

11. A method as claimed in claim 10 wherein said pixel groups are tracked and/or displayed through the sequence of images.

12. A method as claimed in claim 10 or claim 11 wherein pixels groups between different images are matched using a distance and/or size function.

13. A method as claimed in any of claims 10 to 12 wherein movements of tracked groups are averaged over a predetermined number of frames to produce average positions.

14. A method as claimed in claim 13 wherein the number of frames used for the averaging of the movement can be determined by the user within predefined limits.

15. A method as claimed in claim 13 or claim 14 wherein a group movement larger than a predefined limit reset the averaging for that group.

16. A method as claimed in any of claims 10 to 15 wherein each tracked group is assigned an identification tag to enable the tracked groups to be cycled or selected.

17. A method as claimed in any of claims 10 to 16 wherein the user selects the groups for tracking and/or display.

18. A method as claimed in any of claims 10 to 16 wherein the group which is most central in the image and/or has the highest sum of probabilities is selected for tracking and/or display.

19. A method as claimed in claim 18 wherein the tracked and/or displayed group switches to the group having the highest sum of probabilities when the tracked group does not have the highest sum of probabilities for a predetermined number of successive images.

20. A method as claimed in any preceding claim wherein said colour characteristics correspond to human skin colour tones.

21. A method as claimed in any preceding claim wherein the region is a human face.

22. A method as claimed in any of claims 1 to 20 wherein the region is a human hand.

23. A method as claimed in claim 7 or any preceding claim dependent on claim 7 wherein the probability model is calibrated by extracting a region of an image of the subject.

24. A method as claimed in any preceding claim wherein zooming is performed according to the area of a said group of pixels in the image.

25. A method as claimed in claim 24 wherein there is a set of discrete zoom levels.

26. A method as claimed in claim 25 comprising using a temporal hysteresis function to smooth changes between discrete zoom levels.

27. A method of operating a device including image capture means, the method comprising identifying groups of pixels in the captured image having predetermined colour characteristics using a method as claimed in any preceding claim.

28. A method as claimed in claim 27 comprising extracting one or more regions containing said groups of pixels for further processing.

29. A method as claimed in claim 28 wherein the further processing includes display on image display means.

30. A method as claimed in any of claims 27 to 29 wherein the device is an image transmission system.

31. A computer program for executing a method as claimed in any preceding claim.

32. A data storage medium storing a computer program as claimed in claim 31.

33. Apparatus adapted to implement a method as claimed in any preceding claim.

34. Apparatus as claimed in claim 33 comprising data storage means and image data processing means.

35. A transmitter for a communication system comprising an apparatus as claimed in claim 33 or claim 34.

36. Apparatus as claimed in any of claims 33 to 35 which is a mobile videophone.