Visual attention system

• US 6,934,415 B2
• Filed: 10/16/2001
• Issued: 08/23/2005
• Est. Priority Date: 02/17/2000
• Status: Active Grant

• Current Assignee: British Telecommunications Public Limited Company
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Stentiford, Frederick W. M.
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US09/977,263
• Publication Number:
  US 20020081033A1
• Time in Patent Office: 1,407 Days
• Field of Search: 382/218, 382/219, 382/100, 382/205, 382/209, 382/181
• US Class Current: 382/205
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

The most significant features in visual scenes, is identified without prior training, by measuring the difficulty in finding similarities between neighbourhoods in the scene. Pixels in an area that is similar to much of the rest of the scene score low measures of visual attention. On the other hand a region that possesses many dissimilarities with other parts of the image will attract a high measure of visual attention. A trial and error process is used to find dissimilarities between parts of the image and does not require prior knowledge of the nature of the anomalies that may be present. The use of processing dependencies between pixels avoided while yet providing a straightforward parallel implementation for each pixel. Such techniques are of wide application in searching for anomalous patterns in health screening, quality control processes and in analysis of visual ergonomics for assessing the visibility of signs and advertisements. A measure of significant features can be provided to an image processor in order to provide variable rate image compression.

Claims

1. A method of processing visual image, for identifying areas of visual attention, comprising:

storing an image as an array of pixels, each pixel having a value;

selecting test pixels from the array;

for each text pixel;

selecting one or more neighbor groups of pixels neighboring the test pixel;

selecting comparison pixels from the array;

identifying a group of pixels neighboring a selected comparison pixel having the same respective positional relationships to the comparison pixel as a selected neighbor group of pixels has to the test pixel;

comparing the values of the selected neighbor group with the values of the identified group in accordance with a predetermined match criterion, and

generating a measure of visual attention for each test pixel, in dependence upon the number of comparisons made for that test pixel for which the comparison results in a mismatch.

2. A method as in claim 1 wherein, for each test pixel, if one or more of the selected pixels neighboring the test pixel has a value not substantially similar to the value of the corresponding pixel neighboring the comparison pixel, an anomaly value for that test pixel is incremented, and this process is repeated using further comparison pixels with the same test pixel until a comparison pixel is selected for which all the selected pixels have a value substantially similar to the corresponding pixel neighboring the text pixel, in which case a further neighbor group is selected and the process repeated.

3. A method as in claim 1, wherein a plurality of test pixels are analyzed concurrently.

4. A method as in claim 1, wherein a plurality of comparison pixels are compared with a given test pixel concurrently.

5. A method as in claim 1, wherein the value is a three-element vector representative of a color.

6. A method as in claim 1, wherein in addition to neighbor groups, further variable search parameters are selected.

7. A method as in claim 6, wherein the further variable search parameters include a threshold value for the determination of whether two pixel values are substantially similar.

8. A method as in claim 1, the method including storing values for search parameters for which a large number of mismatches has been generated, and selecting, for subsequent test pixels, the same search parameters.

9. A method as in claim 1, wherein the principal subject in a visual scene is identified by identification of the region containing pixels having the largest number of mismatches.

10. A method as in claim 1, wherein a measure of visual attention afforded to a given object in a visual scene is determined by comparison of anomaly values generated for the pixels representing that object with anomaly values generated for other parts of the scene.

11. A method of image compression comprising:

processing an image to locate areas of visual attention using the method of claim 1; and

coding the image according to the measures of visual attention such that areas of high visual attention are coded with more accuracy than areas of the image with low visual attention.

12. A method of image compression as in claim 11 in which the measures of visual attention are used to select a level of quantization for coding the image.

14. A computer programmed to perform the method of claim 1.

15. A computer program product stored on a computer readable medium, directly loadable into the internal memory of a digital computer, comprising software code portions for performing the steps of claim 1 when said product is run on a computer.

21. A method as in claim 1 further comprising:

defining a subset of said pixel array and generating said measure of visual attention in respect of test pixels in said subset.

22. A method as in claim 21, further comprising:

identifying one or more of said test pixels for which said measure is indicative of a large number of mismatches relative to the measures generated for others of said test pixels; and

generating said measures for further test pixels in the vicinity of said one or more identified test pixels.

24. A method as in claim 1 further including:

(a) identifying ones of said positional relationship which give rise to a number of consecutive mismatches which exceeds a threshold;

(b) storing a definition of each such identified relationship; and

(c) utilizing the stored definitions for the processing of further test pixels.

13. Apparatus for processing a visual image, for locating areas of visual attention, said apparatus comprising:

means for storing an image as an array of pixels, each pixel having a value;

means for selecting test pixels from the array,

means for selecting neighbor groups of pixels neighboring the test pixel;

means for selecting comparison pixels from the array;

means for identifying that group of pixels neighboring a selected comparison pixel whose pixels have the same respective positional relationships to the comparison pixel as a selected neighbor group of pixels has to the test pixel;

means for comparing the values of the selected neighbor group with the values of the identified group in accordance with a predetermined match criterion,

means for generating a measure of visual attention for each test pixel, in dependence upon the number of comparisons which identify a non-matching group.

16. A computer program product stored on a computer usable medium, said product comprising:

at least one computer readable program configured for causing a computer to store an image as an array of pixels, each pixel having a value;

at least one computer readable program configured for causing the computer to select test pixels from the array,

at least one computer readable program configured for causing the computer to select, for each test pixel, neighbor groups of pixels neighboring the test pixel;

at least one computer readable program configured for causing the computer to select comparison pixels from the array;

at least one computer readable program configured for causing the computer to identify the group of pixels neighboring a selected comparison pixel having the same respective positional relationships to the comparison pixel as a selected neighbor group of pixels has to the test pixel;

at least one computer readable program configured for causing the computer to compare the values of the selected neighbor group with the values of the identified group in accordance with a predetermined match criterion; and

at least one computer readable program configured for causing the computer to generate a measure of visual attention for each test pixel, in dependence upon the number of comparisons in which the comparison result in a mismatch.

17. A method of processing a sequence of visual images, for identifying areas of visual attention, said method comprising:

storing a sequence of images as a multi dimensional array of pixels, each pixel having a value;

selecting test pixels from the array;

for each test pixel, selecting one or more neighbor groups of pixels neighboring the test pixel;

selecting comparison pixels from the array;

identifying a group of pixels neighboring a selected comparison pixel having the same respective positional relationships to the comparison pixel as a selected neighbor group of pixels has to the test pixel;

comparing the values of the selected neighbor group with the values of the identified group in accordance with a predetermined match criterion; and

generating a measure of visual attention for each test pixel, in dependence upon the number of comparisons made for that test pixel for which the comparison results in a mismatch.

18. A method of processing a moving image, for identifying areas of visual attention, said method comprising:

storing successive pictures of the moving image as respective arrays of picture element values;

defining a test group of picture elements comprising a first test picture element and a second test picture element having a spatial offset and a temporal offset from the first;

defining a comparison group of picture elements comprising a first comparison picture element having a spatial and temporal offset from the first test picture element and a second comparison picture element having a spatial and temporal offset from the first comparison picture element equal respectively to the spatial and temporal offset of the second test picture element from the first test picture element;

comparing the picture element values of the first and second test picture elements with the picture element values of the first and second comparison picture elements respectively, in accordance with a predetermined match criterion;

defining further such comparison groups and comparing the test picture elements with those of the further comparison groups;

generating a visual attention measure for the first test picture element in dependence on the number of comparisons made for it in which the comparison results in a mismatch.

19. A method as in claim 18 further including:

defining at least one further comparison group comprising a first further comparison element having the same spatial offset from the first test picture element as has the first comparison picture element, but a different temporal offset, and a second further comparison picture element having the same offset from the first further comparison picture element as the second test picture element has from the first test picture element, and

wherein the comparing step includes comparing value of the first and second further comparison picture elements with the values of the first and second test picture elements respectively.

20. A method as in claim 18 in which the test group and each comparison group includes at least one additional picture element.

23. A method of analyzing a pattern represented by an ordered set of elements each having a value comprising, in respect of at least some of said elements:

selecting a group of test elements comprising at least two elements of the ordered set;

selecting a group of comparison elements comprising at least two elements of the ordered set, wherein the comparison group has the same number of elements as the test group and wherein the elements of the comparison group have relative to one another the same positions in the ordered set as have the elements of the test group;

comparing the value of each element of the test group with the value of the correspondingly positioned element of the comparison group in accordance with a predetermined match criterion to produce a decision that the test group matches or does not match the comparison group;

selecting further said comparison groups and comparing them with the test group; and

generating for the test group a distinctiveness measure a function of the number of comparison groups for which the decision is that the test group does not match the comparison group.