Visual attention system

• US 20020081033A1
• Filed: 10/16/2001
• Published: 06/27/2002
• Est. Priority Date: 02/17/2000
• Status: Active Grant

• Current Assignee: British Telecommunications Public Limited Company
• Original Assignee: British Telecommunications Public Limited Company
• Inventors: Stentiford, Frederick W.M.
• Granted Patent: US 6,934,415 B2
• Time in Patent Office: Days
• US Class Current: 382/218
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

This invention identifies the most significant features in visual scenes, without prior training, by measuring the difficulty in finding similarities between neighbourhoods in the scene. Pixels in an area that is similar to much of the rest of the scene score low measures of visual attention. On the other hand a region that possesses many dissimilarities with other parts of the image will attract a high measure of visual attention.

The invention makes use of a trial and error process to find dissimilarities between parts of the image and does not require prior knowledge of the nature of the anomalies that may be present. The method avoids the use of processing dependencies between pixels and is capable of a straightforward parallel implementation for each pixel.

The invention is of wide application in searching for anomalous patterns in health screening, quality control processes and in analysis of visual ergonomics for assessing the visibility of signs and advertisements.

The invention provides a measure of significant features to an image processor in order to provide variable rate image compression.

Claims

1. A method of processing a visual image, for identifying areas of visual attention, comprising the steps of:

storing an image as an array of pixels, each pixel having a value;

selecting test pixels from the array,

for each test pixel,

selecting one or more neighbour groups of pixels neighbouring the test pixel;

selecting comparison pixels from the array;

identifying a group of pixels neighbouring a selected comparison pixel having the same respective positional relationships to the comparison pixel as a selected neighbour group of pixels has to the test pixel;

comparing the values of the selected neighbour group with the values of the identified group in accordance with a predetermined match criterion, and

generating a measure of visual attention for each test pixel, in dependence upon the number of comparisons made for that test pixel for which the comparison results in a mismatch.

2. A method according to claim 1 wherein, for each test pixel, if one or more of the selected pixels neighbouring the test pixel has a value not substantially similar to the value of the corresponding pixel neighbouring the comparison pixel, an anomaly value is incremented, and the process is repeated using further comparison pixels with the same test pixel until a comparison pixel is selected for which all the selected pixels have a value substantially similar to the corresponding pixel neighbouring the test pixel, in which case a further neighbour group is selected and the process repeated.

3. A method according to claim 1 or 2, wherein a plurality of test pixels are analysed concurrently.

4. A method according to claim 1, 2 or 3 wherein a plurality of comparison pixels are compared with a given test pixel concurrently.

5. A method according to claim 1, 2, 3 or 4, wherein the value is a three-element vector representative of a colour.

6. A method according to claim 1, 2, 3, 4 or 5 wherein in addition to neighbour groups, further variable search parameters are selected.

7. A method according to claim 6, wherein the further variable search parameters include a threshold value for the determination of whether two pixel values are substantially similar.

8. A method according to claim 1, 2, 3, 4, 5, 6 or 7, the method including the step of storing values for search parameters for which a high anomaly value has been generated, and selecting, for subsequent test pixels, the same search parameters.

9. A method according to claim 1, 2, 3, 4, 5, 6, 7 or 8, wherein the principal subject in a visual scene is identified by identification of the region containing pixels having the greatest anomaly values

10. A method according to claim 1, 2, 3, 4, 5, 6, 7 or 8, wherein a measure of visual attention afforded to a given object in a visual scene is determined by comparison of the anomaly values generated for the pixels representing that object with the anomaly values generated for other parts of the scene.

11. A method of image compression comprising

processing an image to locate areas of visual attention using the method of any one of the preceding claims;

coding the image according to the measures of visual attention such that areas of high visual attention are coded with more accuracy than areas of the image with low visual attention.

12. A method of image compression according to claim 11 in which the measures of visual attention are used to select a level of quantization for coding the image.

14. A computer programmed to perform the method of any of claims 1 to 13.

15. A computer program product directly loadable into the internal memory of a digital computer, comprising software code portions for performing the steps of any of claims 1 to 13 when said product is run on a computer.

19. A method according to claim 18 including:

defining at least one further comparison group comprising a first further comparison element having the same spatial offset from the first test picture element as has the first comparison picture element, but a different temporal offset, and a second further comparison picture element having the same offset from the first further comparison picture element as the second test picture element has from the first test picture element, and the comparing step includes comparing values of the first and second further comparison picture elements with the values of the first and second test picture elements respectively.

20. A method according to claim 18 or 19 in which the test group and the or each comparison group includes at least one additional picture element.

21. A method according to claim 1 comprising defining a subset of said pixel array and generating said measure of visual attention in respect of test pixels in said subset.

22. A method according to claim 21, comprising the further steps of identifying one or more of said test pixels for which said measure is indicative of a large number of mismatches relative to the measures generated for others of said test pixels and generating said measures for further test pixels in the vicinity of said one or more identified test pixels.

24. A method according to claim 1 including the further step of

(a) identifying ones of said positional relationships which give rise to a number of consecutive mismatches which exceeds a threshold;

(b) storing a definition of each such identified relationship; and

(c) utilising the stored definitions for the processing of further test pixels.

13. Apparatus for processing a visual image, for locating areas of visual attention, comprising

means for storing an image as an array of pixels, each pixel having a value;

means for selecting test pixels from the array,

means for selecting neighbour groups of pixels neighbouring the test pixel;

means for selecting comparison pixels from the array;

means for identifying that group of pixels neighbouring a selected comparison pixel whose pixels have the same respective positional relationships to the comparison pixel as a selected neighbour group of pixels has to the test pixel;

means for comparing the values of the selected neighbour group with the values of the identified group in accordance with a predetermined match criterion,

means for generating a measure of visual attention for each test pixel, in dependence upon the number of comparisons which identify a non-matching group.

16. A computer program product stored on a computer usable medium, comprising:

computer-readable program means for causing a computer to store an image as an array of pixels, each pixel having a value;

computer readable program means for causing the computer to select test pixels from the array,

computer readable program means for causing the computer to select, for each test pixel, neighbour groups of pixels neighbouring the test pixel;

computer readable program means for causing the computer to select comparison pixels from the array;

computer readable program means for causing the computer to identify the group of pixels neighbouring a selected comparison pixel having the same respective positional relationships to the comparison pixel as a selected neighbour group of pixels has to the test pixel;

computer readable program means for causing the computer to compare the values of the selected neighbour group with the values of the identified group in accordance with a predetermined match criterion,

computer readable program means for causing the computer to generate a measure of visual attention for each test pixel, in dependence upon the number of comparisons in which the comparison result in a mismatch.

17. A method of processing a sequence of visual images, for identifying areas of visual attention, comprising the steps of:

storing a sequence of images as a multi dimensional array of pixels, each pixel having a value;

selecting test pixels from the array,

for each test pixel, selecting one or more neighbour groups of pixels neighbouring the test pixel;

selecting comparison pixels from the array;

identifying a group of pixels neighbouring a selected comparison pixel having the same respective positional relationships to the comparison pixel as a selected neighbour group of pixels has to the test pixel;

comparing the values of the selected neighbour group with the values of the identified group in accordance with a predetermined match criterion,

generating a measure of visual attention for each test pixel, in dependence upon the number of comparisons made for that test pixel for which the comparison results in a mismatch.

18. A method of processing a moving image, for identifying areas of visual attention, comprising:

storing successive pictures of the moving image as respective arrays of picture element values;

defining a test group of picture elements comprising a first test picture element and a second test picture element having a spatial offset and a temporal offset from the first;

defining a comparison group of picture elements comprising a first comparison picture element having a spatial and temporal offset from the first test picture element and a second comparison picture element having a spatial and temporal offset from the first comparison picture element equal respectively to the spatial and temporal offset of the second test picture element from the first test picture element;

comparing the picture element values of the first and second test picture elements with the picture element values of the first and second comparison picture elements respectively, in accordance with a predetermined match criterion;

defining further such comparison groups and comparing the test pictures element with those of the further comparison groups;

generating a visual attention measure for the first test picture element in dependence on the number of comparisons made for it which the comparison results in a mismatch.

23. A method of analysing a pattern represented by an ordered set of elements each having a value comprising, in respect of at least some of said elements:

selecting a group of test elements comprising at least two elements of the ordered set;

selecting a group of comparison elements comprising at least two elements of the ordered set, wherein the comparison group has the same number of elements as the test group and wherein the elements of the comparison group have relative to one another the same positions in the ordered set as have the elements of the test group;

comparing the value of each element of the test group with the value of the correspondingly positioned element of the comparison group in accordance with a predetermined match criterion to produce a decision that the test group matches or does not match the comparison group;

selecting further said comparison groups and comparing them with the test group;

generating a distinctiveness measure as a function of the number of comparisons for which the comparison indicates a mismatch.