SYSTEM AND METHOD FOR IMPROVED IMAGE ANALYSIS THROUGH GAZE DATA FEEDBACK

• US 20110206283A1
• Filed: 02/23/2010
• Published: 08/25/2011
• Est. Priority Date: 02/23/2010
• Status: Active Grant

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: QUARFORDT, PERNILLA, Biehl, Jacob, Golovchinsky, Gene
• Granted Patent: US 8,311,279 B2
• Time in Patent Office: Days
• US Class Current: 382/220
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

The image analysis system includes a processor and memory and displays an image to a first user. The image analysis system tracks gaze of the first user and collects initial gaze data for the first user. The initial gaze data includes a plurality of gaze points. The image analysis system identifies one or more ignored regions of the image based on a distribution of the gaze data within the image; and displays at least a first subset of the image. The first subset of the image is selected so as to include a respective ignored region of the one or more ignored regions and the first subset of the image is displayed in a manner that draws attention to the respective ignored region. In some embodiments, the ignored region is visually emphasized within the image. In some embodiments, only the first subset of the image is displayed.

Claims

1. A method of enhancing image analysis comprising:

displaying an image to a first user;

tracking gaze of the first user;

collecting initial gaze data for the first user, wherein the initial gaze data includes a plurality of gaze points;

identifying one or more ignored regions of the image based on a distribution of the gaze data within the image; and

displaying at least a first subset of the image, wherein the first subset of the image is selected so as to include a respective ignored region of the one or more ignored regions and the first subset of the image is displayed in a manner that draws attention to the respective ignored region.

2. The method of claim 1, wherein only a subset of the image including the ignored region is displayed.

3. The method of claim 1, displaying the first subset of the image includes visually emphasizing the respective ignored region.

4. The method of claim 1, wherein:

identifying the one or more ignored regions includes;

defining a plurality of gaze groupings of the gaze data, wherein each gaze grouping includes one or more of the gaze points, and has a respective location within the image; and

identifying the one or more ignored regions based on the respective locations of the plurality of gaze groupings.

5. The method of claim 4, wherein defining a respective gaze grouping includes:

identifying a subset of the gaze points that are proximate to each other; and

defining a region that includes the plurality of gaze points as the respective gaze grouping.

6. The method of claim 4, wherein defining a respective gaze grouping includes:

defining a plurality of gaze fixations; and

combining a respective gaze fixation with one or more overlapping gaze fixations to define the respective gaze grouping.

7. The method of claim 6, wherein identifying the one or more ignored regions based on the respective locations of the plurality of gaze groupings includes identifying regions of the image that do not include gaze groupings with a combined fixation duration above a threshold.

8. The method of claim 4, wherein identifying the one or more ignored regions based on the respective locations of the plurality of gaze groupings includes identifying regions of the image that do not include gaze groupings with a number of gaze points above a threshold.

9. The method of claim 1, wherein the image is divided into a plurality of predetermined regularly spaced regions and the ignored regions are respective ones of the predetermined regularly spaced regions that include distributions of gaze data below a threshold.

10. The method of claim 1:

further comprising defining a plurality of gaze groupings of the gaze data, wherein each gaze grouping includes one or more of the gaze points, and has a respective location within the image; and

identifying the one or more ignored regions based on the respective locations of the plurality of gaze groupings, wherein the ignored regions are dynamically defined based on the gaze groupings and are defined so as to be non-overlapping regions that include distributions of gaze data below a threshold.

11. The method of claim 1, wherein the ignored regions are defined so that a respective ignored region intersects with one or more neighboring regions.

12. The method of claim 1, wherein the first subset of the image includes a plurality of ignored regions.

13. The method of claim 1, further comprising:

identifying, based on the initial gaze data, a plurality of attended regions, wherein the attended regions include distributions of gaze data above a threshold; and

identifying a plurality of features that are common to one or more of the attended regions.

14. The method of claim 13, wherein:

the ignored regions are dynamically defined based on gaze groupings; and

,

a respective ignored region is defined so as to include a respective feature that is similar to one of the identified features, and so that the respective feature is proximate to a respective centroid of the respective ignored region.

15. The method of claim 13, further comprising:

identifying one or more of the ignored regions that include at least a subset of the identified features; and

prioritizing display of the one or more ignored regions based on the identified features.

16. The method of claim 15, wherein the prioritizing includes:

ranking the ignored regions in accordance with a first ranking, and the rank of a respective ignored region within the first ranking is determined based on one or more criteria selected from the list consisting of;

a likelihood of match of features in the respective ignored region to the plurality of identified features in the attended regions;

a geometrical relationship between the respective ignored region and the plurality of attended regions; and

a location of the respective ignored region within the image.

17. The method of claim 15, wherein the prioritizing includes:

ranking the ignored regions in accordance with a first ranking, and displaying the first subset of the image to the first user includes displaying an ignored region that is a top ranked ignored region in the first ranking.

18. The method of claim 17, wherein the first subset is displayed to the first user, and the method further comprises:

while displaying the first subset of the image to the first user;

tracking gaze of the first user; and

,

collecting updated gaze data, wherein the updated gaze data includes a plurality of gaze points;

updating one or more of the ignored regions of the image based on a distribution of the gaze data within the image;

generating a second ranking of the updated ignored regions based on the updated gaze data; and

,

displaying at least a second subset of the image, wherein the second subset of the image is selected so as to include a respective ignored region of the updated ignored regions that is a top ranked ignored region in the second ranking, and the second subset of the image is displayed in a manner that draws attention to the respective ignored region.

19. The method of claim 1, further comprising:

displaying the image to a second user;

tracking gaze of the second user; and

collecting additional gaze data for the second user, wherein the additional gaze data includes a plurality of gaze points;

wherein the one or more ignored regions are identified based at least in part on a distribution of the additional gaze data within the image.

20. The method of claim 19, wherein identifying the one or more ignored regions includes evaluating the respective distributions of gaze data for a plurality of respective users;

and in response to determining that the respective distribution of gaze data for each of the respective users is below an individual gaze threshold for a respective region, identifying the respective region as an ignored region.

21. The method of claim 19, wherein identifying one or more ignored regions includes:

determining a gaze difference between the distribution of gaze data from the first user and the distribution of gaze data from the second user for at least a respective region of the image;

comparing the gaze difference for the respective region to a discrepancy threshold; and

when the gaze difference for the respective region is above the discrepancy threshold, identifying the respective region as an ignored regions.

22. An image analysis system, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying an image to a first user;

tracking gaze of the first user;

collecting initial gaze data for the first user, wherein the initial gaze data includes a plurality of gaze points;

identifying one or more ignored regions of the image based on a distribution of the gaze data within the image; and

displaying at least a first subset of the image, wherein the first subset of the image is selected so as to include a respective ignored region of the one or more ignored regions and the first subset of the image is displayed in a manner that draws attention to the respective ignored region.

23. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an image analysis system with memory and one or more processors, cause the image analysis system to:

display an image to a first user;

track gaze of the first user;

collect initial gaze data for the first user, wherein the initial gaze data includes a plurality of gaze points;

identify one or more ignored regions of the image based on a distribution of the gaze data within the image; and

display at least a first subset of the image, wherein the first subset of the image is selected so as to include a respective ignored region of the one or more ignored regions and the first subset of the image is displayed in a manner that draws attention to the respective ignored region.