System and method for quantifying and mapping visual salience

• US 7,922,670 B2
• Filed: 02/23/2006
• Issued: 04/12/2011
• Est. Priority Date: 02/24/2005
• Status: Active Grant

• Current Assignee: MindsEye Labs Holdings, LLC
• Original Assignee: Ami Klin, Warren Jones
• Inventors: Klin, Ami, Jones, Warren
• Primary Examiner(s): Hindenburg; Max
• Assistant Examiner(s): NGUYEN, HUONG Q
• Application Number:
  US11/360,787
• Publication Number:
  US 20060189886A1
• Time in Patent Office: 1,874 Days
• Field of Search: 600/558, 351/209
• US Class Current: 600/558
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/4005 for evaluating the sensory system A61B5/12, A61B5/4058 take precedence
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence

Abstract

A system for quantifying and mapping visual salience to a visual stimulus, including a processor, software for receiving data indicative of a group of individual'"'"'s ocular responses to a visual stimulus, software for determining a distribution of visual resources at each of at least two times for each of at least a portion of the individuals. The system further including software for determining and quantifying a group distribution of visual resources at each of the at least two times and software for generating a display of the group'"'"'s distribution of visual resources to the visual stimulus.

Claims

1. A system for quantifying and mapping visual salience to a dynamic visual stimulus, comprising:

a processor;

software executing on said processor for receiving data indicative of a group of individual'"'"'s ocular responses while viewing a dynamic visual stimulus, the dynamic visual stimulus changing over time;

software executing on said processor for generating a distribution of visual resources for each individual in the group at each of a plurality of times during the viewing of the dynamic visual stimulus, said distribution of visual resources for each individual representing an allocation of each of at least two portions of the individual'"'"'s visual resources across each of at least two portions of the dynamic visual stimulus viewed by the particular individual at the particular time;

software executing on said processor for generating a distribution of the group'"'"'s visual resources at each of the plurality of times during the viewing of the dynamic visual stimulus by aggregating the distributions of each of the individuals at each of the plurality of times; and

software executing on said processor for generating a graphical display of the distribution of the group'"'"'s visual resources on the dynamic visual stimulus over time, the graphical display illustrating the allocation of each portion of the group'"'"'s visual resources across each portion of the dynamic visual stimulus viewed by the group at each of the plurality of times throughout the viewing of the dynamic visual stimulus, wherein the graphical display includes at least a portion of the dynamic visual stimulus.

2. The system according to claim 1,wherein said software for generating the distribution of visual resources generates each distribution based on the both the received data and an actual or mathematical approximation of a distribution of biological visual resources.

3. The system according to claim 1, wherein the graphical display of the group distribution of the group'"'"'s visual resources includes a display of an area of maximal salience on the dynamic visual stimulus at each of the different times.

4. The system according to claim 3, wherein the graphical display includes a three-dimensional extrusion of the areas of maximal salience over a time period including each of the plurality of times.

5. The system according to claim 4, wherein a decrease in area of maximal salience represents a heightened attention to the corresponding area of the visual stimulus.

6. The system according to claim 3, wherein the graphical display includes a three-dimensional attentional funnel comprising each of the areas of maximal salience.

7. The system according to claim 3, wherein the area of maximal salience at each of the plurality of times is based on an average of the group distribution of visual resources at each of the plurality of times.

8. The system according to claim 1, wherein the graphical display of the distribution of the group'"'"'s visual resources is a continuous graphical display changing over time.

9. The system according to claim 1, wherein the dynamic visual stimulus is video imagery, wherein each of the plurality of times corresponds to a different frame of the video imagery.

10. The system according to claim 1, wherein the dynamic visual stimulus is a live event.

11. The system according to claim 1, further comprising:

software executing on said processor for receiving stimulus data, wherein the graphical display of the distribution of the group'"'"'s visual resources includes at least a portion of the stimulus data.

32. The system according to claim 1, wherein the distribution of visual resources for each individual is a non-uniform distribution.

12. A system for quantifying and mapping visual salience to a dynamic visual stimulus, comprising:

a dynamic visual stimulus, the dynamic visual stimulus changing over time;

a processor;

software executing on said processor for receiving data indicative of at least one individual'"'"'s ocular response while viewing the dynamic visual stimulus;

software executing on said processor for determining at least one point of regard of the individual at each of a plurality of times based on the data;

software executing on said processor for generating a graphical display of the at least one individual'"'"'s points of regard to the dynamic visual stimulus over time;

software executing on said processor for comparing the at least one individual'"'"'s points of regard to the dynamic visual stimulus over time to a distribution of a group'"'"'s visual resources to the dynamic visual stimulus over time; and

wherein the graphical display includes a representation of the individual'"'"'s points of regard to the dynamic visual stimulus over time and a representation of an allocation of each of at least two portions of the group'"'"'s visual resources across each of at least two portions of the dynamic visual stimulus viewed by the group at each of the plurality of times throughout the viewing of the dynamic visual stimulus, wherein the display includes at least a portion of the dynamic visual stimulus.

13. The system according to claim 12, further comprising:

a device for tracking movement of at least one eye of the at least one individual.

14. The system according to claim 12,

wherein said software for receiving data receives data indicative of at least two individual'"'"'s ocular response to the dynamic visual stimulus,

wherein said software for determining at least one point of regard determines at least one point of regard for each of the at least two individuals.

15. The system according to claim 14,

wherein the display includes each of the at least two individual'"'"'s points of regard over the plurality of times.

16. A system for quantifying and mapping visual salience to a static visual stimulus, comprising:

a processor;

software executing on said processor for receiving data indicative of a group of individual'"'"'s ocular responses while viewing a static visual stimulus over a period of time;

software executing on said processor for generating a distribution of visual resources for each individual in the group at each of a plurality of times during the viewing of the static visual stimulus, said distribution of visual resources for each individual representing an allocation of each of at least two portions of the individual'"'"'s visual resources across each of at least two portions of the static visual stimulus viewed by the particular individual at the particular time;

software executing on said processor for generating a distribution of the group'"'"'s visual resources at each of the plurality of times during the viewing of the static visual stimulus by aggregating the distributions of each of the individuals at each of the plurality of times; and

software executing on said processor for generating a three-dimensional graphical display of the group distribution of the group'"'"'s visual resources on the static visual stimulus over the period of time, the graphical display illustrating the allocation of each portion of the group'"'"'s visual resources across each portion of the static visual stimulus viewed by the group at each of the plurality of times throughout the viewing of the static visual stimulus, wherein the graphical display includes at least a portion of the static visual stimulus.

17. The system according to claim 16, wherein the graphical display of the group distribution of the group'"'"'s visual resources includes an area of maximal salience at each of the plurality of times, each area of maximal salience corresponding to an area of the static visual stimulus.

18. The system according to claim 17, wherein the graphical display includes a three-dimensional attentional funnel comprising the areas of maximal salience.

19. The system according to claim 18, wherein a convergence of the attentional funnel at a particular time represents a heightened attention to the corresponding area of the static visual stimulus at the particular time.

33. The system according to claim 16, wherein the distribution of visual resources for each individual is a non-uniform distribution.

20. A method for quantifying and mapping visual salience to a visual stimulus, comprising the steps of:

receiving data indicative of a group of individual'"'"'s ocular responses while viewing a dynamic visual stimulus, the dynamic visual stimulus changing over time;

generating a first distribution of visual resources for each individual in the group at a first time during the viewing of the dynamic visual stimulus, said distribution representing an allocation of each portion of the individual'"'"'s visual resources across each portion of the dynamic visual stimulus viewed by the particular individual at the first time;

generating a second distribution of visual resources for each individual in the group at each of at least one additional time during the viewing of the dynamic visual stimulus, said distribution representing the allocation of each portion of the individual'"'"'s visual resources across each portion of the dynamic visual stimulus viewed by the particular individual at each of the at least one additional times;

generating data indicative of a group distribution of the group'"'"'s visual resources at each of the at least two times by aggregating the data indicative of the first and second non-uniform distributions of each of the at least a portion of the individuals; and

generating a graphical display of the distribution of the group'"'"'s visual resources on the dynamic visual stimulus over time, the graphical display illustrating the allocation of each portion of the group'"'"'s visual resources across each portion of the dynamic visual stimulus viewed by the group throughout the viewing of the dynamic visual stimulus.

21. The method according to claim 20, wherein said step of generating the graphical display includes mapping the distribution of the group'"'"'s visual resources to the visual stimulus.

22. The method according to claim 21, wherein the visual stimulus is a video, wherein the mapping includes mapping each of the first and additional times to a particular frame of the video.

23. The method according to claim 20, wherein said step of generating the data indicative of the distribution of the group'"'"'s visual resources includes calculating an average value of relative salience at each of the first and additional times.

24. The method according to claim 23, wherein said step of generating the data indicative of the distribution of the group'"'"'s visual resources includes determining an area of maximal salience at each of the first and additional times based on the average value.

25. The method according to claim 24, wherein the graphical display includes the areas of maximal salience extruded over each of the first and additional times.

26. The method according to claim 24, wherein the graphical display includes a three-dimensional attentional funnel comprising the areas of maximal salience.

27. The method according to claim 20, wherein the received data includes at least two ocular responses per second for each of at least a portion of the individuals of the group.

28. The method according to claim 20, wherein said step of generating a distribution of visual resources including determining a distribution of retinal cells.

29. The method according to claim 20, further comprising the steps of:

receiving subject data indicative of at least one subject'"'"'s ocular responses to the visual stimulus;

determining at least one point of regard at each of the first and additional times for each of the at least one subjects based on the subject data, and

generating a graphical display of the at least one subject'"'"'s points of regard over time.

30. The method according to claim 29, further comprising the step of:

comparing the graphical display of the at least one subject'"'"'s points of regard to the graphical display of the distribution of the group'"'"'s visual resources to the visual stimulus.

31. The method according to claim 30, wherein said step of comparing includes comparing values of relative salience of the group over time to the display of the at least one subject'"'"'s points of regard over time.

34. The method according to claim 20, wherein each of the first and second distributions of visual resources are non-uniform distributions.