System and method for determining a pupillary response to a multimedia data element

• US 9,466,068 B2
• Filed: 02/13/2015
• Issued: 10/11/2016
• Est. Priority Date: 10/26/2005
• Status: Active Grant

• Current Assignee: Cortica Ltd.
• Original Assignee: Cortica Ltd.
• Inventors: Raichelgauz, Igal, Odinaev, Karina, Zeevi, Yehoshua Y.
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US14/621,653
• Publication Number:
  US 20150161213A1
• Time in Patent Office: 606 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute

Abstract

A method for determining a pupillary response to a multimedia data element viewed through a user computing device. The method comprises receiving a first image of a viewer'"'"'s pupil captured prior to display of the MMDE over the user computing device; receiving a second image of a viewer'"'"'s pupil captured after the display of the MMDE over the user computing device; determining, using the first image and the second image, if the viewer'"'"'s pupil has been dilated; querying a deep-content-classification system to find a match between at least one concept structure and the at least second image of the user'"'"'s pupil; upon identification of at least one matching concept, receiving a first set of metadata related to the at least one matching concept structure; determining the viewer'"'"'s attention to the displayed MMDE respective of the first set of metadata; associating the at least one MMDE with the determined user attention.

Claims

1. A method for determining a pupillary response to a multimedia data element (MMDE) viewed through a user computing device, comprising:

receiving a first image of a viewer'"'"'s pupil captured prior to display of the MMDE over the user computing device;

receiving a second image of a viewer'"'"'s pupil captured after the display of the MMDE over the user computing device;

determining, using the first image and the second image, if the viewer'"'"'s pupil has been dilated;

when identifying of a pupil dilation, querying a deep-content-classification (DCC) system to find a match between at least one concept structure and the at least second image of the user'"'"'s pupil;

upon identification of at least one matching concept, receiving a first set of metadata related to the at least one matching concept structure;

determining the viewer'"'"'s attention to the displayed MMDE respective of the first set of metadata; and

associating the at least one MMDE with the determined user attention.

2. The method of claim 1, further comprises:

determining preferences of the viewer with respect to the displayed content.

3. The method of claim 1, wherein determining the preferences of the viewer further comprises:

querying the DCC system to find at least one concept structure matching the MMDE;

receiving a second set of metadata related to the at least one matching concept structure of the MMDE; and

correlating the second set of metadata returned respective of the at least one MMDE with at least one of the determined user attention and the first set of metadata.

4. The method of claim 3, wherein each of the first and second set of metadata is a textual representation of multimedia content associated with a respective matching concept structure.

5. The method of claim 1, wherein determining using the first image and the second image if the viewer'"'"'s pupil has been dilated further comprises:

generating a first signature for the first image;

generating a second signature for the second image;

comparing the first signature to the second signature to determine if the first signature and second signature are substantially matched; and

when the first signature and second signature are substantially matched, determining a pupil dilation.

6. The method of claim 5, further comprising:

comparing the second signature to a signature of a concept structure representing a pupil dilation to confirm a winding of pupil.

7. The method of claim 1, wherein the MMDE is at least one of:

an image, graphics, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, images of signals, medical signals, geophysical signals, subsonic signals, supersonic signals, electromagnetic signals, and infrared signals.

8. The method of claim 1, wherein the DCC system comprises:

an attention processor (AP) for generating a plurality of items from the received sensory signal and determining which of the generated items are of interest for signature generation;

a signature generator (SG) for generating at least one signature responsive to at least one item of interest of the sensory signal; and

a concept generator (CG) for matching between the at least one signature generated responsive to at least one item of interest of the sensory signal and a plurality of signature reduced clusters associated with a plurality of cluster structures to identify at least the first set of metadata.

9. The method of claim 1, wherein a concept structure is a signature reduced cluster of signatures describing multimedia elements.

10. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to claim 1.

11. A system for determining pupillary response to a multimedia data element (MMDE) viewed through a user computing device, comprising:

a processor; and

a memory connected to the processor, the memory contains instructions that when executed by the processor, the system is configured to;

receive a first image of a viewer'"'"'s pupil captured prior to display of the MMDE over the user computing device;

receive a second image of a viewer'"'"'s pupil captured after the display of the MMDE over the user computing device;

determine, using the first image and the second image, if the viewer'"'"'s pupil has been dilated;

when identifying of a pupil dilation, query a deep-content-classification (DCC) system to find a match between at least one concept structure and the at least second image of the user'"'"'s pupil;

upon identification of at least one matching concept, receive a first set of metadata related to the at least one matching concept structure;

determine the viewer'"'"'s attention to the displayed MMDE respective of the first set of metadata; and

associate the at least one MMDE with the determined user attention.

12. The system of claim 11, further configured to:

determine preferences of the viewer with respect to the displayed content.

13. The system of claim 11, wherein when determining the preferences of the viewer the system is further configured to:

query the DCC system to find at least one concept structure matching the MMDE;

receive a second set of metadata related to the at least one matching concept structure of the MMDE; and

correlate the second set of metadata returned respective of the at least one MMDE with at least one of the determined user attention and the first set of metadata.

14. The system of claim 13, wherein each of the first and second set of metadata is a textual representation of multimedia content associated with a respective matching concept structure.

15. The system of claim 11, wherein when determining using the first image and the second image if the viewer'"'"'s pupil has been dilated the system is further configured to:

generate a first signature for the first image;

generate a second signature for the second image;

compare the first signature to the second signature to determine if the first signature and second signature are substantially matched; and

when the first signature and second signature are substantially matched, determine a pupil dilation.

16. The system of claim 15, further comprising:

compare the second signature to a signature of a concept structure representing a pupil dilation to confirm a winding of pupil.

17. The system of claim 11, wherein the MMDE is at least one of:

an image, graphics, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, images of signals, medical signals, geophysical signals, subsonic signals, supersonic signals, electromagnetic signals, and infrared signals.

18. The system of claim 11, wherein the DCC system comprises:

an attention processor (AP) for generating a plurality of items from the received sensory signal and determining which of the generated items are of interest for signature generation;

a signature generator (SG) for generating at least one signature responsive to at least one item of interest of the sensory signal; and

a concept generator (CG) for matching between the at least one signature generated responsive to at least one item of interest of the sensory signal and a plurality of signature reduced clusters associated with a plurality of cluster structures to identify at least the first set of metadata.

19. The system of claim 11, wherein a concept structure is a signature reduced cluster of signatures describing multimedia elements.