Apparatus and method for determining user attention using a deep-content-classification (DCC) system

• US 9,372,940 B2
• Filed: 08/29/2013
• Issued: 06/21/2016
• Est. Priority Date: 10/26/2005
• Status: Expired due to Fees

• Current Assignee: Cortica Ltd.
• Original Assignee: Cortica Ltd.
• Inventors: Raichelgauz, Igal, Odinaev, Karina, Zeevi, Yehoshua Y.
• Primary Examiner(s): Le, Uyen
• Application Number:
  US14/013,636
• Publication Number:
  US 20140006442A1
• Time in Patent Office: 1,027 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A method for determining a user attention to at least one multimedia data element (MMDE) displayed in a web-page over a user computing device. The method comprises receiving a request to determine the user attention, wherein the request includes at least the web-page and an identification of the at least one MMDE in the web-page; receiving at least one sensory signal captured by at least one sensor connected to the user computing device; querying a deep-content-classification (DCC) system to find a match between at least one concept structure and the received sensory signal; receiving a first set of metadata related to the at least one matched concept structure; analyzing the returned set of metadata to determine the user attention with respect to the at least one MMDE; and associating the at least one MMDE with the determined user attention.

Claims

1. A method for determining a user attention to at least one multimedia data element (MMDE) displayed in a web-page over a user computing device, comprising:

receiving a request to determine the user attention, wherein the request includes at least the web-page and an identification of the at least one MMDE in the web-page;

receiving at least one sensory signal captured by at least one sensor connected to the user computing device;

querying a deep-content-classification (DCC) system to find a match between at least one concept structure and the received sensory signal, wherein the at least one concept structure includes a plurality of metadata associated with reduced clusters of MMDEs, the clusters represented by a compressed number of signatures;

receiving a first set of metadata related to the at least one matched concept structure;

analyzing the returned set of metadata to determine the user attention with respect to the at least one MMDE; and

associating the at least one MMDE with the determined user attention.

2. The method of claim 1, further comprising:

querying the DCC system to find a match between at least one concept structure and the at least one MMDE; and

correlating a second set of metadata returned respective of the at least one MMDE with at least one of;

the determined user attention and the first set of metadata.

3. The method of claim 2, wherein each of the first and second set of metadata is a textual representation of multimedia content associated with the respective matched concept structure.

4. The method of claim 2, wherein further comprising:

generating at least one signature respective of the MMDE and the sensory signal prior to querying of the MMDE.

5. The method of claim 1, wherein the at least one sensory signal is at least one of:

an audio signal, a video signal, coordinates, and a sonography signal.

6. The method of claim 1, wherein the MMDE is at least one of:

an image, graphics, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, images of signals, medical signals, geophysical signals, subsonic signals, supersonic signals, electromagnetic signals, and infrared signals.

7. The method of claim 1, wherein the request further includes supplemental information, wherein the supplemental information comprises at least one of:

an IP address of the user computing device, time, date, and a browser type of the IP address of the user computing device.

8. The method of claim 1, wherein the DCC system includes:

an attention processor (AP) for generating a plurality of items from the received sensory signal and determining which of the generated items are of interest for signature generation;

a signature generator (SG) for generating at least one signature responsive to at least one item of interest of the sensory signal; and

a concept generator (CG) for matching between the at least one signature generated responsive to at least one item of interest of the sensory signal and the at least one concept structure to identify at least the first set of metadata.

9. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to claim 1.

10. A system for determining a user attention to at least one multimedia data element (MMDE) displayed in a web-page over a user computing device, comprising:

an interface to a network for receiving a request to determine the user attention, wherein the request includes at least the web-page and an identification of the at least one MMDE in the web-page for receiving at least one sensory signal captured by at least one sensor connected to the user computing device;

a processor; and

a memory connected to the processor, the memory contains instructions that when executed by the processor, the system is configured to;

query a deep-content-classification (DCC) system to find a match between at least one concept structure and the received sensory signal, wherein the at least one concept structure includes a plurality of metadata associated with reduced clusters of the MMDEs, the clusters represented by a compressed number of signatures;

receive a first set of metadata related to the at least one matched concept structure;

analyze the returned set of metadata to determine the user attention with respect to the at least one MMDE; and

associate the at least one MMDE with the determined user attention.

11. The system of claim 10, further comprising a database configured to save associations of the at least one MMDE with the determined user attention.

12. The system of claim 10, wherein the system is further configured to:

query the DCC system to find a match between the at least one concept structure and the at least one MMDE; and

correlate a second set of metadata returned respective of the least one MMDE with at least one of the determined user attention and the first set of metadata.

13. The system of claim 12, wherein each of the first and second set of metadata is a textual representation of multimedia content associated with the respective matched concept structure.

14. The system of claim 12, wherein the system is further configured to:

generate at least one signature to each of the MMDE and the sensory signal prior to querying of the MMDE.

15. The system of claim 10, wherein the at least one sensory signal is at least one of:

an audio signal, a video signal, coordinates, and a sonography signal.

16. The system of claim 10, wherein the MMDE is at least one of:

an image, graphics, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, images of signals, medical signals, geophysical signals, subsonic signals, supersonic signals, electromagnetic signals, and infrared signals.

17. The system of claim 10, wherein the request further includes supplemental information, wherein the supplemental information comprises at least one of:

an IP address of the user computing device, time, date, and a browser type of the IP address of the user computing device.

18. The system of claim 10, wherein the DCC system includes:

an attention processor (AP) for generating a plurality of items from the received sensory signal and determining which of the generated items are of interest for signature generation;

a signature generator (SG) for generating at least one signature responsive to at least one item of interest of the sensory signal; and

a concept generator (CG) for matching between the at least one signature generated responsive to at least one item of interest of the sensory signal and the at least one concept structure to identify at least the first set of metadata.