System and method for customizing a display of a user device based on multimedia content element signatures

• US 10,706,094 B2
• Filed: 05/22/2017
• Issued: 07/07/2020
• Est. Priority Date: 10/26/2005
• Status: Active Grant

• Current Assignee: Cortica Ltd.
• Original Assignee: Cortica Ltd.
• Inventors: Raichelgauz, Igal, Odinaev, Karina, Zeevi, Yehoshua Y
• Primary Examiner(s): Le, Hung D
• Application Number:
  US15/601,303
• Publication Number:
  US 20170262437A1
• Time in Patent Office: 1,142 Days
• Field of Search: None
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/152 using file content signatures, e.g. hash values
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/45 Clustering; Classification
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/483 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/51 : Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/683 : using metadata automatically derived from the content
• CPC Class Codes: G06F 16/685 : using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/739 : in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/783 : using metadata automatically derived from the content
• CPC Class Codes: G06F 16/7834 : using audio features
• CPC Class Codes: G06F 16/7844 : using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/7847 : using low-level visual features of the video content
• CPC Class Codes: G06F 16/904 : Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 16/951 : Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 : Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9558 : Details of hyperlinks; Management of linked annotations
• CPC Class Codes: G06F 17/16 : Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 3/048 : Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0484 : for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 40/134 : Hyperlinking
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 3/045 : Combinations of networks
• CPC Class Codes: G06N 3/048 : Activation functions
• CPC Class Codes: G06N 3/063 : using electronic means
• CPC Class Codes: G06N 3/088 : Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 5/02 : Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/022 : Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/025 : Extracting rules from data
• CPC Class Codes: G06N 5/04 : Inference or reasoning models
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/0201 : Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0246 : Traffic
• CPC Class Codes: G06Q 30/0261 : based on user location
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 20/40 : in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/48 : Matching video sequences
• CPC Class Codes: G06V 2201/10 : Recognition assisted with metadata
• CPC Class Codes: G06V 40/171 : Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G09B 19/0092 : Nutrition
• CPC Class Codes: G10L 15/26 : Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/32 : Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 25/51 : for comparison or discrimination
• CPC Class Codes: H04H 20/103 : Transmitter-side switching
• CPC Class Codes: H04H 20/26 : Arrangements for switching distribution systems
• CPC Class Codes: H04H 20/93 : which locates resources of other pieces of information, e.g. URL [Uniform Resource Locator]
• CPC Class Codes: H04H 2201/90 : characterised by the use of signatures
• CPC Class Codes: H04H 60/33 : Arrangements for monitoring the users' behaviour or opinions
• CPC Class Codes: H04H 60/37 : for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/46 : for recognising users' preferences
• CPC Class Codes: H04H 60/49 : for identifying locations
• CPC Class Codes: H04H 60/56 : Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04H 60/58 : of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/59 : of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04H 60/66 : for using the result on distributors' side
• CPC Class Codes: H04H 60/71 : using meteorological information
• CPC Class Codes: H04L 51/18 : Commands or executable codes
• CPC Class Codes: H04L 51/42 : Mailbox-related aspects, e.g. synchronisation of mailboxes
• CPC Class Codes: H04L 65/75 : Media network packet handling
• CPC Class Codes: H04L 65/765 : intermediate
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/306 : User profiles
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/63 : Routing a service request depending on the request content or context
• CPC Class Codes: H04N 21/23418 : involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/25891 : being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/26603 : for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/2668 : Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/278 : Content descriptor database or directory service for end-user access details of content or meta data based information retrieval of video data in video databases G06F16/739
• CPC Class Codes: H04N 21/466 : Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/8106 : involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 7/17318 : Direct or substantially direct transmission and handling of requests
• CPC Class Codes: Y10S 707/99943 : Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A system and method customizing a display of a user device based on multimedia content elements. The method may include detecting a trigger event; identifying a user profile of the user device, wherein the user profile includes one or more signatures indicating a user interest; accessing signatures for a plurality of multimedia content elements; comparing the signatures of at least one of the plurality of multimedia content elements with the one or more signature of the user profile; and customizing the display of the user device, wherein the customized display includes the at least one interesting multimedia content element.

Claims

1. A method for customizing a display of a user device based on multimedia content element signatures, comprising:

detecting a trigger event;

upon detecting the trigger event, identifying a user profile of the user device, wherein the user profile includes at least one signature of at least one multimedia content element associated with the user, the at least one signature is indicative of at least one user interest, each one of the at least one signature belongs to a concept that comprises the signature and metadata representing the concept;

wherein each signature is generated by a signature generator including a plurality of computational cores, each computational core having properties that are at least partially statistically independent of properties of the other computational cores, wherein the properties of each computational core are set independently of the properties of each other core;

comparing the at least one signature of the user profile to a plurality of signatures of multimedia content elements;

determining, based on the comparison, at least one interesting multimedia content element, wherein the at least one interesting multimedia content element is relevant to at least one of the at least one user interest; and

customizing the display of the user device, wherein the customized display includes the at least one interesting multimedia content element.

2. The method of claim 1, further comprising:

generating at least one user profile, wherein each user profile is generated based on of the at least one multimedia content element associated with the user, wherein the at least one user profile is associated with a user of the user device.

3. The method of claim 2, wherein generating each user profile further comprises:

identifying a plurality of multimedia content elements associated with the respective user;

generating at least one signature for each of the plurality of multimedia content elements associated with the user;

analyzing the generated at least one signature to identify at least one concept matching the multimedia content elements associated with the user;

generating, based on the at least one matching concept, at least one contextual insight, wherein each contextual insight indicates a preference of the user; and

generating, based on the at least one contextual insight, the user profile for the user.

4. The method of claim 1, wherein the computational cores are configured such that each generated signature is robust to noise and distortion.

5. The method of claim 1, where the trigger event is detected based on at least one trigger variable.

6. The method of claim 5, wherein the at least one trigger variable includes at least one of:

at least one personal variable associated with a user of the user device, at least one environmental variable associated with the user device, and at least one user input.

7. The method of claim 6, wherein the at least one personal variable includes demographic information related to the user.

8. The method of claim 6, wherein the at least one environmental variable includes at least one of:

a user search history, a time of day, a list of social friends as posted on a social web site, user device location information, user device motion information, and weather information.

9. The non-transitory computer readable medium of claim 1, where the trigger event is detected based on at least one trigger variable.

10. The non-transitory computer readable medium of claim 9, wherein the at least one trigger variable includes at least one of:

at least one personal variable associated with a user of the user device, at least one environmental variable associated with the user device, and at least one user input.

11. The non-transitory computer readable medium of claim 10, wherein the at least one personal variable includes demographic information related to the user.

12. The non-transitory computer readable medium of claim 10, wherein the at least one environmental variable includes at least one of:

a user search history, a time of day, a list of social friends as posted on a social web site, user device location information, user device motion information, and weather information.

13. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute a method, the method comprising:

detecting a trigger event;

upon detecting the trigger event, identifying a user profile of the user device, wherein the user profile includes at least one signature of at least one multimedia content element associated with the user, the at least one signature is indicative of at least one user interest, each one of the at least one signature belongs to a concept that comprises the at least one signature and metadata representing the concept;

wherein each signature is generated by a signature generator including a plurality of computational cores, each computational core having properties that are at least partially statistically independent of properties of the other computational cores, wherein the properties of each computational core are set independently of the properties of each other core;

comparing the at least one signature of the user profile to a plurality of signatures of multimedia content elements;

determining, based on the comparison, at least one interesting multimedia content element, wherein the at least one interesting multimedia content element is relevant to at least one of the at least one user interest; and

customizing the display of the user device, wherein the customized display includes the at least one interesting multimedia content element.

14. The non-transitory computer readable medium of claim 13, having stored thereon instructions for:

generating at least one user profile, wherein each user profile is generated based on of the at least one multimedia content element associated with the user, wherein the at least one user profile is associated with a user of the user device.

15. The non-transitory computer readable medium of claim 14, wherein generating each user profile further comprises:

identifying a plurality of multimedia content elements associated with the respective user;

generating at least one signature for each of the plurality of multimedia content elements associated with the user;

analyzing the generated at least one signature to identify at least one concept matching the multimedia content elements associated with the user;

generating, based on the at least one matching concept, at least one contextual insight, wherein each contextual insight indicates a preference of the user; and

generating, based on the at least one contextual insight, the user profile for the user.

16. The non-transitory computer readable medium of claim 13, wherein the computational cores are configured such that each generated signature is robust to noise and distortion.

17. A system for customizing a display of a user device based on multimedia content element signatures, comprising:

a processing circuitry; and

a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to;

detecting a trigger event;

upon detecting the trigger event, identifying a user profile of the user device, wherein the user profile includes at least one signature of at least one multimedia content element associated with the user, the at least one signature is indicative of at least one user interest, each one of the at least one signatures belongs to a concept that comprises the at least one signature and metadata representing the concept;

wherein each signature is generated by a signature generator including a plurality of computational cores, each computational core having properties that are at least partially statistically independent of properties of the other computational cores, wherein the properties of each computational core are set independently of the properties of each other core;

comparing the at least one signature of the user profile to a plurality of signatures of multimedia content elements;

determining, based on the comparison, at least one interesting multimedia content element, wherein the at least one interesting multimedia content element is relevant to at least one of the at least one user interest; and

customizing the display of the user device, wherein the customized display includes the at least one interesting multimedia content element.

18. The method of claim 17, further comprising:

generating at least one user profile, wherein each user profile is generated based on the at least one multimedia content element associated with the user, wherein the at least one user profile is associated with a user of the user device.

19. The method of claim 18, wherein generating each user profile further comprises:

identifying a plurality of multimedia content elements associated with the respective user;

generating at least one signature for each of the plurality of multimedia content elements associated with the user;

analyzing the generated at least one signature to identify at least one concept matching the multimedia content elements associated with the user;

generating, based on the at least one matching concept, at least one contextual insight, wherein each contextual insight indicates a preference of the user; and

generating, based on the at least one contextual insight, the user profile for the user.

20. The method of claim 17, wherein the computational cores are configured such that each generated signature is robust to noise and distortion.

21. The method of claim 17, where the trigger event is detected based on at least one trigger variable.

22. The method of claim 21, wherein the at least one trigger variable includes at least one of:

at least one personal variable associated with a user of the user device, at least one environmental variable associated with the user device, and at least one user input.

23. The method of claim 22, wherein the at least one personal variable further includes demographic information related to the user.

24. The method of claim 22, wherein the at least one environmental variable includes at least one of:

a user search history, a time of day, a list of social friends as posted on a social web site, user device location information, user device motion information, and weather information.