System and method for generating a customized augmented reality environment to a user

• US 10,535,192 B2
• Filed: 08/31/2016
• Issued: 01/14/2020
• Est. Priority Date: 10/26/2005
• Status: Active Grant

• Current Assignee: Cortica Ltd.
• Original Assignee: Cortica Ltd.
• Inventors: Raichelgauz, Igal, Odinaev, Karina, Zeevi, Yehoshua Y
• Primary Examiner(s): Le, Uyen T
• Application Number:
  US15/252,790
• Publication Number:
  US 20160371890A1
• Time in Patent Office: 1,231 Days
• Field of Search: None
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 : Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/022 : Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/025 : Extracting rules from data
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 20/46 : Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/48 : Matching video sequences
• CPC Class Codes: G10L 15/26 : Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/32 : Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 25/51 : for comparison or discrimination
• CPC Class Codes: H04H 20/103 : Transmitter-side switching
• CPC Class Codes: H04H 20/26 : Arrangements for switching distribution systems
• CPC Class Codes: H04H 60/37 : for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/46 : for recognising users' preferences
• CPC Class Codes: H04H 60/56 : Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
• CPC Class Codes: H04H 60/66 : for using the result on distributors' side
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04N 21/233 : Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 : involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/25891 : being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2668 : Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/466 : Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/8106 : involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/8146 : involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 21/8545 : for generating interactive applications
• CPC Class Codes: H04N 7/17318 : Direct or substantially direct transmission and handling of requests
• CPC Class Codes: Y10S 707/99943 : Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99948 : Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

Systems and methods for generating a customized augmented reality environment. A method includes causing generation of at least one signature for at least one multimedia content; causing, based on the generated at least one signature, identification of at least one matching concept, wherein each concept is a collection of signatures and metadata representing the concept; determining, based on the identified at least one matching concept, a selection of at least one augmented reality character; and generating a customized augmented reality environment, wherein the customized augmented reality environment includes each augmented reality character superimposed on the at least one multimedia content element as an overlay.

Claims

1. A method for generating a customized augmented reality environment, comprising:

causing generation of at least one signature for at least one multimedia content;

causing, based on the generated at least one signature, identification of at least one matching concept, wherein each concept is a collection of signatures and metadata representing the concept;

determining, based on the identified at least one matching concept, a selection of at least one augmented reality character;

wherein each one of the at least one augmented reality character is associated with a character signature;

identifying at least one potential placement location in the at least one multimedia content element, wherein each potential placement location is associated with at least a portion of the generated at least one signature;

correlating between the at least one potential placement location signature portion and the at least one character signature to select, for each augmented reality character, a placement location in the at least one multimedia content element;

and

generating a customized augmented reality environment, wherein the customized augmented reality environment includes each augmented reality character superimposed, at a selected placement location related to the augmented reality character, on the at least one multimedia content element as an overlay.

2. The method of claim 1, wherein each signature is generated by a signature generator system including a plurality of computational cores, each computational core having properties that are at least partially statistically independent of properties of the other computational cores.

3. The method of claim 1, wherein a concept matches the generated at least one signature when a representative signature of the concept matches one of the generated at least one signature above a predetermined threshold.

4. The method of claim 3, wherein the at least one matching concept is identified by matching the generated at least one signature to a plurality of signatures representing concepts.

5. The method of claim 4, wherein causing identification of the at least one matching concept further comprises:

querying a deep-content classification system using the generated at least one signature, wherein the deep-content classification system matches the generated at least one signature to a plurality of concepts to identify the at least one matching concept.

6. The method of claim 1, wherein determining the selection of the at least one augmented reality character further comprises:

comparing the at least one matching concept to characteristics of a plurality of potential augmented reality characters, wherein the characteristics of each potential augmented reality character includes contextually related concepts.

7. The method of claim 1, wherein determining the selection of the at least one augmented reality character is based on an in-game performance related to a game associated with the customized augmented reality environment.

8. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute a method, the method comprising:

causing generation of at least one signature for at least one multimedia content, wherein each signature is generated by a signature generator system including a plurality of computational cores, each computational core having properties that are at least partially statistically independent of properties of the other computational cores;

causing, based on the generated at least one signature, identification of at least one matching concept;

determining, based on the identified at least one matching concept, a selection of at least one augmented reality character;

wherein each one of the at least one augmented reality character is associated with a character signature;

identifying at least one potential placement location in the at least one multimedia content element, wherein each potential placement location is associated with at least a portion of the generated at least one signature;

correlating between the at least one potential placement location signature portion and the at least one character signature to select, for each augmented reality character, a placement location in the at least one multimedia content element;

and

generating a customized augmented reality environment, wherein the customized augmented reality environment includes each augmented reality character superimposed, at a selected placement location related to the augmented reality character, on the at least one multimedia content element as an overlay.

9. A system for generating a customized augmented reality environment, comprising:

a processing circuitry; and

a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to;

cause generation of at least one signature for at least one multimedia content;

cause, based on the generated at least one signature, identification of at least one matching concept, wherein each concept is a collection of signatures and metadata representing the concept;

determine, based on the identified at least one matching concept, a selection of at least one augmented reality character;

wherein each one of the at least one augmented reality character is associated with a character signature;

identify at least one potential placement location in the at least one multimedia content element, wherein each potential placement location is associated with at least a portion of the generated at least one signature;

correlate between the at least one potential placement location signature portion and the at least one character signature to select, for each augmented reality character, a placement location in the at least one multimedia content element;

and

generate a customized augmented reality environment, wherein the customized augmented reality environment includes each augmented reality character superimposed, at a selected placement location related to the augmented reality character, on the at least one multimedia content element as an overlay.

10. The system of claim 9, wherein each signature is generated by a signature generator system including a plurality of computational cores, each computational core having properties that are at least partially statistically independent of properties of the other computational cores.

11. The system of claim 10, wherein a concept matches the generated at least one signature when a representative signature of the concept matches one of the generated at least one signature above a predetermined threshold.

12. The system of claim 11, wherein the system is further configured to:

query a deep-content classification system using the generated at least one signature, wherein the deep-content classification system matches the generated at least one signature to a plurality of concepts to identify the at least one matching concept.

13. The system of claim 9, wherein the system is further configured to:

compare the at least one matching concept to characteristics of a plurality of potential augmented reality characters, wherein the characteristics of each potential augmented reality character includes contextually related concepts.

14. The system of claim 9, wherein a determining of the selection of the at least one augmented reality character is based on an in-game performance related to a game associated with the customized augmented reality environment.

15. A method for generating a customized augmented reality environment, comprising:

causing generation of at least one signature for at least one multimedia content;

determining, based on the generated at least one signature, a selection of at least one augmented reality character, wherein each augmented reality character is associated with a character signature;

identifying at least one potential placement location in the at least one multimedia content element, wherein each potential placement location is associated with at least a portion of the generated at least one signature;

correlating between the at least one potential placement location signature portion and the at least one character signature to select, for each augmented reality character, a placement location in the at least one multimedia content element; and

generating a customized augmented reality environment, wherein the customized augmented reality environment includes each augmented reality character superimposed on the at least one multimedia content element at the respective placement location.

16. The method of claim 15, further comprising:

causing, based on the generated at least one signature, identification of at least one matching concept, wherein the at least one augmented reality character is determined further based on the identified at least one matching concept.

17. The method of claim 16, wherein each concept is a collection of signatures and metadata representing the concept.

18. The method of claim 17, wherein a concept matches the generated at least one signature when a representative signature of the concept matches one of the generated at least one signature above a predetermined threshold.

19. The method of claim 16, wherein determining the selection of the at least one augmented reality character further comprises:

comparing the at least one matching concept to characteristics of a plurality of potential augmented reality characters, wherein the characteristics of each potential augmented reality character includes contextually related concepts.

20. The method of claim 15, wherein the selection of each placement location is further based on a size of the respective augmented reality character.

21. The method of claim 15, wherein each signature is generated by a signature generator system including a plurality of computational cores, each computational core having properties that are at least partially statistically independent of properties of the other computational cores.

22. A system for generating a customized augmented reality environment, comprising:

a processing circuitry; and

a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to;

cause generation of at least one signature for at least one multimedia content;

determine, based on the generated at least one signature, a selection of at least one augmented reality character, wherein each augmented reality character is associated with a character signature;

identify at least one potential placement location in the at least one multimedia content element, wherein each potential placement location is associated with at least a portion of the generated at least one signature;

correlate between the at least one potential placement location signature portion and the at least one character signature to select, for each augmented reality character, a placement location in the at least one multimedia content element; and

generate a customized augmented reality environment, wherein the customized augmented reality environment includes each augmented reality character superimposed on the at least one multimedia content element at the respective placement location.

23. The system of claim 22, wherein the system is further configured to:

cause, based on the generated at least one signature, identification of at least one matching concept, wherein the at least one augmented reality character is determined further based on the identified at least one matching concept.

24. The system of claim 23, wherein each concept is a collection of signatures and metadata representing the concept.

25. The system of claim 24, wherein a concept matches the generated at least one signature when a representative signature of the concept matches one of the generated at least one signature above a predetermined threshold.

26. The system of claim 22, wherein the system is further configured to:

compare the at least one matching concept to characteristics of a plurality of potential augmented reality characters, wherein the characteristics of each potential augmented reality character includes contextually related concepts.

27. The system of claim 22, wherein the selection of each placement location is further based on a size of the respective augmented reality character.

28. The system of claim 22, wherein each signature is generated by a signature generator system including a plurality of computational cores, each computational core having properties that are at least partially statistically independent of properties of the other computational cores.