Devices, systems, and methods for learning and using artificially intelligent interactive memories

• US 9,582,762 B1
• Filed: 02/05/2016
• Issued: 02/28/2017
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: Storyfile LLC
• Original Assignee: Jasmin Cosic
• Inventors: Cosic, Jasmin
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Zidanic, Michael
• Application Number:
  US15/016,280
• Time in Patent Office: 389 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 65/1106 Call signalling protocols; H.323 and related
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Aspects of the disclosure generally relate to computing devices and may be generally directed to devices, systems, methods, and/or applications for learning conversations among two or more conversation participants, storing this knowledge in a knowledgebase (i.e. neural network, graph, sequences, etc.), and enabling a user to simulate a conversation with an artificially intelligent conversation participant.

Claims

1. A system for using artificially intelligent interactive memories, the system implemented at least in part on one or more computing devices, the system comprising:

one or more processor circuits;

a memory unit, coupled to the one or more processor circuits, that comprises a plurality of interconnected rounds of conversational exchange including a first round of conversational exchange, the first round of conversational exchange comprising a recording of a first conversation participant'"'"'s first conversational activity correlated with a recording of a second conversation participant'"'"'s first conversational activity;

a picture-capturing device, coupled to the one or more processor circuits, configured to capture a stream of digital pictures of a user; and

a sound-capturing device, coupled to the one or more processor circuits, configured to capture a stream of digital sound samples of the user, wherein the one or more processor circuits are configured to;

detect the user'"'"'s first conversational activity from at least one of the stream of digital pictures of the user or the stream of digital sound samples of the user;

compare at least one portion of a recording of the user'"'"'s first conversational activity with at least one portion of the recording of the first conversation participant'"'"'s first conversational activity;

determine at least a partial match between the recording of the user'"'"'s first conversational activity and the recording of the first conversation participant'"'"'s first conversational activity; and

cause a display and a sound-producing device to play at least one portion of the recording of the second conversation participant'"'"'s first conversational activity, the causing performed in response to the determining at least a partial match between the recording of the user'"'"'s first conversational activity and the recording of the first conversation participant'"'"'s first conversational activity.

2. The system of claim 1, wherein the plurality of interconnected rounds of conversational exchange include a second round of conversational exchange connected with the first round of conversational exchange, the second round of conversational exchange comprising a recording of the first conversation participant'"'"'s second conversational activity correlated with a recording of the second conversation participant'"'"'s second conversational activity, and wherein the one or more processor circuits are further configured to:

detect the user'"'"'s second conversational activity from at least one of the stream of digital pictures of the user or the stream of digital sound samples of the user;

compare at least one portion of a recording of the user'"'"'s second conversational activity with at least one portion of the recording of the first conversation participant'"'"'s second conversational activity;

determine at least a partial match between the recording of the user'"'"'s second conversational activity and the recording of the first conversation participant'"'"'s second conversational activity; and

cause the display and the sound-producing device to play at least one portion of the recording of the second conversation participant'"'"'s second conversational activity, the causing performed in response to the determining at least a partial match between the recording of the user'"'"'s second conversational activity and the recording of the first conversation participant'"'"'s second conversational activity.

3. The system of claim 1, wherein the recording of the first conversation participant'"'"'s first conversational activity includes:

a first sub-stream of a stream of digital pictures of the first conversation participant, the first sub-stream of the stream of digital pictures of the first conversation participant comprising the first conversation participant'"'"'s visual expressions or communication in a first part of a conversation and

a first sub-stream of a stream of digital sound samples of the first conversation participant, the first sub-stream of the stream of digital sound samples of the first conversation participant comprising the first conversation participant'"'"'s verbal expressions or communication in the first part of the conversation, and

wherein the recording of the second conversation participant'"'"'s first conversational activity includes;

a first sub-stream of a stream of digital pictures of the second conversation participant, the first sub-stream of the stream of digital pictures of the second conversation participant comprising the second conversation participant'"'"'s visual expressions or communication in the first part of the conversation and

a first sub-stream of a stream of digital sound samples of the second conversation participant, the first sub-stream of the stream of digital sound samples of the second conversation participant comprising the second conversation participant'"'"'s verbal expressions or communication in the first part of the conversation.

4. The system of claim 1, wherein the user'"'"'s first conversational activity includes at least one of:

a user'"'"'s speaking, a user'"'"'s silent facial expression, a user'"'"'s silent body movement, a user'"'"'s motionless silence, a user'"'"'s absence from the conversation, or a user'"'"'s conversational action.

5. The system of claim 1, wherein the plurality of interconnected rounds of conversational exchange include a neural network.

6. The system of claim 1, wherein the plurality of interconnected rounds of conversational exchange include a graph.

7. The system of claim 1, wherein the plurality of interconnected rounds of conversational exchange include one or more sequences.

8. The system of claim 1, wherein the plurality of interconnected rounds of conversational exchange include a knowledgebase, a knowledge structure, or a data structure.

9. The system of claim 1, wherein the determining at least a partial match between the recording of the user'"'"'s first conversational activity and the recording of the first conversation participant'"'"'s first conversational activity includes determining that a similarity between at least one portion of the recording of the user'"'"'s first conversational activity and at least one portion of the recording of the first conversation participant'"'"'s first conversational activity exceeds a similarity threshold.

10. The system of claim 1, wherein the recording of the user'"'"'s first conversational activity includes:

a first sub-stream of the stream of digital pictures of the user, the first sub-stream of the stream of digital pictures of the user comprising the user'"'"'s visual expressions or communication in a first part of a simulated conversation and

a first sub-stream of the stream of digital sound samples of the user, the first sub-stream of the stream of digital sound samples of the user comprising the user'"'"'s verbal expressions or communication in the first part of the simulated conversation.

11. The system of claim 1, wherein the comparing at least one portion of the recording of the user'"'"'s first conversational activity with at least one portion of the recording of the first conversation participant'"'"'s first conversational activity includes comparing at least one word recognized from the recording of the user'"'"'s first conversational activity with at least one word recognized from the recording of the first conversation participant'"'"'s first conversational activity.

12. The system of claim 1, wherein the comparing at least one portion of the recording of the user'"'"'s first conversational activity with at least one portion of the recording of the first conversation participant'"'"'s first conversational activity includes comparing at least one digital picture from the recording of the user'"'"'s first conversational activity with at least one digital picture from the recording of the first conversation participant'"'"'s first conversational activity.

13. The system of claim 1, wherein the determining at least a partial match between the recording of the user'"'"'s first conversational activity and the recording of the first conversation participant'"'"'s first conversational activity includes determining that a number or a percentage of at least partially matching words recognized from the recording of the user'"'"'s first conversational activity and from the recording of the first conversation participant'"'"'s first conversational activity exceeds a threshold.

14. The system of claim 1, wherein the determining at least a partial match between the recording of the user'"'"'s first conversational activity and the recording of the first conversation participant'"'"'s first conversational activity includes determining that a number or a percentage of at least partially matching digital pictures from the recording of the user'"'"'s first conversational activity and from the recording of the first conversation participant'"'"'s first conversational activity exceeds a threshold.

15. The system of claim 1, wherein the one or more processor circuits, the memory unit, the picture-capturing device, the sound-capturing device, the display, and the sound-producing device of the system are included in a single device.

16. The system of claim 1, wherein at least one of:

the one or more processor circuits or the memory unit of the system are included in a server, and wherein the picture-capturing device, the sound-capturing device, the display, and the sound-producing device of the system are included in a user device, the user device coupled to the server via a network.

17. A non-transitory computer storage medium having a computer program stored thereon, the program comprising instructions that when executed by one or more processor circuits cause the one or more processor circuits to perform operations comprising:

accessing a memory unit that comprises a plurality of interconnected rounds of conversational exchange including a first round of conversational exchange, the first round of conversational exchange comprising a recording of a first conversation participant'"'"'s first conversational activity correlated with a recording of a second conversation participant'"'"'s first conversational activity;

receiving a stream of digital pictures of a user from a picture-capturing device;

receiving a stream of digital sound samples of the user from a sound-capturing device;

detecting the user'"'"'s first conversational activity from at least one of the stream of digital pictures of the user or the stream of digital sound samples of the user;

comparing at least one portion of a recording of the user'"'"'s first conversational activity with at least one portion of the recording of the first conversation participant'"'"'s first conversational activity;

determining at least a partial match between the recording of the user'"'"'s first conversational activity and the recording of the first conversation participant'"'"'s first conversational activity; and

causing a display and a sound-producing device to play at least one portion of the recording of the second conversation participant'"'"'s first conversational activity in response to the determining at least a partial match between the recording of the user'"'"'s first conversational activity and the recording of the first conversation participant'"'"'s first conversational activity.

18. The non-transitory computer storage medium of claim 17, wherein the plurality of interconnected rounds of conversational exchange include a second round of conversational exchange connected with the first round of conversational exchange, the second round of conversational exchange comprising a recording of the first conversation participant'"'"'s second conversational activity correlated with a recording of the second conversation participant'"'"'s second conversational activity, and wherein the operations further comprising:

detecting the user'"'"'s second conversational activity from at least one of the stream of digital pictures of the user or the stream of digital sound samples of the user;

comparing at least one portion of a recording of the user'"'"'s second conversational activity with at least one portion of the recording of the first conversation participant'"'"'s second conversational activity;

determining at least a partial match between the recording of the user'"'"'s second conversational activity and the recording of the first conversation participant'"'"'s second conversational activity; and

causing the display and the sound-producing device to play at least one portion of the recording of the second conversation participant'"'"'s second conversational activity in response to the determining at least a partial match between the recording of the user'"'"'s second conversational activity and the recording of the first conversation participant'"'"'s second conversational activity.

19. A method comprising:

(a) accessing a memory unit that comprises a plurality of interconnected rounds of conversational exchange including a first round of conversational exchange, the first round of conversational exchange comprising a recording of a first conversation participant'"'"'s first conversational activity correlated with a recording of a second conversation participant'"'"'s first conversational activity, the accessing of (a) performed by one or more processor circuits;

(b) capturing a stream of digital pictures of a user by a picture-capturing device;

(c) capturing a stream of digital sound samples of the user by a sound-capturing device;

(d) detecting the user'"'"'s first conversational activity from at least one of the stream of digital pictures of the user or the stream of digital sound samples of the user, the detecting of (d) performed by the one or more processor circuits;

(e) comparing at least one portion of a recording of the user'"'"'s first conversational activity with at least one portion of the recording of the first conversation participant'"'"'s first conversational activity, the comparing of (e) performed by the one or more processor circuits;

(f) determining at least a partial match between the recording of the user'"'"'s first conversational activity and the recording of the first conversation participant'"'"'s first conversational activity, the determining of (f) performed by the one or more processor circuits; and

(g) playing at least one portion of the recording of the second conversation participant'"'"'s first conversational activity by a display and a sound-producing device, the playing of (g) caused by the one or more processor circuits in response to the determining of (f).

20. The method of claim 19, wherein the plurality of interconnected rounds of conversational exchange include a second round of conversational exchange connected with the first round of conversational exchange, the second round of conversational exchange comprising a recording of the first conversation participant'"'"'s second conversational activity correlated with a recording of the second conversation participant'"'"'s second conversational activity, and wherein the method further comprising:

(h) detecting the user'"'"'s second conversational activity from at least one of the stream of digital pictures of the user or the stream of digital sound samples of the user, the detecting of (h) performed by the one or more processor circuits;

(i) comparing at least one portion of a recording of the user'"'"'s second conversational activity with at least one portion of the recording of the first conversation participant'"'"'s second conversational activity, the comparing of (i) performed by the one or more processor circuits;

(j) determining at least a partial match between the recording of the user'"'"'s second conversational activity and the recording of the first conversation participant'"'"'s second conversational activity, the determining of (j) performed by the one or more processor circuits; and

(k) playing at least one portion of the recording of the second conversation participant'"'"'s second conversational activity by the display and the sound-producing device, the playing of (k) caused by the one or more processor circuits in response to the determining of (j).