Evolution of a user interface based on learned idiosyncrasies and collected data of a user

• US 8,725,659 B2
• Filed: 12/21/2012
• Issued: 05/13/2014
• Est. Priority Date: 09/21/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Weising, George
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Olude Afolabi, Ola
• Application Number:
  US13/723,943
• Publication Number:
  US 20130117201A1
• Time in Patent Office: 508 Days
• Field of Search: 706/11, 455/557
• US Class Current: 706/11
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A user interface evolves based on learned idiosyncrasies and collected data of a user. Learned idiosyncrasies and collected data of the user can be stored in a knowledge base. Information from the surrounding environment of the user can be obtained during learning of idiosyncrasies or collection of data. Thought-based statements can be generated based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data. The thought-based statements serve to invoke or respond to subsequent actions of the user. The user interface can be presented so as to allow for interaction with the user based at least in part on the thought-based statements. Furthermore, personality nuances of the user interface can be developed that affect the interaction between the user and the user interface.

Claims

1. A method for providing a user interface that evolves based on learned idiosyncrasies and collected data of a user, the method comprising:

storing learned idiosyncrasies and collected data of the user in a knowledge base;

executing instructions stored in memory to obtain information from the surrounding environment of the user during learning of idiosyncrasies or collection of data;

executing instructions stored in memory to generate thought-based statements based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data, the thought-based statements serving to invoke or respond to subsequent actions of the user;

evaluate the importance of the thought-based statements; and

executing instructions stored in memory to present the user interface and to allow for interaction with the user based at least in part on the thought-based statements.

2. The method of claim 1, wherein executing instructions to obtain the information from the surrounding environments includes deciphering detected speech from the user.

3. The method of claim 1, wherein executing instructions to obtain the information from the surrounding environments includes optically detecting a body part of the user.

4. The method of claim 3, wherein optically detecting the body part includes detecting a manual gesture.

5. The method of claim 3, wherein optically detecting the body part includes performing facial recognition.

6. The method of claim 1, wherein executing instructions to obtain the information from the surrounding environment includes determining one or more of the location of the user or motion of the user.

7. The method of claim 1, wherein executing instructions to obtain the information from the surrounding environments includes providing a response to an audio signal in the surrounding environment of the system.

8. The method of claim 1, wherein executing instructions to generate the thought-based statements includes prioritizing the thought-based statements based on importance.

9. The method of claim 1, wherein the user interface includes an avatar.

10. The method of claim 1, further comprising executing instructions stored in memory to develop personality nuances of the user interface that affect the interaction between the user and the user interface.

11. A system for providing a user interface that evolves based on learned idiosyncrasies and collected data of a user, the system comprising:

a knowledge base configured to store learned idiosyncrasies and collected data of the user;

a sensory engine stored in memory and executable to obtain information from the surrounding environment of the user during learning of idiosyncrasies or collection of data;

a thought engine stored in memory and executable to generate thought-based statements based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data, the thought-based statements serving to invoke or respond to subsequent actions of the user;

an interaction engine stored in memory and executable to present the user interface and to allow for interaction with the user based at least in part on the thought-based statements; and

a conversation analysis module stored in memory and executable to analyze past verbal input by the user.

12. The system of claim 11, further comprising a personality engine stored in memory and executable to develop personality nuances of the user interface that affect the interaction between the user and the user interface.

13. A non-transitory computer-readable storage medium having a program embodied thereon, the program executable by a processor to perform a method for providing a user interface that evolves based on learned idiosyncrasies and collected data of a user, the method comprising:

storing learned idiosyncrasies and collected data of the user in a knowledge base;

obtaining information from the surrounding environment of the user during learning of idiosyncrasies or collection of data;

generating thought-based statements based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data, the thought-based statements serving to invoke or respond to subsequent actions of the use;

evaluating the importance of the thought-based statements; and

presenting the user interface and to allow for interaction with the user based at least in part on the thought-based statements.

14. The method of claim 13, wherein obtaining the information from the surrounding environments includes deciphering detected speech from the user.