Evolution of a user interface based on learned idiosyncrasies and collected data of a user

• US 8,504,487 B2
• Filed: 09/21/2010
• Issued: 08/06/2013
• Est. Priority Date: 09/21/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Weising, George
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): OLUDE AFOLABI, OLATOYOSI
• Application Number:
  US12/887,397
• Publication Number:
  US 20120072379A1
• Time in Patent Office: 1,050 Days
• Field of Search: 706/11, 455/557
• US Class Current: 706/11
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A user interface evolves based on learned idiosyncrasies and collected data of a user. Learned idiosyncrasies and collected data of the user can be stored in a knowledge base. Information from the surrounding environment of the user can be obtained during learning of idiosyncrasies or collection of data. Thought-based statements can be generated based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data. The thought-based statements serve to invoke or respond to subsequent actions of the user. The user interface can be presented so as to allow for interaction with the user based at least in part on the thought-based statements. Furthermore, personality nuances of the user interface can be developed that affect the interaction between the user and the user interface.

Claims

1. A method for providing a user interface that evolves based on learned idiosyncrasies and collected data of a user, the method comprising:

storing learned idiosyncrasies and collected data of the user in a knowledge base;

executing instructions stored in memory to obtain information from the surrounding environment of the user during learning of idiosyncrasies or collection of data;

executing instructions stored in memory to generate thought-based statements based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data, the thought-based statements serving to invoke or respond to subsequent actions of the user;

executing instructions stored in memory to present the user interface and to allow for interaction with the user based at least in part on the thought-based statements; and

executing instructions stored in memory to develop personality nuances of the user interface that affect the interaction between the user and the user interface, wherein executing instructions to develop the personality nuances includes analyzing past verbal input by the user, wherein the generated thought-based statements are further based on the analysis of the past verbal input.

2. The method of claim 1, wherein executing instructions to obtain the information from the surrounding environments includes deciphering detected speech from the user.

3. The method of claim 1, wherein executing instructions to obtain the information from the surrounding environments includes optically detecting a body part of the user.

4. The method of claim 3, wherein optically detecting the body part includes detecting a manual gesture.

5. The method of claim 3, wherein optically detecting the body part includes performing facial recognition.

6. The method of claim 1, wherein executing instructions to obtain the information from the surrounding environment includes determining one or more of the location of the user or motion of the user.

7. The method of claim 1, wherein executing instructions to obtain the information from the surrounding environments includes providing a response to an audio signal in the surrounding environment of the system.

8. The method of claim 1, wherein executing instructions to generate the thought-based statements includes prioritizing the thought-based statements based on importance.

9. The method of claim 1, wherein the user interface includes an avatar.

10. The method of claim 1, wherein executing instructions to develop the personality nuances includes analyzing past interactions between the user and the system, wherein the generated thought-based statements are further based on the analysis of the past interactions.

11. The method of claim 1, wherein executing instructions to develop the personality nuances includes forming desires of the intelligent user interface, wherein the generated thought-based statements are further based on the desires.

12. The method of claim 1, wherein executing instructions to develop the personality nuances includes forming goals of the intelligent user interface, wherein the generated thought-based statements are further based on the goals.

13. A method for providing a user interface that evolves based on learned idiosyncrasies and collected data of a user, the method comprising:

storing learned idiosyncrasies and collected data of the user in a knowledge base;

executing instructions stored in memory to obtain information from the surrounding environment of the user during learning of idiosyncrasies or collection of data;

executing instructions stored in memory to generate thought-based statements based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data, the thought-based statements serving to invoke or respond to subsequent actions of the user;

executing instructions stored in memory to present the user interface and to allow for interaction with the user based at least in part on the thought-based statements; and

executing instructions stored in memory to develop personality nuances of the user interface that affect the interaction between the user and the user interface, wherein executing instructions to develop the personality nuances includes analyzing past interactions between the user and the system, wherein the generated thought-based statements are further based on the analysis of the past interactions.

14. The method of claim 13, wherein executing instructions to obtain the information from the surrounding environments includes deciphering detected speech from the user.

15. The method of claim 13, wherein executing instructions to obtain the information from the surrounding environments includes optically detecting a body part of the user.

16. The method of claim 15, wherein optically detecting the body part includes detecting a manual gesture.

17. The method of claim 15, wherein optically detecting the body part includes performing facial recognition.

18. The method of claim 13, wherein executing instructions to obtain the information from the surrounding environment includes determining one or more of the location of the user or motion of the user.

19. The method of claim 13, wherein executing instructions to obtain the information from the surrounding environments includes providing a response to an audio signal in the surrounding environment of the system.

20. The method of claim 13, wherein executing instructions to generate the thought-based statements includes prioritizing the thought-based statements based on importance.

21. The method of claim 13, wherein the user interface includes an avatar.

22. A method for providing a user interface that evolves based on learned idiosyncrasies and collected data of a user, the method comprising:

storing learned idiosyncrasies and collected data of the user in a knowledge base;

executing instructions stored in memory to obtain information from the surrounding environment of the user during learning of idiosyncrasies or collection of data;

executing instructions stored in memory to generate thought-based statements based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data, the thought-based statements serving to invoke or respond to subsequent actions of the user;

executing instructions stored in memory to present the user interface and to allow for interaction with the user based at least in part on the thought-based statements; and

executing instructions stored in memory to develop personality nuances of the user interface that affect the interaction between the user and the user interface, wherein executing instructions to develop the personality nuances includes forming desires of the intelligent user interface, wherein the generated thought-based statements are further based on the desires.

23. The method of claim 22, wherein executing instructions to obtain the information from the surrounding environments includes deciphering detected speech from the user.

24. The method of claim 22, wherein executing instructions to obtain the information from the surrounding environments includes optically detecting a body part of the user.

25. The method of claim 24, wherein optically detecting the body part includes detecting a manual gesture.

26. The method of claim 24, wherein optically detecting the body part includes performing facial recognition.

27. The method of claim 22, wherein executing instructions to obtain the information from the surrounding environment includes determining one or more of the location of the user or motion of the user.

28. The method of claim 22, wherein executing instructions to obtain the information from the surrounding environments includes providing a response to an audio signal in the surrounding environment of the system.

29. The method of claim 22, wherein executing instructions to generate the thought-based statements includes prioritizing the thought-based statements based on importance.

30. The method of claim 22, wherein the user interface includes an avatar.

31. A method for providing a user interface that evolves based on learned idiosyncrasies and collected data of a user, the method comprising:

storing learned idiosyncrasies and collected data of the user in a knowledge base;

executing instructions stored in memory to obtain information from the surrounding environment of the user during learning of idiosyncrasies or collection of data;

executing instructions stored in memory to generate thought-based statements based at least in part on the knowledge base and the information from the environment surrounding the user during learning of idiosyncrasies or collection of data, the thought-based statements serving to invoke or respond to subsequent actions of the user;

executing instructions stored in memory to present the user interface and to allow for interaction with the user based at least in part on the thought-based statements; and

executing instructions stored in memory to develop personality nuances of the user interface that affect the interaction between the user and the user interface, wherein executing instructions to develop the personality nuances includes forming goals of the intelligent user interface, wherein the generated thought-based statements are further based on the goals.

32. The method of claim 31, wherein executing instructions to obtain the information from the surrounding environments includes deciphering detected speech from the user.

33. The method of claim 31, wherein executing instructions to obtain the information from the surrounding environments includes optically detecting a body part of the user.

34. The method of claim 33, wherein optically detecting the body part includes detecting a manual gesture.

35. The method of claim 33, wherein optically detecting the body part includes performing facial recognition.

36. The method of claim 31, wherein executing instructions to obtain the information from the surrounding environment includes determining one or more of the location of the user or motion of the user.

37. The method of claim 31, wherein executing instructions to obtain the information from the surrounding environments includes providing a response to an audio signal in the surrounding environment of the system.

38. The method of claim 31, wherein executing instructions to generate the thought-based statements includes prioritizing the thought-based statements based on importance.

39. The method of claim 31, wherein the user interface includes an avatar.