Proactive user interface including emotional agent

• US 7,725,419 B2
• Filed: 09/03/2004
• Issued: 05/25/2010
• Est. Priority Date: 09/05/2003
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Linder, Natan, Toledano, Eyal, Ben-Yair, Ran, Eisenberg, Yariv, Lee, Jong-Goo
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): COUGHLAN, PETER D
• Application Number:
  US10/933,582
• Publication Number:
  US 20050143138A1
• Time in Patent Office: 2,090 Days
• Field of Search: 706/1, 706/15, 706/45, 706/11, 706/50, 706/52, 706/5, 706/21, 700/1, 700/90, 364/419, 709/221, 709/202, 704/260, 705/14, 705/27, 705/37, 705/28, 186/39, 434/236
• US Class Current: 706/60
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 1/72448 with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 1/72469 for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
• CPC Class Codes: H04M 1/72472 wherein the items are sorted according to specific criteria, e.g. frequency of use
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10

Abstract

A proactive user interface, installed in (or otherwise control and/or be associated with) any type of computational device. The proactive user interface actively makes suggestions to the user, based upon prior experience with a particular user and/or various preprogrammed patterns from which the computational device could select, depending upon user behavior. These suggestions could be made by altering the appearance of at least a portion of the display, for example by changing a menu or a portion thereof; providing different menus for display; and/or altering touch screen functionality. The suggestions could also be made audibly.

Claims

1. A proactive user interface in combination with a computational device having an operating system, the proactive user interface comprising:

(a) an interface unit for communicating between a user of the proactive user interface and said operating system, said interface unit including an emotional agent for communicating with the user;

(b) at least one software application controlled by the operating system;

(c) an artificial intelligence (AI) framework for supporting said at least one software application, communicating with a host platform having the operating system, detecting at least one pattern of interaction of the user with said interface unit, actively suggesting, to the user, options for altering at least one function of the user interface according to said detected pattern, and altering at least one function of the user interface according to an option selected by the user among the suggested options; and

(d) a knowledge base for storing information selected from the group consisting of a pattern determined according to at least one previous interaction of the user with said interface unit, a predetermined pattern, and a combination thereof,

wherein said agent expresses at least one emotion according to a reaction of the user to said suggestion, and wherein said proactive user interface anticipates requests of the user and assists the user in selecting a desired function of the computational device.

2. The combination of claim 1, wherein said AI framework comprises:

an artificial intelligence/machine learning (AI/ML) module;

an application manager for communicating with said at least one software application;

a storage manager for managing storage and handling of data with regard to the knowledge base;

an action manager for enabling the proactive user interface to determine which action to take through an operation of the AI/ML module;

a user interface (UI) manager for managing an appearance and a function of the user interface by directing changes to the user interface; and

a device world mapper for determining a state of the computational device, a state of a virtual world, and relationship between said two states.

3. The combination of claim 2, wherein said AI/ML module determines a behavior of the proactive user interface in response to various stimuli, and enables the proactive user interface to learn from a response of the user to different types of actions of the proactive user interface.

4. The combination of claim 3, wherein said AI framework further comprises an event handler, and between said at least one software application and said event handler, a plurality of different low level managers support receipt and handling of different events, said low level managers including the action manager, the UI manager, the storage manager, and the application manager.

5. The combination of claim 3, wherein the application manager is capable of starting, pausing, resuming and stopping each of said at least one software application.

6. The combination of claim 2, wherein said AI framework further comprises an event handler, and between said at least one software application and said event handler, a plurality of different low level managers support receipt and handling of different events, said low level managers including the action manager, the UI manager, the storage manager, and the application manager.

7. The combination of claim 1, wherein said computational device is selected from the group consisting of a regular computer, an ATM, mobile information devices including a cellular telephone, a PDA, and a consumer appliance having an operating system.

8. The combination of claim 7, wherein said agent is created through a 3D graphic model.

9. The combination of claim 8, wherein said proactive user interface controls the avatar to be displayed independently of visual display aspects of the user interface.

10. The combination of claim 1, wherein said intelligent agent communicates with an object in a virtual environment.

11. The combination of claim 10, wherein said object in the virtual environment includes at least one of a ball, an animal, food, a house, and toys.

12. The combination of claim 11, wherein said object in the virtual environment includes a graded input to a state of said agent.

13. The combination of claim 12, wherein said object in the virtual environment becomes an incentive or disincentive for said agent to continue a behavior for which feedback has been provided.

14. The combination of claim 12, wherein said graded input includes reward points provided to the agent, and the agent responds to the reward points.

15. The combination of claim 1, wherein said suggestion is determined by a rule based strategy.

16. The combination of claim 15, wherein said rule based strategy comprises:

querying a knowledge base when an event occurs, and receiving a response therefrom;

determining whether the event is valid or not;

generating an action corresponding to the event and determining priority for the action; and

performing a highest priority action from among actions corresponding to the event.

17. The combination of claim 16, wherein the highest priority action is an action that maximizes an aggregated total reward from a virtual environment or a graded input in the form of encouraging or discouraging feedback.

18. The combination of claim 15, wherein the proactive user interface creates an emotion of the agent based on a change of at least one of a likelihood of success (LOS) and a likelihood of failure (LOF) of an abstract goal of the agent.

19. The combination of claim 18, wherein the created emotion is accumulated, and the agent expresses an emotional state each time a dominant behavior pattern changes.

20. The combination of claim 19, wherein the emotional state is expressed using at least one of textual communication, three dimensional animation, facial expressions, two dimensional animated effects and sounds.

21. The combination of claim 19, wherein the emotional state is expressed using at least one of textual communication, three dimensional animation, facial expressions, two dimensional animated effects and sounds.

22. The combination of claim 19, wherein the emotional state is expressed using at least one of textual communication, three dimensional animation, facial expressions, two dimensional animated effects and sounds.