Processing device with intuitive learning capability

• US 7,483,867 B2
• Filed: 08/30/2002
• Issued: 01/27/2009
• Est. Priority Date: 06/26/2001
• Status: Active Grant

• Current Assignee: Intuition Intelligence Incorporated
• Original Assignee: Intuition Intelligence Incorporated
• Inventors: Ansari, Arif M, Shiek Ansari, Yusuf Sulaiman M
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): Wong; Lut
• Application Number:
  US10/231,875
• Publication Number:
  US 20030149675A1
• Time in Patent Office: 2,342 Days
• Field of Search: 706/21, 706/25, 379/114.01, 379/114.28, 379/114.29
• US Class Current: 706/25
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/338 using television networks
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/798 for assessing skills or for ranking players, e.g. for generating a hall of fame computing the game score A63F13/46
• CPC Class Codes: A63F 2300/558 by assessing the players' skills or ranking
• CPC Class Codes: A63F 2300/6027 using adaptive systems learning from user actions, e.g. for skill level adjustment
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04N 21/44222 Analytics of user selections, e.g. selection of programs or purchase activity monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/466 Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/4751 for defining user accounts, e.g. accounts for children

Abstract

A method and apparatus for providing learning capability to processing device, such as a computer game, educational toy, telephone, or television remote control, is provided to achieve one or more objective. One of a plurality of actions (e.g., game actions, educational prompts, listed phone numbers, or listed television channels) to be performed on the processing device is selected. A user input indicative of a user action (e.g., a player action, educational input, called phone number, or watched television channel) is received. An outcome of the selected action and/or user action is determined. An action probability distribution having probability values corresponding to the plurality of actions is then updated based on the determined outcome. The next action will then be selected based on this updated action probability distribution. The foregoing steps can be modified based on a performance index to achieve the objective of the processing device so that it learns.

Claims

1. A processing device having one or more objectives, comprising:

an executable probabilistic learning software module having a learning automaton configured to learn a plurality of processor actions in response to a plurality of actions performed by a user, wherein the probabilistic learning software module includes an action probability distribution comprising a plurality of probability values corresponding to said plurality of processor actions;

a user interface configured to communicate said processor actions to said user;

an executable intuition software module configured to modify a functionality of said probabilistic learning module based on said one or more objectives to prevent said action probability distribution from substantially converging to any processor action; and

at least one or more hardware processors configured to execute said probabilistic learning software module and said intuition software module.

2. The processing device of claim 1, wherein said intuition module is deterministic.

3. The processing device of claim 1, wherein said intuition module is probabilistic.

4. The processing device of claim 1, wherein said intuition module comprises one or more of an expert system, a neural network, and fuzzy logic.

5. The processing device of claim 1, wherein said probabilistic learning module comprises:

an action selection module configured to select one of a plurality of processor actions based on the action probability distribution;

an outcome evaluation module configured to determine an outcome of one or both of said identified user action and said selected processor action; and

a probability update module configured to update said action probability distribution based on said outcome.

6. The processing device of claim 5, wherein said outcome is of said identified user action.

7. The processing device of claim 5, wherein said outcome is of said selected processor action.

8. The processing device of claim 5, wherein said outcome is of said selected processor action relative to said identified user action.

9. The processing device of claim 5, wherein said intuition module is configured to modify a functionality of said action selection module based on said one or more objectives.

10. The processing device of claim 5, wherein said intuition module is configured to modify a functionality of said outcome evaluation module based on said one or more objectives.

11. The processing device of claim 5, wherein said intuition module is configured to modify a functionality of said probability update module based on said one or more objectives.

12. The processing device of claim 5, wherein said intuition module, for each iteration, is configured to modify said functionality of said probabilistic learning module independent of said processor action selection, said outcome determination, and said action probability distribution update.

13. The processing device of claim 1, wherein said intuition module is configured to select one of a predetermined plurality of algorithms employed by said learning module.

14. The processing device of claim 1, wherein said intuition module is configured to modify a parameter of an algorithm employed by said learning module.

15. The processing device of claim 1, wherein said processing device has a function independent of determining an optimum action, and said learning automaton is configured to affect said processing device function.

16. The processing device of claim 1, wherein said intuition software module configured to determine a performance index indicative of a performance of said probabilistic learning module relative to said one or more objectives and for modifying the functionality of said probabilistic learning module based on said performance index.

17. The processing device of claim 1, wherein said processing device is a computer gaming console, said user actions are player moves, and said processor actions are game moves.

18. The processing device of claim 1, wherein said processing device is an educational toy, said user actions are child actions, and said processor actions are toy actions.

19. The processing device of claim 1, wherein said processing device is a telephone system, said user actions are called phone numbers, and said processor actions are listed phone numbers.

20. The processing device of claim 1, wherein said processing device is a television channel control system, said user actions are watched television channels, and said processor actions are listed television channels.