Method for designing an interactive system

• US 6,975,970 B2
• Filed: 12/15/2000
• Issued: 12/13/2005
• Est. Priority Date: 12/15/2000
• Status: Active Grant

• Current Assignee: Cufer Asset Ltd LLC (Intellectual Ventures LLC)
• Original Assignee: Soliloquy, Inc.
• Inventors: Thorisson, Kristinn R.
• Primary Examiner(s): PALADINI, ALBERT WILLIAM
• Application Number:
  US09/737,426
• Publication Number:
  US 20020077726A1
• Time in Patent Office: 1,824 Days
• Field of Search: 703/1, 703/13, 706/19, 706/20, 706/292, 715/500.1, 705/1
• US Class Current: 703/1
• CPC Class Codes: B25J 13/00 Controls for manipulators programme controls B25J9/16; control in general G05

Abstract

A method of effectively designing an interactive system is disclosed. The method may preferably include the steps of (1) determining the overall scope of the desired interactive system, including available sources of input and output; (2) creating a list of desired features for the system; (3) creating a rough outline of perceptions, decisions, and actions that the system must be capable of if it is to possess the desired features; (4) designing perception features that implement the identified perception specifications and decision features that implement the decision-making process required by the identified decision specifications; (5) creating any necessary supporting components (e.g., buffers for storing data collected by perception features) to implement the perception and decision features; and (6) creating a behavior feature hierarchy that includes the action features required to implement action features of the interactive system. Each perception feature and decision feature is preferably assigned a priority level which may be a function of several factors. The present method may find particular application in designing an interactive system that implements an anytime algorithm and in designing an interactive system that implements a distributed, asynchronous architecture.

Claims

1. A method of designing an interactive system, the method comprising:

providing one or more sensors, each sensor adapted to capture one or more real-time events;

providing one or more perception features, each perception feature adapted to receive information concerning one or more captured events from one or more sensors and to generate one or more perceptions;

providing one or more decision features, each decision feature adapted to receive one or more perceptions from one or more perception features and to generate one or more decisions;

providing one or more output mechanisms, each output mechanism receiving one or more decisions from one or more decision features, and executing one or more actions;

assigning a priority level to each perception feature and each decision feature on the basis of a plurality of factors comprising duration of an action, complexity of an action, and pre-planning requirements of an action; and

utilizing said priority levels to load-balance the running of perception features and decision features under time constraints and normal system operation, and to select among conflicting actions requested by two or more decision features.

2. The method of claim 1, wherein one or more of said sensors is a visual sensor.

3. The method of claim 1, wherein one or more of said sensors is a spatial sensor.

4. The method of claim 1, wherein one or more of said sensors is an auditory sensor.

5. The method of claim 1, wherein one or more of said sensors is a tactile sensor.

6. The method of claim 1, wherein one or more of said sensors is a balance sensor.

7. The method of claim 1, wherein one or more of said actions comprises generating speech.

8. The method of claim 1, wherein one or more of said actions comprises generating text.

9. The method of claim 1, wherein one or more of said actions comprises generating phonemes.

10. The method of claim 1, wherein one or more of said actions comprises generating an intonation.

11. The method of claim 1, wherein one or more of said actions comprises generating an animated movement.

12. The method of claim 11, wherein the movement includes facial expression.

13. The method of claim 11, wherein the movement includes body movement.

14. The method of claim 11, wherein the movement includes hand gesture.

15. The method of claim 11, wherein the movement includes arm gesture.

16. The method of claim 11, wherein the movement includes visemes.

17. The method of claim 1, wherein one or more of said actions comprises generating physical movement.

18. The method of claim 17, wherein the movement includes facial expression.

19. The method of claim 17, wherein the movement includes body movement.

20. The method of claim 17, wherein the movement includes hand gesture.

21. The method of claim 17, wherein the movement includes arm gesture.

22. The method of claim 17, wherein the movement includes visemes.

23. The method of claim 1, further comprising:

selecting one of said decision features; and

assigning a high priority level to the selected decision feature on the basis of one or more criteria, including;

a. when output from the selected decision feature results in an action requiring less than approximately 500 milliseconds to perform;

b. when output from the selected decision feature results in a simple action that requires few motors to perform;

c. when the input to the said decision feature are output produced by a perception feature or features with a high priority;

d. when the inputs to the decision feature come from a relatively few perception features;

e. when the decision feature requires relatively simple processing;

f. when the amount of preplanning required for the decision is low or none.

24. The method of claim 1, further comprising:

selecting one of said decision features; and

assigning a medium priority level to the selected decision feature on the basis of one or more criteria, including;

a. when the selected decision feature results in an action requiring approximately 500–

2000 milliseconds to perform;

b. when the selected decision feature receives output from a relatively large number of features;

c. when the selected decision feature is clearly related to hand-holding of the interaction between the system and a system user;

d. when the input to the said selected decision feature is from a perception feature with a high or medium priority level;

e. when the amount of preplanning required for the decision is intermediate;

f. when the action resulting from the decision feature'"'"'s decision output is complex, or requires a high number of motors to be performed.

25. The method of claim 1, further comprising:

selecting one of said decision features; and

assigning a low priority level to the selected decision feature on the basis of one or more criteria, including;

a. when the duration of an action resulting from said selected decision feature is approximately longer than 2000 milliseconds;

b. when said selected decision feature requires complex processing;

c. when said selected decision feature requires large amounts of pre-planning;

d. when said selected decision feature is related to a knowledge-based topic other than the interaction itself;

e. when the input to said selected decision feature is from a feature with a medium or low priority level.

26. The method of claim 1, further comprising:

selecting one of said perception features; and

assigning a high priority level to the selected perception feature on the basis of one or more criteria, including;

a. when input of said selected perception feature is a single data entry, or a collection of very few data entries;

b. when input of said selected perception feature comes from a single sensor;

c. when the processing time of said selected perception feature supports one or more actions having a perception-to-action loop time that is shorter than 500 milliseconds;

d. if one or more decision features supported by said selected perception feature supports one or more outputs that result in an action requiring less than approximately 500 milliseconds to perform;

e. when data needed already exists as raw data;

f. when data gathering takes roughly 300 milliseconds or less;

g. when input of said selected perception feature is produced by one or more perception features each of which have been assigned a high priority.

27. The method of claim 1, further comprising:

selecting one of said perception features; and

assigning a medium priority level to the selected perception feature on the basis of one or more criteria, including;

a. when data gathering of said selected perception feature spans approximately between 300 and 1500 milliseconds;

b. when input to said selected perception feature comprises a relatively large number of data points from more than one sensor;

c. when a decision feature supported by the output of said selected perception feature is directly related to an interaction between said interactive system and its users;

d. when data processing of said selected perception feature requires information from various parts of said interactive system, including the context of an interaction;

e. when said selected perception feature supports perception-to-action loop time between 500 and 1500 milliseconds;

f. when said selected perception feature takes between 300 and 1500 milliseconds to produce output;

g. when said selected perception feature'"'"'s input is the output of other perception features that have been assigned a medium or high priority level;

h. when said selected perception feature'"'"'s output supports a decision feature whose resulting action takes between 500 and 1500 milliseconds to perform.

28. The method of claim 1, further comprising:

selecting one of said perception features; and

assigning a low priority level to the selected perception feature based on one or more criteria, including;

a. when data gathering of said selected perception feature requires longer than approximately 1500 milliseconds;

b. when data processing of said selected perception feature requires complex data or knowledge structures for producing output;

c. when said selected perception feature produces complex, knowledge-based output;

d. when said selected perception feature'"'"'s output supports processes for high-level decision making such as complex natural language processing or planning ahead;

e. when data processing of said selected perception feature takes more than 1500 milliseconds to perform;

f. when data processing of said selected perception feature requires information from multiple parts of said interactive system, including the context of an interaction;

g. when said selected perception feature'"'"'s input is the output of other perception features that have been assigned a low or medium priority level;

h. when said selected perception feature'"'"'s output supports a decision feature whose resulting action takes longer than 1500 milliseconds to perform;

i. when said perception feature is difficult or impossible to decompose into small perceptual features;

j. when said perception feature typically cannot support a perception-to-action loop of shorter than 1500 millisecond duration.

29. The method of claim 1, wherein the method is used to design an interactive system that implements one or more anytime algorithms.

30. The method of claim 1, wherein the method is used to design an interactive system that is implemented in a distributed, asynchronous architecture.