Information processing apparatus and method capable of processing plurality type of input information

• US 7,523,031 B1
• Filed: 12/23/1999
• Issued: 04/21/2009
• Est. Priority Date: 12/24/1998
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Wakai, Masanori, Jeyachandran, Suresh, Suda, Aruna Rohra
• Primary Examiner(s): Edouard; Patrick N
• Assistant Examiner(s): Spooner; Lamont M
• Application Number:
  US09/471,030
• Time in Patent Office: 3,407 Days
• Field of Search: 704/1, 704/9, 704/10, 704/270, 704/277, 704/275, 707/100, 707/102, 707 2- 6
• US Class Current: 704/9
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

An information processing apparatus has: an input unit capable of inputting a plurality type of information such as hand-written information, key information and speech information; and a knowledge database storing input information and information necessary for generating a concept instance in one-to-one correspondence. A combination of at least two types of input information is analyzed by referring to the knowledge database to generate a concept instance from each piece of the input information. A plurality type of input information can therefore be understood collectively by correlating the input information each other and considering the input order of the input is information.

Claims

1. An information processing apparatus comprising:

a plurality of input means for inputting different types of information;

conversion means for converting information input from any of said plurality of input means into a character string representing the information;

storage means for storing a plurality of character strings converted by said conversion means with an input time thereof;

sorting means for sorting the plurality of character strings stored in said storage means in an order of the input time;

a knowledge base for storing knowledge of each concept which includes a character string representing the concept, a concept type of the concept, a concept instance rule for defining a property of a concept instance to be filled in each slot provided for the concept in correspondence with a slot type of the slot, and a surface rule for defining an order of a character string representing the concept instance to be filled in each slot and a grammar of the character string representing the concept;

input concept instance generating means for generating a plurality of input concept instances corresponding to the plurality of character strings sorted in the input time order by said sorting means, by referring to the knowledge stored in said knowledge base;

request information list generating means for generating a request information list which includes, for each of the plurality of input concept instances, I) an instance request for filling respective slots with specified concept instances in accordance with the concept instance rule of the input concept instance, II) a surface request for obeying a specified order of character strings and a specific grammar in accordance with the surface rule of the input concept instance, and, III) a concept definition request which does not overlap with any existing requests in accordance with a different concept instance rule defined in a concept definition for the concept type of the input concept instance;

applicable request list generating means for generating an applicable request list by generating combinations each of which includes an applicable concept request in the request information list and a corresponding concept instance in the plurality of input concept instances, and excluding a combination which does not satisfy the surface request in the request information list and a combination which includes an applicable concept request competing with other requests from the generated combinations, included in the request list; and

concept instance unifying means for unifying the plurality of input concept instances by filling a slot of one of the input concept instances with another one of the input concept instances having a concept type which matches with the property of the concept instance for the slot by applying an applicable request in the applicable request list.

2. An information processing apparatus according to claim 1, further comprising status acquisition means for acquiring an operation status of a program executed in said apparatus at an input time.

3. An information processing apparatus according to claim 1, further comprising status storage means for storing a past status, wherein said status concept instance generating means generates the status concept instance in accordance with the past status read from said status storage means.

4. An information processing apparatus according to claim 1, wherein said input means can input key information.

5. An information processing apparatus according to claim 4, wherein said input means can input character information by converting the key information.

6. An information processing apparatus according to claim 1, wherein said input means can input speech information.

7. An information processing apparatus according to claim 6, wherein said input means can input character information by recognizing the speech information and converting the speech information into character information.

8. An information processing apparatus according to claim 1, wherein said input means can optically input image information.

9. An information processing apparatus according to claim 8, wherein said input means can input character information of the image information by optically recognizing the image information.

10. An information processing apparatus according to claim 1, wherein said input means can input hand-written information.

11. An information processing apparatus according to claim 10, wherein said input means can input the hand-written character information by recognizing the hand-written character information on line.

12. An information processing method performed in an apparatus, said method comprising:

an input step, of inputting different types of information by a plurality of input units;

a conversion step, of converting information input from any of said plurality of input units into a character string representing the information;

a storing step, of storing a plurality of character strings converted in said conversion step with an input time thereof in a storage unit;

a sorting step, of sorting the plurality of character strings stored in the storage unit in an order of the input time;

an input concept instance generating step, of generating a plurality of input concept instances corresponding to the plurality of words sorted in the input time order in said sorting step, by referring to knowledge stored in a knowledge base, which stores knowledge of each concept which includes a character string representing the concept, a concept type of the concept, a concept instance rule for defining a property of a concept instance to be filled in each slot provided for the concept in correspondence with a slot type of the slot, and a surface rule for defining an order of a character string representing the concept instance to be filled in each slot and a grammar of the character string representing the concept;

a request information list generating step, of generating a request information list which includes, for each of the plurality of input concept instances, I) an instance request for filling respective slots with specified concept instances in accordance with the concept instance rule of the input concept instance, II) a surface request for obeying a specified order of character strings and a specific grammar in accordance with the surface rule of the input concept instance, and, III) a concept definition request which does not overlap with any existing requests in accordance with a different concept instance rule defined in a concept definition for the concept type of the input concept instance;

an applicable request list generating step, of generating an applicable request list by generating combinations each of which includes an applicable concept request in the request information list and a corresponding concept instance in the plurality of input concept instances, and excluding a combination which does not satisfy the surface request in the request information list and a combination which includes an applicable concept request competing with other requests from the generated combinations, included in the request list; and

a concept instance unifying step, of unifying the plurality of input concept instances by filling a slot of one of the input concept instances with another one of the input concept instances having a concept type which matches with the property of the concept instance for the slot by applying an applicable request in the applicable request list.

13. An information processing method according to claim 12, further comprising a status acquisition step, for acquiring an operation status of a program executed in said apparatus at an input time.

14. An information processing method according to claim 12, further comprising a status storing step, of storing a past status, wherein said status concept instance generating step generates the status concept instance in accordance with the past status read in said status storing step.

15. An information processing method according to claim 12, wherein said input step can input key information.

16. An information processing method according to claim 15, wherein said input step can input character information by converting the key information.

17. An information processing method according to claim 12, wherein said input step can input speech information.

18. An information processing method according to claim 17, wherein said input step can input character information by recognizing the speech information and converting the speech information into character information.

19. An information processing method according to claim 12, wherein said input step can optically input image information.

20. An information processing method according to claim 19, wherein said input step can input character information of the image information by optically recognizing the image information.

21. An information processing method according to claim 12, wherein said input step can input hand written information.

22. An information processing method according to claim 21, wherein said input step can input the hand-written character information by recognizing the hand-written character information on line.

23. A computer-readable storage medium storing a computer-executable information processing program for controlling a computer to perform information processing in an apparatus, said program comprising:

code for an input step, of inputting different types of information;

code for a conversion step, of converting information input from any of said plurality of input units into a character string representing the information;

code for a storing step, of storing a plurality of character strings converted by said conversion code with an input time thereof in a storage unit;

code for a sorting step, of sorting the plurality of character strings stored in the storage unit in an order of the input time;

code for an input concept instance generating step, of generating a plurality of input concept instances corresponding to the plurality of character strings sorted in the input time order by said sorting code, by referring to knowledge stored in a knowledge base, which stores knowledge of each concept which includes a character string representing the concept, a concept type of the concept, a concept instance rule for defining a property of a concept instance to be filled in each slot provided for the concept in correspondence with a slot type of the slot, and a surface rule for defining an order of a character string representing the concept instance to be filled in each slot and a grammar of the character string representing the concept;

code for a request information list generating step, of generating a request information list which includes, for each of the plurality of input concept instances, I) an instance request for filling respective slots with specified concept instances in accordance with the concept instance rule of the input concept instance, II) a surface request for obeying a specified order of character strings and a specific grammar in accordance with the surface rule of the input concept instance, and, III) a concept definition request which does not overlap with any existing requests in accordance with a different concept instance rule defined in a concept definition for the concept type of each of the input concept instance;

code for an applicable request list generating step, of generating an applicable request list by generating combinations each of which includes an applicable concept request in the request information list and a corresponding concept instance in the plurality of input concept instances, and excluding a combination which does not satisfy the surface request in the request information list and a combination which includes an applicable concept request competing with other requests from the generated combinations, included in the request list; and

code for a concept instance unifying step, of unifying the plurality of input concept instances by filling a slot of one of the input concept instances with another one of the input concept instances having a concept type which matches with the property of the concept instance for the slot by applying an applicable request in the applicable request list.