Information service system responsive to user's pointing manipulation

• US 5,687,333 A
• Filed: 03/30/1995
• Issued: 11/11/1997
• Est. Priority Date: 03/31/1994
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Dobashi, Hironori, Hashiya, Seiichi
• Primary Examiner(s): Bayerl, Raymond J.
• Assistant Examiner(s): SAX, STEVEN PAUL
• Application Number:
  US08/413,761
• Time in Patent Office: 957 Days
• Field of Search: 395/336, 395/326, 395/329, 395/330, 395/334, 395/335, 395/339, 395/340, 395/357
• US Class Current: 715/705
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041

Abstract

A predetermined position on a paper sheet is pointed by a finger, and a first image of the paper sheet picked up by an image pickup device before pointing by the finger is stored. A second image of the paper sheet picked up by the image pickup device upon pointing by the finger is stored. By subtracting the first image from the second image, the pointed position on the paper sheet by the finger is detected. The meaning or content data of a term at the detected position is read out from a database, and is displayed.

Claims

1. An information service system comprising:

a database which stores data for explaining a plurality of items or terms;

means for picking up image information of a visual medium on which one or more items or terms are displayed;

first obtaining means for obtaining first image information of the visual medium from said picking up means;

second obtaining means for obtaining second image information representing a visual pointing object and a specific item or term on the visual medium from said picking up means, wherein the specific item or term is pointed by the visual pointing object;

tip position detecting means for detecting a tip position of the visual pointing object from the first and second image information obtained by said first and second obtaining means;

word recognizing means for recognizing a keyword included in image data of the specific item or term from the first image information obtained by said first obtaining means, wherein the recognized keyword is located at a particular position of the visual medium and the particular position corresponds to the tip position of the visual pointing object detected by said detecting means;

means for searching said database in accordance with the keyword corresponding to the image data of the specific item or term recognized by said word recognizing means, so as to find explanation information for the specific item or term; and

means for outputting the explanation information found by said searching means.

2. The system of claim 1, wherein said searching means includes:

character recognizing means for recognizing one or more characters constituting the specific item or term picked up by said word recognizing means, so as to provide a character code corresponding to the recognized characters; and

means for comparing the character code provided by said character recognizing means with the data stored in said database, so as to provide the explanation information for the specific item or term.

3. The system of claim 1, wherein said outputting means includes means for displaying not only an image of the visual medium but also an image of the explanation information found by said searching means.

4. The system of claim 1, wherein said outputting means includes means for providing a voice service relating to the explanation information found by said searching means.

5. The system of claim 1, wherein the specific item or term on the visual medium includes any of a word, picture, color, or pattern.

6. The system of claim 1, wherein an image of the visual pointing object includes a finger image of a system user who needs explanation of the specific item or term pointed by his or her finger.

7. The system of claim 1, wherein an image of the visual pointing object includes a rod whose tip points the specific item or term.

8. The system of claim 1, wherein an image of the visual pointing object includes a light beam whose tip points the specific item or term.

9. The system of claim 1, wherein said tip position detecting means includes:

difference detecting means for detecting a difference between the first image information obtained by said first obtaining means and the second image information obtained by said second obtaining means, so as to extract an image of the visual pointing object;

start position detecting means for detecting a start position of the specific item or term which start position is pointed by the visual pointing object; and

end position detecting means for detecting an end position of the specific item or term which end position is pointed by the visual pointing object,

and wherein said searching means includes;

means for extracting an item or term within a specific area on the visual medium which specific area is defined by the start position detected by said start position detecting means and the end position detected by said end position detecting means.

10. The system of claim 1, further comprising:

means for recording an image of the visual medium;

means for reproducing the image of the visual medium recorded by said recording means; and

means for capturing an image reproduced by said reproducing means so as to supply the captured image as the first image information to said first obtaining means.

11. An information service method using a database which stores data for explaining a plurality of items or terms, said information service method comprising the steps of:

picking up image information of a visual medium on which one or more items or terms exist;

obtaining first image information of the visual medium from said image information;

obtaining second image information representing a visual pointing object and a specific item or term on the visual medium from said image information, wherein the specific item or term is pointed by the visual pointing object;

detecting a tip position of the visual pointing object from the first and second image information;

recognizing a keyword included in image data of the specific item or term from the first image information, wherein the recognized keyword is located at a particular position of the visual medium and the particular position corresponds to the tip position of the visual pointing object;

searching said database in accordance with the keyword corresponding to the image data of the specific item or term recognized by said recognizing step, so as to find explanation information for the specific item or term; and

outputting the explanation information.

12. The method of claim 11, wherein said searching step includes:

recognizing one or more characters constituting the specific item or term so as to provide a character code corresponding to the recognized characters; and

comparing the character code with the data stored in said database so as to provide the explanation information for the specific item or term.

13. The method of claim 11, wherein said outputting step includes:

displaying not only an image of the visual medium but also an image of the explanation information.

14. The method of claim 11, wherein said outputting step includes:

providing a voice service relating to the explanation information.

15. The method of claim 11, wherein said detecting step includes:

detecting a difference between the first image information and the second image information so as to extract an image of the visual pointing object;

detecting a start position of the specific item or term which start position is pointed by the visual pointing object; and

detecting an end position of the specific item or term which end position is pointed by the visual pointing object,

and wherein said recognizing step includes;

extracting an item or term within a specific area on the visual medium which specific area is defined by the start position and the end position.

16. The method of claim 11, further comprising the steps of:

recording an image of the visual medium;

reproducing the recorded image of the visual medium; and

capturing the reproduced recorded image so as to supply the captured image as the first image information to said first image obtaining step.

17. An apparatus for assisting a knowledge of a user in response to a pointing operation of the user, said apparatus comprising:

means for picking up an image of a medium;

means for pointing a prescribed position on the medium;

first memory means for storing a first image of the medium picked up by said picking up means;

second memory means for storing a second image of the medium picked up by said picking up means, said second image including an image of said pointing means;

means for detecting the prescribed position pointed by said pointing means in accordance with the first image stored in said first memory means and the second image stored in said second memory means;

means for extracting an item or term placed at the prescribed position of said medium; and

means for recognizing respective characters constituting the item or term extracted by said extracting means so as to provide a result of recognition.

18. The apparatus of claim 17, further comprising:

third memory means for storing a plurality of explanation texts corresponding to items or terms any one of which can be pointed by the user; and

means for reading out a specific text from said third memory means in accordance with the result of recognition provided by said recognizing means.