Input/output apparatus having a pen, and method of associating and processing handwritten image data and voice data

• US 5,546,565 A
• Filed: 06/01/1994
• Issued: 08/13/1996
• Est. Priority Date: 06/21/1993
• Status: Expired due to Term

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Suzuki, Hideo
• Primary Examiner(s): Teska, Kevin J.
• Assistant Examiner(s): Walker, Tyrone V.
• Application Number:
  US08/252,280
• Time in Patent Office: 804 Days
• Field of Search: 395/500, 395/200, 395/2, 395/2.6, 395/2.80, 395/2.81, 395/2.82, 395/2.84, 364/900, 382/146, 382/154, 382/186, 382/187, 382/188, 382/189, 434/112-116, 434/159, 434/169, 434/201, 434/205, 434/319, 434/320
• US Class Current: 382/187
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

An input/output apparatus records both a handwritten image and a speech to facilitate the collation between the recorded image and speech. When a command recognizing section recognizes that a handwritten image input through a coordinate input section is a voice command image, a voice processor starts recording the speech, input from a microphone, into a voice recording memory. During this recording, the voice processor sets the input handwritten image as a key image. When an end command image is written, the voice processor registers the speech, recorded in the voice recording memory, into a voice register file in association with the key image. When an image identical to the registered key image is input, the voice processor reads the speech corresponding to the key image from the voice register file into the voice recording memory, and reproduces the speech from a loudspeaker.

Claims

1. An input/output apparatus for inputting and displaying a desired handwritten image using a pen on a display screen, said apparatus comprising:

first input means for inputting an image through an operation of said pen;

second input means for inputting voices;

memory means for storing said image, input through said first input means, as image data and storing said voices, input through said second input means, as voice data;

determining means for determining whether or not image data corresponding to a new image, when the new image is input through said first input means, is stored in said memory means; and

voice output means, in response to said determining means, for reading said voice data from said memory means, converting said voice data to voices and outputting said voices, when the determining means determines that the image data corresponding to a new image is stored in said memory means; and

said memory means including means for storing plural pieces of image data and plural pieces of voice data in one-to-one association; and

said voice output means including means for, when the image data corresponding to the new image is stored in said memory means, reading voice data stored in said memory means in association with said image data, and for outputting voices acquired by converting said voice data.

2. An input/output apparatus for inputting and displaying a desired handwritten image using a pen on a display screen, said apparatus comprising:

image input means for inputting an image through an operation of said pen;

voice input means for inputting voices;

image analyzing means for analyzing a handwritten image input through said image input means;

first control means for controlling start of voice input by said voice input means in response to inputting of the image when the image analyzed by said image analyzing means is a command indicating start of voice recording;

second control means for controlling termination of voice input by said voice input means in response to inputting of the image when the image analyzed by said image analyzing means is a command indicating termination of voice recording;

voice data recording means for recording voices, input through said voice input means, as voice data after inputting of said voices by said voice input means has started and until termination of said inputting of said voices is controlled;

image data storage means for storing an image, input through said image input means, as image data after inputting of said voices by said voice input means has started and until termination of said inputting of said voices is controlled; and

voice output means for determining whether or not image data corresponding to a new image, when the new image is input through said image input means, is stored in said image data storage means, and when the image data corresponding to the new image is stored in said image data storage means, reading voice data from said voice data recording means, converting said voice data to voices and outputting said voices.

3. An input/output apparatus for inputting and displaying a desired handwritten image using a pen on a display screen, said apparatus comprising:

first input means for inputting an image through an operation of said pen;

second input means for inputting voices;

memory means for storing voices input through said second input means as voice data, and storing, as image data, an image input through said first input means in association with said voice data;

retrieval means for searching said memory means for voice data equivalent to new voices, when the new voices are input through said second input means; and

third input means for reading said image data stored in said memory means in association with said voice data retrieved by said retrieval means, and displaying an image corresponding to said image data on said display screen.

4. The input/output apparatus according to claim 3, wherein said third input means includes means for inputting an image corresponding to said image data at a display position on said display screen specified by said pen.

5. A method of associating and processing handwritten image data and voice data, comprising:

a first step of inputting a start command instructing a start of recording of voice data through an input device;

a second step of inputting an end command instructing an end of recording of the voice data through the input device;

a third step of starting recording of the voice data in response to the start command input in the first step and ending the recording in response to the end command input in the second step;

a fourth step of inputting image data for specifying the voice data recorded in the third step;

a fifth step of associating the voice data recorded in the third step with the image data input in the fourth step;

a sixth step of inputting image data substantially equivalent to the image data input in the fourth step after the step is executed; and

a seventh step of outputting the voice data which was recorded in the third step and associates with the image data input in the sixth step, in accordance with the association set in the fifth step.

6. A method of associating and processing handwritten image data and voice data, comprising:

a first step of inputting a start command instructing a start of recording of the voice data;

a second step of inputting an end command instructing an end of recording of the voice data;

a third step of starting recording of the voice data in response to the start command, and ending the recording in response to the end command;

a fourth step of inputting the image data by handwriting;

a fifth step of associating the voice data recorded in the third step and the image data input in the fourth step;

a sixth step of inputting voice data equivalent to the voice data recorded in the third step;

a seventh step of outputting the image data which was recorded in the fourth step and associating with voice data equivalent to the voice data input in the sixth step, in accordance with the association set in the fifth step.

7. A method of associating and processing handwritten image data and voice data, comprising:

a first step of recording voice data;

a second step of inputting and storing handwritten image data;

a third step of associating voice data recorded in the first step and the handwritten image data input in the second step;

a fourth step of inputting handwritten image data;

a fifth step of retrieving handwritten image data, from the handwritten image data input in the second step, which is substantially equivalent to the handwritten image data input in the fourth step;

a sixth step of retrieving voice data associated with the handwritten image data retrieved in the fifth step, in accordance with the association set in the third step; and

a seventh step of outputting the voice data retrieved in the sixth step.

8. A method of associating and processing handwritten image data and voice data, comprising:

a first step of recording voice data;

a second step of inputting handwritten image data;

a third step of associating voice data recorded in the first step and the handwritten image data input in the second step;

a fourth step of outputting voice data;

a fifth step retrieving voice data from the voice data input in the first step, which is substantially equivalent to the voice data input in the fourth step;

a sixth step of retrieving handwritten image data associated with the voice data retrieved in the fifth step, in accordance with the association set in the third step; and

a seventh step of outputting the image data retrieved in the sixth step.