SYSTEM AND METHOD FOR INPUT OF TEXT TO AN APPLICATION OPERATING ON A DEVICE

• US 20090112572A1
• Filed: 10/30/2007
• Published: 04/30/2009
• Est. Priority Date: 10/30/2007
• Status: Abandoned Application

• Current Assignee: Sony Ericsson Mobile Communications USA Incorporated (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications USA Incorporated (Sony Group Corp.)
• Inventors: THORN, Karl Ola
• Application Number:
  US11/928,162
• Publication Number:
  US 20090112572A1
• Time in Patent Office: Days
• US Class Current: 704/3
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A device comprise an a display screen and an audio circuit for generating an audio signal representing spoken words uttered by the user. A processor executes a first application, a second application, and a text mark-up object. The first application may render a depiction of text on the display screen. The text mark-up object may: i) receiving at least a portion of the audio signal representing spoken words uttered by the user; ii) performing speech recognition to generate a text representation of the spoken words uttered by the user; iii) determining a selected text segment, and iv) performing an input function to input the selected text segment to the second application. The selected text segment may be text which corresponds to both a portion of the depiction of text on the display screen and the text representation of the spoken words uttered by the user.

Claims

1. A device comprising:

a display screen;

an audio circuit for generating an audio signal representing spoken words uttered by the user; and

a processor executing a first application, a second application, and a text mark-up object;

the first application rendering a depiction of text on the display screen;

the text mark-up object;

receiving at least a portion of the audio signal representing spoken words uttered by the user;

performing speech recognition to generate a text representation of the spoken words uttered by the user;

determining a selected text segment, the selected text segment being text which corresponds to both a portion of the depiction of text on the display screen and the text representation of the spoken words uttered by the user; and

performing an input function to input the selected text segment to the second application.

2. The device of claim 1,

the text mark-up object drives rendering of a marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment;

and performs the paste function only upon detection of an input command while rendering the marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment.

3. The device of claim 2, wherein the paste command is an audio command uttered by the user and the text mark-up object detects the command within the audio signal by speech recognition.

4. The device of claim 1, wherein:

the first application is an application rendering a digital image including the depiction of text on the display screen;

the text mark-up object further performs character recognition on the depiction of text to generate a character string; and

and the selected text segment comprises text which corresponds to both a portion of the character string and the text representation of the spoken words uttered by the user.

5. The device of claim 4:

further comprising a digital camera; and

wherein the application renders an image captured by the digital camera as the image including the depiction of text on the display screen.

6. The device of claim 4,

the text mark-up object drives rendering of a marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment;

and performs the paste function only upon detection of an input command while rendering the marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment.

7. The device of claim 6, wherein the paste command is an audio command uttered by the user and the text mark-up object detects the command within the audio signal by speech recognition.

8. The device of claim 1:

further comprising a digital photograph database storing a plurality of images;

the text mark-up object further performs character recognition on text depicted in each image and associates with each image, a character string corresponding to the text depicted therein;

the first application is an application rendering a digital image including the depiction of text on the display screen; and

determining the selected text segment comprising selecting the portion of the character string associated, in the database, with the image rendered on the display screen, which corresponds to the text representation of the spoken words uttered by the user.

9. The device of claim 8,

the text mark-up object drives rendering of a marking of the portion of the depiction of text on the display screen which corresponds to the selected text;

and performs the paste function only upon input of an input command by the user while the rendering of the marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment.

10. The device of claim 9, wherein the paste command is an audio command uttered by the user and the text mark-up object detects the command within the audio signal by speech recognition.

11. The device of claim 1, wherein the selected text segment is text which corresponds to the portion of the depiction of text on the display screen that is between a first text representation of spoken words uttered by the user and a second text representation of spoken words uttered by the user.

12. A method of operating a device to select and paste a selected text segment from a first application to a second application, the method comprising:

driving the first application to render a depiction of text on a display screen;

receiving at least a portion of an audio signal representing spoken words uttered by the user;

performing speech recognition to generate a text representation of the spoken words uttered by the user; and

determining the selected text segment, the selected text segment being text which corresponds to both a portion of the depiction of text on the display screen and the text representation of the spoken words uttered by the user; and

performing an input function to input the selected text segment to the second application.

13. The method of claim 12,

further comprising rendering a marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment;

and performing the paste function only upon detection of an input command while rendering the marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment.

14. The method of claim 13, wherein the paste command is an audio command uttered by the user and recognized within the audio signal.

15. The method of claim 12, wherein:

the first application is an application rendering a digital image including the depiction of text on the display screen;

the text mark-up object further performs character recognition on the depiction of text to generate a character string; and

and the selected text segment comprises text which corresponds to both a portion of the character string and the text representation of the spoken words uttered by the user.

16. The method of claim 15,

further comprising rendering a marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment;

and performing the paste function only upon detection of an input command while rendering the marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment.

17. The method of claim 16, wherein the paste command is an audio command uttered by the user and recognized within the audio signal.

18. The method of claim 12:

the first application is an application rendering a digital image including the depiction of text on the display screen, the digital image being obtained from a database storing a plurality of digital images;

receiving at least a portion of an audio signal representing spoken words uttered by the user;

performing speech recognition to generate a text representation of the words uttered by the user;

determining the selected text segment comprising selecting the portion of the character string associated, in the database, with the image rendered on the display screen, which corresponds to the text representation of the spoken words uttered by the user; and

wherein the characters string associated, in the database, with the image rendered on the display screen is generated and written to the database during a character recognition process operated at time prior to rendering the determining the selected text segment.

19. The method of claim 18,

further comprising rendering a marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment;

and performing the paste function only upon detection of an input command while rendering the marking of the portion of the depiction of text on the display screen which corresponds to the selected text segment.

20. The method of claim 19, wherein the paste command is an audio command uttered by the user and recognized within the audio signal.

21. The method of claim 12, wherein the selected text segment is text which corresponds to the portion of the depiction of text on the display screen that is between a first text representation of spoken words uttered by the user and a second text representation of spoken words uttered by the user.