Selection biasing

• US 10,048,842 B2
• Filed: 06/15/2015
• Issued: 08/14/2018
• Est. Priority Date: 06/15/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Foerster, Jakob Nicolaus, Melendo Casado, Diego, Shires, Glen
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): Distefano, Gregory A
• Application Number:
  US14/739,853
• Publication Number:
  US 20160364118A1
• Time in Patent Office: 1,156 Days
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

In some implementations, data indicating a touch received on a proximity-sensitive display is received while the proximity-sensitive display is presenting one or more items. In one aspect, the techniques describe may involve a process for disambiguating touch selections of hypothesized items, such as text or graphical objects that have been generated based on input data, on a proximity-sensitive display. This process may allow a user to more easily select hypothesized items that the user may wish to correct, by determining whether a touch received through the proximity-sensitive display represents a selection of each hypothesized item based at least on a level of confidence that the hypothesized item accurately represents the input data.

Claims

1. A computer-implemented method comprising:

providing a text object for output at a first location on a proximity-sensitive display;

receiving data indicating a touch received at a second location on the proximity-sensitive display;

determining a confidence value that reflects an input-to-text engine'"'"'s confidence that text associated with the text object accurately represents an input;

determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the first location of the text object on the proximity-sensitive display, and (iii) the second location of the touch on the proximity-sensitive display; and

providing an indication of whether the touch received through the proximity-sensitive display represents a selection of the text.

2. The computer-implemented method of claim 1, comprising:

receiving, by the input-to-text engine, input;

generating, by the input-to-text engine, text based on the received input; and

wherein determining the confidence value that reflects the input-to-text engine'"'"'s confidence that text associated with the text object accurately represents an input comprises determining a confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents the received input.

3. The computer-implemented method of claim 2, wherein:

receiving, by the input-to-text engine, input comprises receiving, by the input-to-text engine, at least one image of a document; and

determining the confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents the received input comprises determining a confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents one or more textual characters included in the document.

4. The computer-implemented method of claim 2, wherein:

receiving, by the input-to-text engine, input comprises receiving, by the input-to-text engine, audio data that encodes an utterance; and

determining the confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents the received input comprises determining a confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents one or more terms spoken in the utterance.

5. The computer-implemented method of claim 2, wherein:

receiving, by the input-to-text engine, input comprises receiving, by the input-to-text engine, text in a first language;

generating, by the input-to-text engine, text based on the received input comprises generating, by the input-to-text engine, text in a second language based on the received text in the first language; and

determining the confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents the received input comprises determining a confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text is an accurate translation of the received text.

6. The computer-implemented method of claim 1, comprising:

determining a distance between the first location and the second location; and

wherein determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the first location of the text object on the proximity-sensitive display, and (iii) the second location of the touch on the proximity-sensitive display, comprises determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the distance between the first location and the second location.

7. The computer-implemented method of claim 1, comprising:

providing another text object for output at a third location on the proximity-sensitive display;

determining a confidence value that reflects an input-to-text engine'"'"'s confidence that text associated with the other text object accurately represents another input;

wherein determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the first location of the text object on the proximity-sensitive display, and (iii) the second location of the touch on the proximity-sensitive display, comprises determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the first location of the text object on the proximity-sensitive display, (iii) the second location of the touch on the proximity-sensitive display, (iv) the confidence value that reflects the input-to-text engine'"'"'s confidence that the other text is an accurate representation of the other input, and (v) the third location of the other text object on the proximity-sensitive display.

8. The computer-implemented method of claim 1, wherein the first location of the text object on the proximity-sensitive display corresponds to a location of the text object'"'"'s centroid on the proximity-sensitive display.

9. The computer-implemented method of claim 1, wherein the first location of the text object on the proximity-sensitive display corresponds to a location of a portion of the text object'"'"'s bounding box on the proximity-sensitive display.

10. A computer-implemented method comprising:

providing an item for output at a first location on a proximity-sensitive display;

receiving data indicating a touch received at a second location on the proximity-sensitive display;

determining a confidence value that reflects a mapping engine'"'"'s confidence that the item accurately represents an input;

determining whether the touch received through the proximity-sensitive display represents a selection of the item based at least on (i) the confidence value that reflects the mapping engine'"'"'s confidence that the item is an accurate representation of the input, (ii) the first location of the item on the proximity-sensitive display, and (iii) the second location of the touch on the proximity-sensitive display; and

providing an indication of whether the touch received through the proximity-sensitive display represents a selection of the item.

11. The computer-implemented method of claim 10, comprising:

providing another item for output at a third location on the proximity-sensitive display;

determining a confidence value that reflects the mapping engine'"'"'s confidence that the other item accurately represents another input;

wherein determining whether the touch received through the proximity-sensitive display represents a selection of the item based at least on (i) the confidence value that reflects the mapping engine'"'"'s confidence that the item is an accurate representation of the input, (ii) the first location of the item on the proximity-sensitive display, and (iii) the second location of the touch on the proximity-sensitive display, comprises determining whether the touch received through the proximity-sensitive display represents a selection of the item based at least on (i) the confidence value that reflects the mapping engine'"'"'s confidence that the item is an accurate representation of the input, (ii) the first location of the item on the proximity-sensitive display, (iii) the second location of the touch on the proximity-sensitive display, (iv) the confidence value that reflects the mapping engine'"'"'s confidence that the other item is an accurate representation of the other input, and (v) the third location of the other item on the proximity-sensitive display.

13. The storage device of claim 11, wherein the operations further comprise:

receiving, by the input-to-text engine, input;

generating, by the input-to-text engine, text based on the received input; and

wherein determining the confidence value that reflects the input-to-text engine'"'"'s confidence that text associated with the text object accurately represents an input comprises determining a confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents the received input.

17. The storage device of claim 11, wherein the operations further comprise:

determining a distance between the first location and the second location; and

wherein determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the first location of the text object on the proximity-sensitive display, and (iii) the second location of the touch on the proximity-sensitive display, comprises determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the distance between the first location and the second location.

12. A non-transitory computer-readable storage device having instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations comprising:

providing a text object for output at a first location on a proximity-sensitive display;

receiving data indicating a touch received at a second location on the proximity-sensitive display;

determining a confidence value that reflects an input-to-text engine'"'"'s confidence that text associated with the text object accurately represents an input;

determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the first location of the text object on the proximity-sensitive display, and (iii) the second location of the touch on the proximity-sensitive display; and

providing an indication of whether the touch received through the proximity-sensitive display represents a selection of the text.

14. The storage device of claim 12, wherein:

receiving, by the input-to-text engine, input comprises receiving, by the input-to-text engine, at least one image of a document; and

determining the confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents the received input comprises determining a confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents one or more textual characters included in the document.

15. The storage device of claim 12, wherein:

receiving, by the input-to-text engine, input comprises receiving, by the input-to-text engine, audio data that encodes an utterance; and

determining the confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents the received input comprises determining a confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents one or more terms spoken in the utterance.

16. The storage device of claim 12, wherein:

receiving, by the input-to-text engine, input comprises receiving, by the input-to-text engine, text in a first language;

generating, by the input-to-text engine, text based on the received input comprises generating, by the input-to-text engine, text in a second language based on the received text in the first language; and

determining the confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text accurately represents the received input comprises determining a confidence value that reflects the input-to-text engine'"'"'s confidence that the generated text is an accurate translation of the received text.

18. The storage device of claim 12, wherein the operations further comprise:

providing another text object for output at a third location on the proximity-sensitive display;

determining a confidence value that reflects an input-to-text engine'"'"'s confidence that text associated with the other text object accurately represents another input;

wherein determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the first location of the text object on the proximity-sensitive display, and (iii) the second location of the touch on the proximity-sensitive display, comprises determining whether the touch received through the proximity-sensitive display represents a selection of the text based at least on (i) the confidence value that reflects the input-to-text engine'"'"'s confidence that the text is an accurate representation of the input, (ii) the first location of the text object on the proximity-sensitive display, (iii) the second location of the touch on the proximity-sensitive display, (iv) the confidence value that reflects the input-to-text engine'"'"'s confidence that the other text is an accurate representation of the other input, and (v) the third location of the other text object on the proximity-sensitive display.

19. The storage device of claim 12, wherein the first location of the text object on the proximity-sensitive display corresponds to a location of the text object'"'"'s centroid on the proximity-sensitive display.

20. The storage device of claim 12, wherein the first location of the text object on the proximity-sensitive display corresponds to a location of a portion of the text object'"'"'s bounding box on the proximity-sensitive display.