Selection biasing

• US 10,545,647 B2
• Filed: 07/25/2018
• Issued: 01/28/2020
• Est. Priority Date: 06/15/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Foerster, Jakob Nicolaus, Melendo Casado, Diego, Shires, Glen
• Primary Examiner(s): Bashore, William L
• Assistant Examiner(s): DiStefano, Gregory A
• Application Number:
  US16/044,656
• Publication Number:
  US 20190012064A1
• Time in Patent Office: 552 Days
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

In some implementations, data indicating a touch received on a proximity-sensitive display is received while the proximity-sensitive display is presenting one or more items. In one aspect, the techniques describe may involve a process for disambiguating touch selections of hypothesized items, such as text or graphical objects that have been generated based on input data, on a proximity-sensitive display. This process may allow a user to more easily select hypothesized items that the user may wish to correct, by determining whether a touch received through the proximity-sensitive display represents a selection of each hypothesized item based at least on a level of confidence that the hypothesized item accurately represents the input data.

Claims

1. A computer-implemented method performed using a system that processes selections of items, the method comprising:

providing an item of information for output at a first location on a display;

receiving data for a user input that includes image data and that was received at a second location on the display;

generating an object tag for the item of information based on objects depicted in the image data;

determining a confidence value that indicates a confidence that the item of information accurately represents a particular type of input based on a recognition process that was used to generate the object tag;

determining whether the user input represents a selection of the item of information based at least on;

(i) the confidence value, and

(ii) the second location of the user input; and

providing an indication of whether the user input represents selection of the item of information.

2. The computer-implemented method of claim 1, further comprising:

receiving, at the system, the particular type of input;

generating text for the item of information as a representation of the received particular type of input; and

determining the confidence value based on a mapping process used to generate the text.

3. The computer-implemented method of claim 1, wherein the particular type of user input includes audio data and the method further comprises:

generating a transcription as a representation of the audio data, wherein the transcription corresponds to the item of information; and

determining the confidence value based on a mapping process used to generate the transcription.

4. The computer-implemented method of claim 1, wherein generating the object tag comprises:

generating the object tag as a graphic superimposed onto an image rendered using the image data; and

causing the object tag to be positioned over an object in the image, the object in the image corresponding to a biometric feature of a person.

5. The computer-implemented method of claim 1, wherein determining whether the user input represents the selection of the item of information further comprises:

computing a distance between the first location and the second location; and

determining whether the user input represents the selection of the item of information based on the distance between the first location and the second location.

6. A computer-implemented method comprising:

providing an item of information for output at a first location on a display;

receiving data for a user input received at a second location on the display;

determining a confidence value that indicates a confidence that the item of information accurately represents a particular type of input determining whether the user input represents a selection of the item of information based at least on;

(i) the confidence value, and

(ii) the second location of the user input; and

providing an indication of whether the user input represents selection of the item of information, wherein the particular type of user input corresponds to text in a first language and the method further comprises;

generating the item of information using text in a second language as a translation of the text in a first language; and

determining the confidence value based on a translation process used to obtain the text in the second language.

7. The computer-implemented method of claim 6, further comprising:

obtaining a geographic location of a device that includes the display;

generating the item of information using text in the second language as the translation of the text in the first language based on the obtained geographic location of the device; and

determining the confidence value based on the translation process and the geographic location of the device.

8. A system comprising:

a processing device;

a non-transitory computer-readable storage device having instructions stored thereon that, when executed by the processing device, causes performance of operations comprising;

providing an item of information for output at a first location on a display;

receiving data for a user input that includes image data and that was received at a second location on the display;

generating an object tag for the item of information based on objects depicted in the image data;

determining a confidence value that indicates a confidence that the item of information accurately represents a particular type of input based on a recognition process that was used to generate the object tag;

determining whether the user input represents a selection of the item of information based at least on;

(i) the confidence value, and

(ii) the second location of the user input; and

providing an indication of whether the user input represents selection of the item of information.

9. The system of claim 8, wherein the operations further comprises:

receiving, at the system, the particular type of input;

generating text for the item of information as a representation of the received particular type of input; and

determining the confidence value based on a mapping process used to generate the text.

10. The system of claim 8, wherein the particular type of user input includes audio data and the operations further comprise:

generating a transcription as a representation of the audio data, wherein the transcription corresponds to the item of information; and

determining the confidence value based on a mapping process used to generate the transcription.

11. The system of claim 8, wherein generating the object tag comprises:

generating the object tag as a graphic superimposed onto an image rendered using the image data; and

causing the object tag to be positioned over an object in the image, the object in the image corresponding to a biometric feature of a person.

12. The system of claim 8, wherein determining whether the user input represents the selection of the item of information further comprises:

computing a distance between the first location and the second location; and

determining whether the user input represents the selection of the item of information based on the distance between the first location and the second location.

13. A system comprising:

a processing device;

a non-transitory computer-readable storage device having instructions stored thereon that, when executed by the processing device, causes performance of operations comprising;

providing an item of information for output at a first location on a display;

receiving data for a user input received at a second location on the display;

determining a confidence value that indicates a confidence that the item of information accurately represents a particular type of input;

determining whether the user input represents a selection of the item of information based at least on;

(i) the confidence value, and

(ii) the second location of the user input; and

providing an indication of whether the user input represents selection of the item of information, wherein the particular type of user input corresponds to text in a first language and the method further comprises;

generating the item of information using text in a second language as a translation of the text in a first language; and

determining the confidence value based on a translation process used to obtain the text in the second language.

14. The system of claim 13, wherein the operations further comprise:

obtaining a geographic location of a device that includes the display;

generating the item of information using text in the second language as the translation of the text in the first language based on the obtained geographic location of the device; and

determining the confidence value based on the translation process and the geographic location of the device.

15. One or more non-transitory machine-readable storage devices storing instructions that are executable by one or more processing devices to cause performance of operations comprising:

providing an item of information for output at a first location on a display;

receiving data for a user input that includes image data and that was received at a second location on the display;

generating an object tag for the item of information based on objects depicted in the image data;

determining a confidence value that indicates a confidence that the item of information accurately represents a particular type of input based on a recognition process that was used to generate the object tag;

determining whether the user input represents a selection of the item of information based at least on;

(i) the confidence value, and

(ii) the second location of the user input; and

providing an indication of whether the user input represents selection of the item of information.

16. The non-transitory machine-readable storage devices of claim 15, wherein the particular type of user input includes audio data and the operations further comprise:

generating a transcription as a representation of the audio data, wherein the transcription corresponds to the item of information; and

determining the confidence value based on a mapping process used to generate the transcription.

17. One or more non-transitory machine-readable storage devices storing instructions that are executable by one or more processing devices to cause performance of operations comprising:

providing an item of information for output at a first location on a display;

receiving data for a user input received at a second location on the display;

determining a confidence value that indicates a confidence that the item of information accurately represents a particular type of input;

determining whether the user input represents a selection of the item of information based at least on;

(i) the confidence value, and

(ii) the second location of the user input; and

providing an indication of whether the user input represents selection of the item of information, wherein the particular type of user input corresponds to text in a first language and the method further comprises;

generating the item of information using text in a second language as a translation of the text in a first language; and

determining the confidence value based on a translation process used to obtain the text in the second language.