ASSISTED TEXT INPUT FOR COMPUTING DEVICES

• US 20160028945A1
• Filed: 10/02/2015
• Published: 01/28/2016
• Est. Priority Date: 12/11/2013
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: DHUA, ARNAB SANAT KUMAR, KRAFT, ADAM WIGGEN, MA, KATHY WING LAM, LIN, XIAOFAN, LOU, YU
• Granted Patent: US 9,350,913 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06Q 30/0603 Catalogue ordering
• CPC Class Codes: G06Q 30/0625 Directed, with specific intent or strategy
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 2210/22 Cropping
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: H04N 1/00 Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Various approaches provide for detecting and recognizing text to enable a user to perform various functions or tasks. For example, a user could point a camera at an object with text, in order to capture an image of that object. The camera can be integrated with a portable computing device that is capable of taking the image and processing the image (or providing the image for processing) to recognize, identify, and/or isolate the text in order to send the image of the object as well as recognized text to an application, function, or system, such as an electronic marketplace.

Claims

1. A computing device, comprising:

a first camera;

a second camera separated a distance from the first camera;

a display element;

at least one processor; and

a memory device including instructions that, when executed by the at least one processor, cause the computing device to;

capture first image data of an object by the first camera, the object including text displayed on a surface of the object;

capture second image data of the object by the second camera;

process the first image data to recognize the text displayed on the surface of the object;

determine a set of words from the recognized text;

generate using the first image data and the second image data a three-dimensional representation of the object;

display an interface that includes a selectable list of a subset of the set of words; and

enable selection of a word from the selectable list to be displayed with the three-dimensional representation of the object.

2. The computing device of claim 1, wherein the instructions, when executed to process the first image data, further cause the computing device to:

determine a foreground region of the first image data and a background region of the first image data;

identify a representation of the object;

determine that the representation of the object is in the foreground region; and

crop the background region from the first image data.

3. The computing device of claim 1, wherein the instructions, when executed, further cause the computing device to:

receive a selection of a text input field being displayed on the interface;

receive a selection of a word from of the selectable list; and

populate the text input field with the word, wherein the text input field includes at least one of a title field, a product description field, a product search phrases field, a keyword search field, a price field, a quantity field, or a quality field.

4. The computing device of claim 3, wherein the instructions, when executed, further cause the computing device to:

receive an indication that input to the text input field is complete; and

generate a product listing using entries in the text input field.

5. The computing device of claim 4, wherein the instructions, when executed, further cause the computing device to:

provide, to an electronic marketplace, the product listing and information corresponding to the three-dimensional representation of the object.

6. The computing device of claim 1, wherein the instructions, when executed, further cause the computing device to:

cause the computing device to perform one of submitting a product listing to an electronic marketplace, copying the selection of the word to a clipboard, a notepad, or a browser, or storing the three-dimensional representation of the object or the selection of the word to a database.

7. The computing device of claim 1, wherein the instructions, when executed, further cause the computing device to:

identify words of the subset of the set of words matching a noun;

for each word of the subset of the set of words matching a particular noun, determine a number of occurrences of the particular noun;

determine a relevance score for each word of the subset of the set of words based at least in part on the number of occurrences of a matched particular noun; and

arrange the subset of the set of words based at least in part on a respective relevance score for each word.

8. The computing device of claim 1, wherein the instructions, when executed, further cause the computing device to:

process the first image data and the second image data to adjust at least one of image sharpness, image color, image brightness, image file size, image orientation, image rectification, or image scaling.

9. The computing device of claim 1, wherein the instructions, when executed, further cause the computing device to:

acquire third image data, the third image data including a view of the object that is different from the first image data and the second image data;

analyzing the third image data to determine a second set of words; and

display an interface that includes a selectable list of at least a subset of the second set of words.

10. The computing device of claim 1, wherein the instructions, when executed, further cause the computing device to:

acquire a plurality of images, each image of the plurality of images including at least a portion of a representation of the object; and

determine, based at least in part on the plurality of images, at least one of two or more views of the object, each view of the two of more views being acquired from a different point of view, or physical dimensions of the object.

11. A computer-implemented method, comprising:

capturing first image data of an object by a first camera of a computing device, the object including text displayed on a surface of the object;

capturing second image data of the object by a second camera of the computing device;

processing the first image data to recognize the text displayed on the surface of the object;

determining a set of words from the recognized text;

generating using the first image data and the second image data a three-dimensional representation of the object;

displaying an interface that includes a selectable list of a subset of the set of words; and

enabling selection of at least one word from one of the selectable list to be displayed with the three-dimensional representation of the object.

12. The computer-implemented method of claim 11, further comprising:

determining a foreground region of the first image data and a background region of the first image data;

identifying a representation of the object;

determining that the representation of the object is in the foreground region; and

cropping the background region from the first image data.

13. The computer-implemented method of claim 11, further comprising:

receiving a selection of a text input field displayed on the interface;

receiving a selection of a word from of the selectable list; and

populating the text input field with the word, wherein the text input field includes at least one of a title field, a product description field, a product search phrases field, a keyword search field, a price field, a quantity field, or a quality field.

14. The computer-implemented method of claim 13, further comprising:

receiving an indication that input to the text input field is complete;

generating a product listing being based at least in part on entries in the text input field; and

providing, to an electronic marketplace, the product listing and information of the three-dimensional representation of the object.

15. The computer-implemented method of claim 11, further comprising:

causing the computing device to perform one of submitting a product listing to an electronic marketplace, copying the selection of the word to a clipboard, a notepad, or a browser, or storing the three-dimensional representation of the object or the selection of the word to a database.

16. The computer-implemented method of claim 11, further comprising:

identifying words of the subset of the set of words matching a noun;

for each word of the subset of the set of words matching a particular noun, determine a number of occurrences of the particular noun;

determining a relevance score for each word of the subset of the set of words based at least in part on the number of occurrences of a matched particular noun; and

arranging the subset of the set of words based at least in part on a respective relevance score for each word.

17. A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of operations comprising:

capturing first image data of an object by a first camera of a computing device, the object including text displayed on a surface of the object;

capturing second image data of the object by a second camera of the computing device;

processing the first image data to recognize the text displayed on the surface of the object;

determining a set of words from the recognized text;

generating using the first image data and the second image data a three-dimensional representation of the object;

displaying an interface that includes a selectable list of a subset of the set of words; and

enabling selection of at least one word from one of the selectable list to be displayed with the three-dimensional representation of the object.

18. The non-transitory computer readable storage medium of claim 17, further comprising instructions executed by the one or more processors to perform the set of operations of:

acquiring a plurality of images, each image of the plurality of images including at least a portion of a representation of the object; and

determining, based at least in part on the plurality of images, at least one of two or more views of the object, each view of the two of more views being acquired from a different point of view or physical dimensions of the object.

19. The non-transitory computer readable storage medium of claim 17, further comprising instructions executed by the one or more processors to perform the set of operations of:

determining a foreground region of the first image data and a background region of the first image data;

identifying a representation of the object;

determining that the representation of the object is in the foreground region; and

cropping the background region from the first image data.

20. The non-transitory computer readable storage medium of claim 17, further comprising instructions executed by the one or more processors to perform the set of operations of:

receiving a selection of a text input field displayed on the interface;

receiving a selection of a word from of the selectable list; and

populating the text input field with the word, wherein the text input field includes at least one of a title field, a product description field, a product search phrases field, a keyword search field, a price field, a quantity field, or a quality field.