Automated highlighting of identified text

• US 9,870,633 B2
• Filed: 12/21/2016
• Issued: 01/16/2018
• Est. Priority Date: 09/25/2013
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Kraft, Adam Wiggen, Dhua, Arnab Sanat Kumar, Gray, Douglas Ryan, Lin, Xiaofan, Lou, Yu, Ramesh, Sunil, Taylor, Colin Jon, Mott, David Creighton
• Primary Examiner(s): YANG, ANDREW GUS
• Application Number:
  US15/387,219
• Publication Number:
  US 20170103560A1
• Time in Patent Office: 391 Days
• Field of Search: None
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/80 Recognising image objects characterised by unique random patterns

Abstract

Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

Claims

1. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:

receive a user preference, the user preference selected for an individual user;

access a user identified word of a plurality of user identified words, the user identified word associated with the user preference and a color selected for the individual user;

obtain image data that includes a representation of text;

use an object recognition algorithm to determine a recognized word represented in the image data;

associate the user identified word with the recognized word;

display the recognized word emphasized by a visual color attribute, the visual color attribute matching the color associated with the user identified word; and

arrange the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

2. The non-transitory computer-readable storage medium of claim 1, wherein the visual color attribute comprises at least one of a bounding box around the recognized word, a graphical representation of the recognized word in color, or a highlighting of the recognized word with a color.

3. The non-transitory computer-readable storage medium of claim 2, wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

receive the plurality of user identified words from a user; and

enable the user to assign a first overlay to a first set of words and a second overlay to a second set of words of the plurality of user identified words.

4. The non-transitory computer-readable storage medium of claim 3, wherein the first overlay is different in appearance relative to the second overlay.

5. The non-transitory computer-readable storage medium of claim 1, wherein an overlay associated with each occurrence of the recognized word moves in corresponding motion with at least one word of the representation of text on a display element in a live field of view of a camera.

6. The non-transitory computer-readable storage medium of claim 1, wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

identify a category associated with the recognized word; and

apply, based at least in part on the identified category, a list of one or more user identified words.

7. A computer-implemented method, comprising:

receiving a user preference, the user preference selected for an individual user;

accessing a user identified word of a plurality of user identified words, the user identified word associated with user preference and a color selected for the individual user;

obtaining image data that includes a representation of text;

using an object recognition algorithm to determine a recognized word represented in the image data;

associating the user identified word with the recognized word;

displaying the recognized word emphasized by a visual color attribute, the visual color attribute matching the color associated with the user identified word; and

arranging the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

8. The computer-implemented method of claim 7, wherein the image data is obtained by a camera, and wherein the image data is displayed on a display element as at least one of a live field of view of the camera or a real-time video captured by the camera.

9. The computer-implemented method of claim 8, wherein the visual color attribute comprises displaying at least one of a bounding box around the recognized word, or displaying a graphical representation of the recognized word.

10. The computer-implemented method of claim 8, wherein the visual color attribute comprises displaying a graphical overlay at least one of above or adjacent the recognized word.

11. The computer-implemented method of claim 10, wherein the graphical overlay moves in corresponding motion with the recognized word on the display element in the live field of view of the camera.

12. The computer-implemented method of claim 10, wherein the graphical overlay includes an additional visual attribute operable to cause the recognized word to be emphasized relative to a remainder of words of the representation of text displayed on the display element.

13. The computer-implemented method of claim 10, wherein the graphical overlay further comprises at least one of a bounding box around the recognized word, a graphical representation of the recognized word in color, or color highlighting.

14. The computer-implemented method of claim 10, further comprising:

receiving the plurality of user identified words from a user; and

enabling the user to assign an overlay to individual ones of the plurality of user identified words.

15. The computer-implemented method of claim 7, further comprising:

identifying a category associated with the recognized word; and

applying a list of user identified words based at least in part on the identified category.

16. A computing device, comprising:

a processor;

a camera;

a display screen; and

memory including instructions that, when executed by the processor, cause the computing device to;

receive a user preference associated with behavior of an individual user;

access a user identified word of a plurality of user identified words, the user identified word associated with the user preference and a color selected for the individual user;

obtain image data that includes a representation of text;

use an object recognition algorithm to determine a recognized word represented in the image data;

associated the user identified word with the recognized word;

display associated with the recognized word a visual color attribute, the visual color attribute matching the color associated with the user identified word; and

arrange the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

17. The computing device of claim 16, wherein the instructions, when executed by the processor, further enable the computing device to:

identify a category associated with the recognized word; and

determine a user identified word list based at least in part on the identified category.

18. The computing device of claim 16, wherein the visual color attribute comprises displaying at least one of a bounding box around the recognized word, displaying a graphical representation of the recognized word, or color highlighting.

19. The computing device of claim 16, wherein the visual color attribute comprises displaying a graphical overlay above or adjacent to the recognized word.

20. The computing device of claim 19, wherein the graphical overlay above or adjacent the recognized word moves in corresponding motion with the recognized word on the display screen in a live field of view of the camera.