Text recognition for search results

• US 9,934,526 B1
• Filed: 06/27/2013
• Issued: 04/03/2018
• Est. Priority Date: 06/27/2013
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Dhua, Arnab Sunat Kumar, Gray, Douglas Ryan, Lin, Xiaofan, Lou, Yu, Kraft, Adam Wiggen, Ramesh, Sunil
• Primary Examiner(s): Misiaszek, Michael
• Assistant Examiner(s): Ramphal, Latasha D
• Application Number:
  US13/929,689
• Time in Patent Office: 1,741 Days
• Field of Search: 705 261- 272
• CPC Class Codes: G06Q 30/0623 Item investigation

Abstract

Various embodiments enable a process to automatically attempt to select the most relevant words associated with products available for purchase from an electronic marketplace from an image frame. For example, an image frame containing text can be obtained and analyzed with an optical character recognition. The recognized words can then be preprocessed using various filtering and scoring techniques to narrow down a volume of text to a few relevant query terms. These query terms can then be sent to a search engine associated with the electronic marketplace to return relevant products to a user.

Claims

1. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:

obtain at least one image frame containing text captured using a camera of the computing device;

cause the text within the at least one image frame to be recognized with an optical character recognition (OCR) engine, an output of the OCR engine including recognized text strings and a score for each text string associated with a respective recognition confidence;

filter the recognized text strings from the output of the OCR engine that are at least one of a determined distance from an edge of the at least one image frame or are associated with at least two lines of the text to generate a set of text strings to generate a set of filtered text strings;

adjust the score of each text string as a function of distance from a center of the at least one image frame, the score associated with a text string near the center being adjusted upward relative to a text string closer to an edge of the at least one image frame;

rank the set of filtered text strings according to the score for each text string;

compare each text string of the set of filtered text strings to content references associated with content items;

identify a combined threshold indicating a number of matches or approximate matches to within an allowable deviation between the ranked set of filtered text strings and the content references, the content references including text strings corresponding to identifying features of the content;

submit the ranked set of filtered text strings associated with the identified number of matches or the approximate matches within the allowable deviation to a search engine to return content search results for the ranked set of filtered text strings;

compare the ranked set of filtered text strings to title text strings of the content search results; and

provide a respective content search result for display on the computing device when a number of text strings from the ranked set of filtered text strings appearing in a title text string of the respective content is at least equal to or exceeds the combined threshold.

2. The non-transitory computer-readable storage medium of claim 1, wherein the determined distance from an edge of the at least one image frame is a function of height of a respective text string.

3. The non-transitory computer-readable storage medium of claim 1, wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

compare each of the set of filtered text strings to a list of stop words, the list of stop words including words unassociated with content; and

adjust the score of each text string in a line of text strings downward when at least one of the text strings in the line is a stop word.

4. The non-transitory computer-readable storage medium of claim 1, wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

adjust the score of each text string as a function of text size relative to other text strings of the set of filtered text strings, the score being adjusted upward for larger font sizes relative to smaller font sizes.

5. The non-transitory computer-readable storage medium of claim 1, wherein identifying the approximate matches within the allowable deviation includes:

determining a match score for each of the approximate matches relative to a respective matching content reference based on a Levenshtein distance; and

based at least in part on a match score for a respective approximate match being above a matching threshold, determining that the respective approximate match matches a respective content reference.

6. A computer-implemented method, comprising:

receiving an output from a character recognition engine;

removing text strings from the output associated with descriptive text to generate a set of candidate text strings, the descriptive text including text strings associated with two or more lines of text from the output;

comparing the set of candidate text strings to content references associated with content, the content references including text strings corresponding to identifying features of the content;

determining a score for each candidate text string of the set of candidate text strings, the score for each candidate text string determined as a function of distance from a center of at least one image frame processed by the character recognition engine, the score associated with a candidate text string near the center being adjusted upward relative to a text string closer to an edge of the at least one image frame;

identifying a number of semantically relevant candidate text strings of the set at least approximately matching at least one content reference text string to within an allowable deviation, the identified number of semantically relevant candidate text strings at least equaling or exceeding a combined threshold indicating a number of matches or approximate matches to within the allowable deviation; and

submitting the identified number of semantically relevant candidate text strings for content search to at least one content database.

7. The computer-implemented method of claim 6, further comprising:

scoring each text string of the set of candidate text strings based at least in part on at least one of font size or the distance from a center of an image frame associated with the output from the recognition engine.

8. The computer-implemented method of claim 7, wherein the identified number of semantically relevant candidate text strings correspond to highest scoring text strings of the set of candidate text string at least approximately matching the at least one content reference to within the allowable deviation.

9. The computer-implemented method of claim 6, wherein the at least one content reference is determined by generating a list of at least one of noun phrases, brand names, or content numbers associated with the content.

10. The computer-implemented method of claim 7, further comprising:

comparing each of the set of candidate text strings to a list of stop words, the list of stop words including words unassociated with the content; and

lowering scores of text strings in a line of text strings containing a stop word from the list of stop words.

11. The computer-implemented method of claim 9, further comprising:

classifying each text string of the set of candidate text strings at least approximately matching at least one content reference to within the allowable deviation as one of a noun, a brand name, or a model number.

12. The computer-implemented method of claim 11, wherein identified candidate text strings are submitted when a determined number of at least one of nouns, brand names, or model numbers are identified.

13. The computer-implemented method of claim 6, further comprising:

determining a match score for each of the approximately matching text strings relative to a respective content reference based on a customized Levenshtein distance; and

based at least in part on a match score for a respective approximate match being above a matching threshold, determining that the respective approximate match matches a respective content reference.

14. The computer-implemented method of claim 13, wherein the customized Levenshtein distance uses a modified confusion matrix of text characters, the modified confusion matrix being determined by reducing an edit distance between at least one text character of a standard confusion matrix of text characters.

15. The computer-implemented method of claim 6, further comprising:

obtaining, from the at least one content database, content search results for the content;

comparing the identified number of candidate text strings to the content search results; and

providing a respective content search result for display on a computing device when a threshold number of the identified number of candidate text strings appear in a title of the respective content.

16. A computing device, comprising:

a processor;

an imagining element;

a display element; and

memory including instructions that, when executed by the processor, cause the computing device to;

capture, using the imagining element, an image of text;

cause the text within the image to be recognized with a character recognition engine;

receive, from the character recognition engine, a score for each text string associated with the text, each score corresponding to a level of recognition confidence for a respective text string;

adjust the score for each text string based on a function of distance from a center of the image, the score associated with a text string near the center being adjusted upward relative to a text string closer to an edge of the image;

filter out at least a portion of the text that is at least one of a determined distance from an edge of the image or are associated with a determined volume of text;

compare the text to content references associated with content, the content references including text strings corresponding to corresponding to identifying features of the content;

identify a determined number of text strings matching content reference text strings at least equaling or exceeding a combined threshold, the combined threshold indicating a number of matches or approximate matches to within an allowable deviation; and

submit the determined number of text strings to at least one content database.

17. The computing device of claim 16, wherein the instructions that, when executed by the processor, further cause the computing device to:

adjust the score for each text string in a line of text associated with at least two lines of text downward.

18. The computing device of claim 16, wherein the instructions that, when executed by the processor, further cause the computing device to:

adjust the score for each text string of the text based at least in part on font size.

19. The computing device of claim 18, wherein the instructions that, when executed by the processor, further cause the computing device to:

compare each text string to a list of stop words including words unassociated with the content; and

lowering scores of text strings in a line of text strings containing a stop word from the list of stop words.