Identifying matching canonical documents in response to a visual query
First Claim
1. A computer-implemented method of processing a visual query comprising:
- on a server system having one or more processors and memory storing one or more programs for execution by the one or more processors;
receiving a visual query from a client system;
performing optical character recognition (OCR) on the visual query to produce text recognition data representing a plurality of textual characters, including a plurality of textual characters in a contiguous region of the visual query;
scoring each textual character in the plurality of textual characters based on at least high quality textual character scores and low quality textual character scores of other textual characters surrounding the textual character, wherein the scoring of each textual character is based, in part, on a transition cost associated with each textual character such that each textual character will score more similarly to the other textual characters surrounding the textual character as the transition cost increases;
identifying one or more high quality text segments in the contiguous region of the visual query in accordance with a determination that a respective high quality text segment of the one or more high quality text segments comprises a plurality of high scoring textual characters;
identifying a document containing at least one high quality text segment of the one or more high quality text segments;
retrieving the document containing the at least one high quality text segment; and
sending at least a portion of the document to the client system.
2 Assignments
0 Petitions
Accused Products
Abstract
A server system receives a visual query from a client system. The visual query is an image containing text such as a picture of a document. At the receiving server or another server, optical character recognition (OCR) is performed on the visual query to produce text recognition data representing textual characters. Each character in a contiguous region of the visual query is individually scored according to its quality. The quality score of a respective character is influenced by the quality scores of neighboring or nearby characters. Using the scores, one or more high quality strings of characters are identified. Each high quality string has a plurality of high quality characters. A canonical document containing the one or more high quality textual strings is retrieved. At least a portion of the canonical document is sent to the client system.
73 Citations
22 Claims
-
1. A computer-implemented method of processing a visual query comprising:
on a server system having one or more processors and memory storing one or more programs for execution by the one or more processors; receiving a visual query from a client system; performing optical character recognition (OCR) on the visual query to produce text recognition data representing a plurality of textual characters, including a plurality of textual characters in a contiguous region of the visual query; scoring each textual character in the plurality of textual characters based on at least high quality textual character scores and low quality textual character scores of other textual characters surrounding the textual character, wherein the scoring of each textual character is based, in part, on a transition cost associated with each textual character such that each textual character will score more similarly to the other textual characters surrounding the textual character as the transition cost increases; identifying one or more high quality text segments in the contiguous region of the visual query in accordance with a determination that a respective high quality text segment of the one or more high quality text segments comprises a plurality of high scoring textual characters; identifying a document containing at least one high quality text segment of the one or more high quality text segments; retrieving the document containing the at least one high quality text segment; and sending at least a portion of the document to the client system. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
11. A server system, for processing a visual query, comprising:
-
one or more central processing units for executing programs; memory storing one or more programs be executed by the one or more central processing units; the one or more programs comprising instructions for; receiving a visual query from a client system; performing optical character recognition (OCR) on the visual query to produce text recognition data representing a plurality of textual characters, including a plurality of textual characters in a contiguous region of the visual query; scoring each textual character in the plurality of textual characters based on at least high quality textual character scores and low quality textual character scores of other textual characters surrounding the textual character, wherein the scoring of each textual character is based, in part, on a transition cost associated with each textual character such that each textual character will score more similarly to the other textual characters surrounding the textual character as the transition cost increases; identifying one or more high quality text segments in the contiguous region of the visual query in accordance with a determination that a respective high quality text segment of the one or more high quality text segments comprises a plurality of high scoring textual characters; identifying, a document containing at least one high quality text segment of the one or more high quality text segments; retrieving the document containing the at least one high quality text segment; and sending at least a portion of the document to the client system. - View Dependent Claims (12, 13, 14, 15, 16)
-
-
17. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:
-
receiving a visual query from a client system; performing optical character recognition (OCR) on the visual query to produce text recognition data representing a plurality of textual characters, including a plurality of textual characters in a contiguous region of the visual query; scoring each textual character in the plurality of textual characters based on at least high quality textual character scores and low quality textual character scores of other textual characters surrounding the textual character, wherein the scoring of each textual character is based, in part, on a transition cost associated with each textual character such that each textual character will score more similarly to the other textual characters surrounding the textual character as the transition cost increases; identifying one or more high quality text segments in the contiguous region of the visual query in accordance with a determination that a respective high quality text segment of the one or more high quality text segments comprises a plurality of high scoring textual characters; identifying, a document containing at least one high quality text segment; retrieving the document containing at least one high quality text segment; and sending at least a portion of the document to the client system. - View Dependent Claims (18, 19, 20, 21, 22)
-
Specification