Digital image archiving and retrieval in a mobile device system

• US 7,986,843 B2
• Filed: 11/29/2006
• Issued: 07/26/2011
• Est. Priority Date: 11/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Phukan, Prasenjit, Saraf, Arvind, Chaudhury, Krishnendu, Garg, Ashutosh
• Primary Examiner(s): Ahmed; Samir A
• Assistant Examiner(s): Li; Ruiping
• Application Number:
  US11/564,823
• Publication Number:
  US 20080126415A1
• Time in Patent Office: 1,700 Days
• Field of Search: 707/104.1, 707/E17.032
• US Class Current: 382/229
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

A computer-implemented method of managing information is disclosed. The method can include receiving a message from a mobile device configured to connect to a mobile device network (the message including a digital image taken by the mobile device and including information corresponding to words), determining the words from the digital image information using optical character recognition, indexing the digital image based on the words, and storing the digital image for later retrieval of the digital image based on one or more received search terms.

Claims

1. A computer-implemented method of managing information, the method comprising:

receiving a message from a mobile device configured to connect to a mobile device network, the mobile device comprising a digital camera, and the message comprising a digital image taken by the digital camera and including information corresponding to words, wherein receiving the message comprises receiving an indication of type for a document represented in the digital image, the indication of type comprising a user specified category selected from a group including business cards and credit card receipts;

determining the words from the digital image information using optical character recognition;

post-processing the words to identify and correct common character misidentifications resulting from the optical character recognition, the post-processing comprising post-processing the words in accordance with a dictionary based language model selected from at least two dictionary based language models according to the indication of type for the document;

indexing the digital image based on the words; and

storing the digital image for later retrieval of the digital image based on one or more received search terms.

2. The method of claim 1, further comprising validating the mobile device.

3. The method of claim 1, further comprising:

receiving the one or more search terms; and

retrieving the digital image based on the one or more search terms.

4. The method of claim 1, wherein receiving the message comprises receiving an email message having the digital image attached;

the method further comprises adding at least one of the words, and a pre-defined label corresponding to the mobile device, to the email message; and

wherein the determining, indexing and storing are performed in an electronic mail system.

5. The method of claim 1, wherein receiving the digital image comprises receiving at least two digital images taken of a single object in response to a single input to the digital camera, and determining the words comprises performing correlative optical character recognition on the at least two digital images to find the words.

6. The method of claim 1, wherein determining the words comprises performing the optical character recognition at multiple scales.

7. The method of claim 1, further comprising pre-processing the digital image to improve the optical character recognition.

8. The method of claim 7, wherein the pre-processing comprises identifying a binarization threshold for the digital image by minimizing positional variance of left and right margins of a document represented in the digital image.

9. The method of claim 7, wherein the pre-processing comprises obtaining a gray level at a higher resolution pixel by iteratively taking a weighted combination of gray levels of neighboring pixels at a lower resolution.

10. A system comprising:

a mobile device network;

a plurality of mobile devices configured to take digital images, connect to the mobile device network, and transmit the digital images over the mobile device network;

one or more computers configured to receive the digital images from the mobile devices, apply optical character recognition to extract words from the digital images, index the digital images based on the extracted words, and store the digital images for later retrieval based on received search terms;

wherein the one or more computers are configured to receive indications of document type along with the digital images, select between at least two dictionary based language models according to the indications of document type, and post-process the extracted words in accordance with the selected dictionary based language model; and

wherein an indication of document type comprises a user specified category selected from a group including business cards and credit card receipts.

11. The system of claim 10, wherein the one or more computers comprise a first back-end component and a second back-end component, the first back-end component configured to receive the digital images, validate the mobile devices and apply the optical character recognition, and the second back-end component configured to index the digital images and store the digital images.

12. The system of claim 11, wherein the second back-end component comprises an electronic mail system.

13. The system of claim 10, wherein the mobile devices comprise mobile phones, and the mobile device network comprises a mobile phone network.

14. The system of claim 10, wherein the one or more computers comprise a personal computer.

15. The system of claim 10, wherein the one or more computers comprise a search appliance.

16. The system of claim 10, wherein the one or more computers are configured to validate the mobile devices based on mobile phone numbers associated with the mobile devices.

17. The system of claim 10, wherein the one or more computers are configured to receive the search terms, and retrieve the digital images based on the search terms.

18. The system of claim 10, wherein the one or more computers are configured to add extracted words and a pre-defined label to messages including the digital images.

19. The system of claim 10, wherein the one or more computers are configured to perform correlative optical character recognition.

20. The system of claim 10, wherein the one or more computers are configured to perform the optical character recognition at multiple scales.

21. The system of claim 10, wherein the one or more computers are configured to pre-process the digital images to improve the optical character recognition.

22. The system of claim 10, wherein the one or more computers are configured to identify a binarization threshold for a digital image by minimizing positional variance of left and right margins of a document represented in the digital image.

23. The system of claim 10, wherein the one or more computers are configured to obtain a gray level at a higher resolution pixel by iteratively taking a weighted combination of gray levels of neighboring pixels at a lower resolution.

24. A system comprising:

a mobile device network configured to transmit digital images;

a server environment configured to provide electronic search service over a computer network; and

means for connecting the mobile device network with the server environment, the means for connecting comprising means for applying optical character recognition to extract words from the digital images and means for providing the extracted words and the digital images to the server environment for electronic search service of the digital images via the computer network;

wherein the means for applying comprises means for selecting between at least two dictionary based language models according to received indications of document type, and means for post-processing the extracted words in accordance with the selected dictionary based language model; and

wherein an indication of document type comprises a user specified category selected from a group including business cards and credit card receipts.

25. The system of claim 24, wherein the means for connecting comprises means for validating mobile devices in the mobile device network.

26. The system of claim 24, wherein the means for providing comprises means for adding extracted words and a pre-defined label to messages including the digital images.

27. The system of claim 24, wherein the means for applying comprises means for performing correlative optical character recognition.

28. The system of claim 24, wherein the means for applying comprises means for performing the optical character recognition at multiple scales.

29. The system of claim 24, wherein the means for applying comprises means for pre-processing the digital images to improve the optical character recognition.

30. The system of claim 24, wherein the means for applying comprises means for identifying a binarization threshold for a digital image by minimizing positional variance of left and right margins of a document represented in the digital image.

31. The system of claim 24, wherein the means for applying comprises means for obtaining a gray level at a higher resolution pixel by iteratively taking a weighted combination of gray levels of neighboring pixels at a lower resolution.