Method and apparatus for capturing and processing content from context sensitive documents on a mobile device

• US 10,140,257 B2
• Filed: 08/02/2013
• Issued: 11/27/2018
• Est. Priority Date: 08/02/2013
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Inventors: Kannan, Anu, Epshteyn, Alan J, He, Duanfeng
• Primary Examiner(s): Bloomquist, Keith
• Assistant Examiner(s): Khan, Shahid
• Application Number:
  US13/957,751
• Publication Number:
  US 20150040002A1
• Time in Patent Office: 1,943 Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables

Abstract

A mobile device captures information from a document. The mobile device selects a template stored in a memory of the mobile device. The selected template describes data content of at least one document. The mobile device then captures an image of a document, determines that the image corresponds to one document described by the selected template by verifying that at least one first region in the image corresponds to a region of the document described by the selected template, identifies at least one region in the image that corresponds with a region in the document described by the selected template, and processes information in the identified region in the image. The mobile device then aggregates application specific information from the processed information and presents the aggregated application specific information to a user program operating on the mobile device.

Claims

1. A method, comprising:

selecting, by a mobile device based on at least one run-time characteristic of the mobile device, a template stored in a memory of the mobile device, the selected template describes data content of at least one document;

capturing, on the mobile device, an image of a document;

determining, by the mobile device, that the image corresponds to the at least one document described by the selected template based on at least one of an operating condition at the mobile device and contextual information in the document by verifying that at least one region in the image corresponds to the at least one document described by the selected template;

identifying, by the mobile device, the at least one region in the image in correspondence with a region in the at least one document described by the selected template based on at least one of the operating condition at the mobile device and the contextual information in the document;

processing information in the identified region in the image based at least in part on a location of the mobile device, including validating accuracy of a postal address in the identified region in the image based on the location of the mobile device and routing a package to the postal address in the identified region in the image when the location of the mobile device does not correspond to the postal address;

aggregating, by the mobile device, application specific information from the processed information based on at least one of the operating condition at the mobile device and contextual information in the document; and

presenting, by the mobile device, aggregated application specific information to a user program operating on one of the mobile device and a remote device connected with the mobile device.

2. The method of claim 1, wherein the processing information in the identified region in the image comprises processing information in the identified region based on at least one network signal received by the mobile device.

3. The method of claim 1, wherein the processing information in the identified region in the image comprises at least one of recognizing and processing alphanumeric characters, converting check marks or boxes to numerical values, processing an image, converting text to audio, and performing customized decoding on each identified region based on an application being executed on the mobile device.

4. The method of claim 1, wherein the processing information in the identified region in the image comprises performing customized decoding on each identified region based on at least one of the location, an orientation and a proximity of the mobile device.

5. The method of claim 1, wherein the processing information in the identified region in the image comprises performing customized decoding on each identified region based on a coverage area of a network with which the mobile device is associated.

6. The method of claim 1, wherein the processing information in the identified region in the image comprises at least one of:

decoding at least one barcode in the identified region and capturing at least one picture associated with the at least one barcode in one decode session, refining the image of the identified region, converting a checked state of at least one check box in the identified region to a numerical value, and identifying alphanumeric characters in the identified region in the image.

7. The method of claim 1, wherein responsive to the presenting, at least one of:

receiving, on the mobile device, feedback on the aggregated application specific information from the user program; and

selecting, by at least one of the user program and a processor on the mobile device, at least one predefined rule for using the aggregated information, based on at least one of the operating condition at the mobile device and the contextual information in the document.

8. The method of claim 1, further comprising associating, by the mobile device, the aggregated application specific information with at least one predefined confidence level and selecting, by at least one of the user program and a processor on the mobile device, at least one predefined rule for using the aggregated information, based on the at least one predefined confidence level.

9. The method of claim 1, further comprising storing information retrieved from the document and using the stored information during processing of another document.

10. The method of claim 1, further comprising:

selecting, by a processor on the mobile device, at least one predefined rule for using the aggregated application specific information; and

executing the at least one predefined rule subsequent to generating the application specific information.

11. A mobile device comprising:

an image capturing device configured to capture information from a document;

a memory for storing a plurality of templates at least one of which is associated with the document;

a processor configured to;

select, prior to the document being captured by the image capturing device and based on at least one run-time characteristic of the mobile device, a template stored in a memory of the mobile device, the selected template describes data content of at least one document;

determine, subsequent to the image of the document being captured by the image capturing device, that the image corresponds to the at least one document described by the selected template based on at least one of an operating condition at the mobile device and contextual information in the document by verifying that at least one region in the image corresponds to the at least one document described by the selected template;

identify at least one region in the image that corresponds with a region in the at least one document described by the selected template based on at least one of the operating condition at the mobile device and the contextual information in the document;

process information in the identified region in the image based at least in part on a location of the mobile device, including validate accuracy of a postal address in the identified region in the image based on the location of the mobile device and route a package to the postal address in the identified region in the image when the location of the mobile device does not correspond to the postal address;

aggregate application specific information from the processed information based on at least one of the operating condition at the mobile device and contextual information in the document; and

present aggregated application specific information to a user program operating on one of the mobile device and a remote device connected with the mobile device.

12. The mobile device of claim 11, wherein the processor is further configured to the process information in the identified region in the image based on at least one network signal received by the mobile station.

13. The mobile device of claim 11, wherein the processor is further configured to process information in the identified region in the image by performing at least one of recognizing and processing alphanumeric characters, converting check marks or boxes to numerical values, processing an image, converting text to audio, and performing customized decoding on each identified region based on an application being executed on the mobile device.

14. The mobile device of claim 11, wherein the processor is further configured to process information in the identified region in the image by performing customized decoding on each identified region based on at least one of a location, an orientation and a proximity of the mobile device.

15. The mobile device of claim 11, wherein the processor is further configured to process information in the identified region in the image by performing customized decoding on each identified region based on a coverage area of a network with which the mobile device is associated.

16. The mobile device of claim 11, wherein the processor is further configured to process information by at least one of:

decoding at least one barcode in the identified region and capturing at least one picture associated with the at least one barcode in one decode session, refining the image of the identified region, converting a checked state of at least one check box in the identified region to a numerical value, and identifying alphanumeric characters in the identified region in the image.

17. The mobile device of claim 11, wherein responsive to the presenting, the mobile device is configured to at least one of:

receive feedback on the aggregated application specific information from the user program; and

select at least one predefined rule for using the aggregated information, based on at least one of the operating condition at the mobile device and the contextual information in the document.

18. The mobile device of claim 11, wherein the processor is further configured to associate the aggregated application specific information with at least one predefined confidence level and select at least one predefined rule for using the aggregated information, based on the at least one predefined confidence level.

19. The mobile device of claim 11, wherein the processor is further configured to select at least one predefined rule for using the aggregated information and execute the at least one predefined rule subsequent to generating the aggregated application specific information.

20. A mobile device comprising:

an image capturing device configured to capture an image of a document;

a memory configured to store a plurality of templates at least one of which is associated with the document;

a processor configured to;

select a template stored in the memory, the selected template describes data content of at least one document, prior to the document being captured by the image capturing device;

determine, subsequent to the image of the document being captured by the image capturing device, that the image corresponds to the at least one document described by the selected template by verifying that at least one region in the image corresponds to the at least one document described by the selected template;

identify at least one region in the image that corresponds with a region in the at least one document described by the selected template;

process information in the identified region in the image based on (a) a location of the mobile device, wherein the processor is further configured to validate accuracy of a postal address in the identified region in the image based on the location of the mobile device and route a package to the postal address in the identified region in the image when the location of the mobile device does not correspond to the postal address, and (b) at least one of the operating condition at the mobile device and the contextual information in the at least one document;

aggregate application specific information from the processed information; and

present, based on at least one of the operating condition at the mobile device and the contextual information in the at least one document, aggregated application specific information to a user program operating on one of the mobile device and a remote device connected with the mobile device.