Smart optical input/output (I/O) extension for context-dependent workflows

• US 9,349,046 B2
• Filed: 04/14/2015
• Issued: 05/24/2016
• Est. Priority Date: 02/10/2009
• Status: Active Grant

• Current Assignee: Tungsten Automation Corp.
• Original Assignee: Kofax Incorporated
• Inventors: Macciola, Anthony, Amtrup, Jan W.
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US14/686,644
• Publication Number:
  US 20150220778A1
• Time in Patent Office: 406 Days
• Field of Search: 382/112
• US Class Current: 1/1
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: H04N 1/40 Picture signal circuits H04N1/387 takes precedence

Abstract

Systems, methods, and computer program products for smart, automated capture of textual information using optical sensors of a mobile device are disclosed. The textual information is provided to a mobile application or workflow without requiring the user to manually enter or transfer the data without requiring user intervention such as a copy/paste operation. The capture and provision context-aware, and can normalize or validate the captured textual information prior to entry in the workflow or mobile application. Other information necessary by the workflow and available to the mobile device optical sensors may also be captured and provided, in a single automatic process. As a result, the overall process of capturing information from optical input using a mobile device is significantly simplified and improved in terms of accuracy of data transfer/entry, speed and efficiency of workflows, and user experience.

Claims

1. A computer-implemented method, comprising:

invoking a user input interface on a mobile device;

invoking an optical input extension of the user input interface;

capturing optical input via one or more optical sensors of the mobile device;

determining textual information from the captured optical input;

based on analyzing the optical input, wherein the analyzing comprises;

determining a restricted optical character recognition (OCR) alphabet based on the optical input; and

performing OCR using the restricted OCR alphabet;

identifying desired textual information among the determined textual information based on the OCR;

selectively providing the desired textual information to the user input interface without relying on feedback from the user; and

wherein the user input interface is invoked in response to detecting a user interaction with a user interface element configured to receive textual information.

2. The method as recited in claim 1, wherein the desired textual information comprises a plurality of identifiers, each identifier corresponding to one of a plurality of user interface elements configured to receive textual information, and

wherein the providing further comprises automatically and selectively providing each of the plurality of identifiers to the corresponding user interface element without relying on feedback from the user.

3. The method as recited in claim 2, further comprising normalizing at least one of the identifiers to conform with one or more of an expected format of the textual information and an expected range of values for the textual information.

4. The method as recited in claim 3, further comprising determining one or more of reference content from a complementary document and business rules applicable to at least one of the identifiers,

wherein the determining is based on the user interface element corresponding to the identifier(s), and

wherein the normalizing is based on one or more of the reference content and the business rules.

5. The method as recited in claim 1, further comprising normalizing the desired textual information to conform with one or more of an expected format of the textual information and an expected range of values for the textual information, wherein the normalizing does not rely on feedback from the user.

6. The method as recited in claim 5, wherein the normalizing is based on one or more of reference content from a complementary document and business rules.

7. The method as recited in claim 6, further comprising determining one or more of the complementary document and the business rules based on the element with which the user interacted.

8. The method as recited in claim 1, further comprising automatically invoking an optical input capture interface in response to detecting the invocation of the optical input extension;

wherein the user input interface comprises a virtual keyboard displayed on the mobile device, and

wherein the optical input extension comprises a camera button displayed on the virtual keyboard.

9. A method, comprising:

invoking a user input interface on a mobile device;

invoking an optical input extension of the user input interface;

pre-analyzing optical input prior to capturing the optical input, wherein the pre-analyzing comprises;

detecting an object depicted in the optical input;

determining one or more characteristics of the object depicted in the optical input; and

determining one or more analysis parameters based at least in part on the determined characteristic(s), wherein the one or more analysis parameters comprise a restricted optical character recognition (OCR) alphabet; and

capturing the optical input via one or more optical sensors of the mobile device;

analyzing the optical input to determine textual information, wherein the analyzing comprises;

performing OCR using the restricted OCR alphabet

identifying expected textual information among the determined textual information based on the OCR; and

selectively providing the expected textual information to the user input interface without relying on feedback from the user; and

wherein the user input interface is invoked in response to detecting a user interaction with a user interface element configured to receive textual information.

10. The method as recited in claim 9, wherein the one or more analysis parameters comprise OCR parameters;

and

wherein the one or more characteristics comprise the object depicting one or more of textual information having a non-standard color profile and a background having a non-standard color profile.

15. The method as recited in claim 9, wherein the pre-analyzing is performed automatically without relying on input from a user.

11. A computer program product comprising:

a computer readable storage medium having program code embodied therewith, the program code readable/executable by a processor to;

invoke a user input interface on a mobile device;

invoke an optical input extension of the user input interface;

automatically pre-analyze optical input received via one or more optical sensors of the mobile device, the pre-analysis comprising;

determining a restricted optical character recognition (OCR) alphabet based on the optical input; and

performing OCR using the restricted OCR alphabet; and

identifying textual information based on the OCR;

capture optical input via the one or more optical sensors of the mobile device in response to identifying textual information in the pre-analysis;

determine expected textual information from the identified textual information; and

selectively provide the expected textual information to the user input interface without relying on feedback from the user; and

wherein the user input interface is invoked in response to detecting a user interaction with a user interface element configured to receive textual information.

12. The computer program product as recited in claim 11, wherein the expected textual information is selectively provided to an appropriate element of the user input interface based on determining the expected textual information includes required textual information for the appropriate element of the user input interface.

13. The computer program product as recited in claim 11, the program code being further readable/executable by the processor to:

automatically normalize, without relying on feedback from a user, at least some of the expected textual information to conform with one or more of an expected format of the expected textual information and an expected range of values for the expected textual information.

14. The computer program product as recited in claim 13, wherein the program code readable/executable by the processor to automatically normalize, without relying on feedback from the user, the expected textual information is further readable/executable by the processor to:

determine an appropriate formatting for the expected textual information based on one or more of reference content from a complementary document and business rules; and

modify a format of the expected textual information to match the appropriate formatting.