System and method for capturing and processing business data

• US 20060264236A1
• Filed: 05/18/2005
• Published: 11/23/2006
• Est. Priority Date: 05/18/2005
• Status: Active Grant

• Current Assignee: Flint Mobile, Inc.
• Original Assignee: Mobilescan Incorporated
• Inventors: Molnar, Joseph R.S., Mutz, Andrew H., Rashkovskiy, Oleg, Dury, Chris P., Ruano, Rudy, Laszlo, Andrew
• Granted Patent: US 7,640,037 B2
• Time in Patent Office: Days
• US Class Current: 455/557
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00281 with a telecommunication apparatus, e.g. a switched network of teleprinters for the distribution of text-based information, a selective call terminal details of transmission H04N1/00095; establishing a communication with one of a facsimile machine or another apparatus sharing a single line H04N1/32704; interfacing cordless telephone terminals with an accessory to increase the functionality of user interface H04M1/72409
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00312 with a digital transmission apparatus, e.g. a switched network of teleprinters for the distribution of text-based information, SMS or ISDN device transmitting or receiving via a computer, e.g. using e-mail, a computer network or the internet H04N1/00214; transmission of digital information H04L
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/0093 Facsimile machine H04N2201/0094 takes precedence
• CPC Class Codes: H04N 2201/3243 of type information, e.g. handwritten or text document
• CPC Class Codes: H04N 2201/3278 Transmission
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 4/12 Messaging; Mailboxes; Announcements
• CPC Class Codes: H04W 88/02 Terminal devices

Abstract

A method of processing information received in the form of a message from a mobile handheld device with a camera, where the message includes data representing an image of a document and an identifier of a type of said document. The method preferably provides image enhancement, information recognition and analysis selected based on the identified document type. An output of the system is preferably provided to the user in the form specified by the user, such as an e-mail.

Claims

1. A method of capturing information using a mobile handheld device with a camera comprising the steps of:

providing a user with a capability to select a type of information to be captured;

capturing information identified by the user, wherein the step of capturing information is adjusted for the selected type of the information;

pre-processing information so as to obtain pre-processed information, wherein the step of pre-processing the information is adjusted for the selected type of information; and

providing the pre-processed information to a server for post-processing in a message that includes data identifying the type of information.

2. The method of claim 1, wherein the type of information is selected from the group consisting of a business card, a receipt, a form, a printed document, handwritten information, a whiteboard, and graphic indicia.

3. The method of claim 1, wherein the step of pre-processing the information further includes the step of:

removing background information from the captured information.

4. The method of claim 1, wherein the handheld device includes a flash and the step of capturing the information includes the step of adjusting the flash based on the type of information to be captured.

5. The method of claim 1, further comprising the step of:

receiving confirmation from a user that the capture of the information was successful.

6. The method of claim 1, further comprising the step of:

capturing successive images to so as to capture a whiteboard.

7. The method of claim 1, wherein the handheld device is a cellular telephone.

8. The method of claim 1, further comprising the step of:

compressing the pre-processed information and wherein the pre-processed information provided to the server for post-processing is previously compressed.

9. The method of claim 8, wherein the step of compressing is adjusted for the selected type of information.

10. The method of claim 1, wherein the message in the step of providing the pre-processed information to a server further includes information identifying capabilities of the handheld device.

11. The method of claim 1, wherein the handheld device is a cellular phone with a camera and wireless Internet capabilities.

12. The method of claim 1, wherein the step of capturing the information includes the step of taking a photograph with the camera.

13. The method of claim 1, wherein the step of capturing information includes the step of receiving an image from another device over a wireless connection with the other device.

14. A method of processing information comprising the steps of:

receiving from a mobile handheld device that includes a camera a message including data representing an image of a document and an identifier of a type of the document;

storing the data and the associated identifier;

performing image enhancement based on the identifier so as to obtain an enhanced image;

recognizing information in the enhanced image so as to obtain data representing the image in a non-image format, wherein a recognition technique is determined in accordance with the identifier;

for specific documents, analyzing at least some of the data representing the image in a non-image format so as to obtain the meaning of information in said data and storing portions of the data in accordance with the meaning;

creating an output message including the result of one of the step of recognizing and the step of analyzing depending on the value of the identifier having information formatted in accordance with the identifier; and

providing said message to a user.

15. The method of claim 14, wherein said data in the step of receiving the message was obtained by performing the steps comprising selecting the type of data and storing the identifier of the type, adjusting the camera according to the identifier, and pre-processing the data based on the identifier.

16. The method of claim 14, wherein in the data received at the step of receiving the message is previously compressed and further including the step of decompressing compressed data according to the value of the identifier.

17. The method of claim 14, wherein the step of recognizing the information includes the step of performing a character recognition operation on the information.

18. The method of claim 14, wherein the step of recognizing comprises the information includes the step of performing written text recognition on the information.

19. The method of claim 14, wherein the step of analyzing at least some of the data includes the steps of tokenization, initial classification, syntactic parsing, and choosing a best fit.

20. The method of claim 14, wherein the step of analyzing at least some of the data includes using a dictionary and grammar rules selected on the basis of the identifier.

21. The method of claim 14, wherein the step of performing image enhancement includes the step of correcting shadow lightness gradients.

22. The method of claim 14, wherein the step of performing image enhancement includes the step of removing background noise.

23. The method of claim 14, wherein the step of performing image enhancement includes image segmentation.

24. The method of claim 14, wherein the handheld device is a cellular telephone.

25. The method of claim 14, wherein the received message includes data representing capabilities of the handheld device.

26. The method of claim 14, wherein the received message includes data representing the mode of delivery of the output message to the user.

27. The method of claim 14, wherein the step of providing the message to a user includes the step of sending an e-mail to the user.

28. The method of claim 14, wherein the step of providing said message to a user comprises the step of sending a fax to the user.

29. A mobile handheld device for capturing information comprising:

a camera;

a flash;

user interface software including software for providing a user with a capability to select a type of information to be captured by the handheld device;

software for adjusting a capture of the information identified by a user on the basis of the type of the information selected by the user;

means for pre-processing information so as to obtain pre-processed information, wherein the means for pre-processing makes adjustments based on the selected type of information; and

means for providing the pre-processed information to a server for post-processing, the pre-processed information being in the form of a message that includes data identifying the type of information.

30. The device of claim 29, wherein the message includes data identifying capabilities of the mobile handheld device.

31. A network system for capturing and processing information comprising:

a mobile handheld device that includes a camera and a user interface that that is configured to permit a user to select the type of information that is to be captured from a menu;

the camera being adapted to capture the selected information, the handheld device including software that is configured to pre-process the captured information and optionally compress the captured information;

a network including one or more servers including a post-processing server that is in communication with the handheld device such that the pre-processed information is transmitted to the post-processing server which is configured to convert the pre-processed information into post-processed information having a predetermined format; and

means for transmitting the post-processed information in the predetermined format to the user.

32. The system of claim 31, wherein the mobile handheld device comprises a cellular phone and the information comprises business related information with the menu listing the different types of business related information that can be captured as a digital image using the camera.

33. The system of claim 32, wherein the business related information that is listed as entries on the menu includes a business card, a receipt, a form, a printed document, handwritten information, a whiteboard, and graphic indicia.

34. The system of claim 32, further including a submenu for each entry in the menu providing the user with options for at least one of sending the business related information and storing the business related information.

35. The system of claim 32, wherein the predetermined format is in the form of an email that is delivered to a mail server that is associated with the user.

36. The system of claim 31, wherein the software associated with the handheld device is configured to adjust how the information identified by the user is captured on the basis of the type of the information selected by the user, wherein the information is processed so as to obtain pre-processed information utilizing software that makes adjustments in the pre-processing of the information based on the selected type of information.