Providing camera-based services using a portable communication device

• US 20100128131A1
• Filed: 05/29/2009
• Published: 05/27/2010
• Est. Priority Date: 11/21/2008
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Beyo GmbH
• Inventors: Goektekin, Cueneyt, Tenchio, Oliver
• Granted Patent: US 8,218,020 B2
• Time in Patent Office: Days
• US Class Current: 348/207.100
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/141 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields

Abstract

Camera-based services are provided to a user of a portable communication device by recognizing text contained in an image. An image of an environment is captured using a camera within the portable communication device so as to obtain image data. The image data is processed such that text data is recognized and extracted from the image data. Data related to the text data is then output in a form recognizable by a user of the portable communication device. The text data can be processed on the portable communication device to obtain the data related to the text data. Alternatively, the processing is performed by a processing unit external to the portable communication device. Translated and audio versions of the text data are output to the user. One camera-based service provides price and product information related to a product described in an image captured by the camera.

Claims

1. A method comprising:

capturing an image of an environment using a camera within a portable communication device so as to obtain image data;

processing the image data such that text data is recognized and extracted from the image data; and

outputting data related to the text data in a form recognizable by a user of the portable communication device.

2. The method of claim 1, further comprising:

processing the text data to obtain the data related to the text data.

3. The method of claim 1, wherein the processing of the image data is performed on the portable communication device.

4. The method of claim 1, wherein the processing of the image data is performed by a processing unit external to the portable communication device.

5. The method of claim 4, wherein the text data is transmitted from the portable communication device to the external processing unit, processed by the external processing unit to generate the data related to the text data, and wherein the data related to the text data is transmitted from the external processing unit to the portable communication device.

6. The method of claim 4, wherein the external processing unit includes a first server that communicates with the portable communication device via a first communication network.

7. The method of claim 6, wherein the external processing unit includes a second server that communicates with the first server via a second communication network.

8. The method of claim 7, wherein the processing the image data is performed by the first server.

9. The method of claim 7, wherein the second server performs the processing the image data.

10. The method of claim 9, wherein the first server is adapted to receive the image data from the portable communication device via the first communication network and to transmit the received image data to the second server for processing.

11. The method of claim 7, wherein the first server is adapted to store billing information related to a service provided using the second server.

12. The method of claim 7, wherein the second server stores the image data, and wherein the portable communication device accesses the image data in real-time from the second server.

13. Method of one of claims 7 to 13, wherein the second server stores the text data, wherein the first server communicates with a personal computer, and wherein the personal computer accesses the text data on the second server via the first server.

14. The method of claim 1, wherein the text data is archived by indexing the text data according to keywords.

15. The method of claim 1, wherein the image data has a quality, and wherein the quality of the image data that will be obtained by capturing the image is indicated to the user of the portable communication device in a manner taken from the group consisting of:

by the manner in which the portable communication device vibrates, by an audio feedback to the user of the portable communication device, and by displaying a reticule on a display of the portable communication device.

16. The method of claim 1, wherein the camera captures a first image of a first portion of the environment and a second image of a second portion of the environment, and wherein the portable communication device stitches the first image and the second image together.

17. The method of claim 1, wherein the image is rotated and unwarped before the image data is obtained.

18. The method of claim 1, further comprising:

performing block recognition, wherein the processing the image data is performed in parallel on image data within recognized blocks.

19. The method of claim 1, wherein the image data are stored in association with position data indicating a position where the image was captured.

20. A portable communication device, comprising:

a camera adapted to capture an image of an environment, wherein image data obtained from the image is processed to generate text data;

a display adapted to display an indication of a service selectable by a user of the portable communication device; and

outputting means that outputs data related to the text data in a form recognizable by the user of the portable communication device, wherein the portable communication device performs the service selected by the user.

21. The portable communication device of claim 20, further comprising:

a first worker module that displays the image on the display and that detects an orientation of text included in the image;

a second worker module that performs text recognition on the image data to generate the text data; and

a third worker module that converts the text data into speech and generates an audio output, wherein the first, second and third worker modules work independently of one other and communicate with an operation system of the portable communication device via dedicated interfaces.

22. The portable communication device of claim 21, wherein the first, second and third worker modules are part of a client application, wherein the portable communication device communicates with a server, and wherein the first, second and third worker modules are configured dynamically through the server without reinstalling the client application on the portable communication device.

23. The portable communication device of claim 22, wherein each of the first, second and third worker modules also runs on the server so as to achieve a load balancing.

24. The portable communication device of claim 20, further comprising:

a service module that communicates with a server and requests performance of the service selected by the user.

25. A server, comprising:

a first interface that receives text data from a portable communication device via a first communication network; and

a second interface that transmits the received text data to another server in a data format readable by the other server, wherein the second interface receives data from the other server that is generated by processing the text data, wherein the first interface transmits the processed data received from the other server to the portable communication device in a data format readable by the portable communication device.