Character recognition device, mobile communication system, mobile terminal device, fixed station device, character recognition method and character recognition program

• US 8,135,218 B2
• Filed: 10/04/2010
• Issued: 03/13/2012
• Est. Priority Date: 01/08/2004
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Takahashi, Katsuhiko, Nishiwaki, Daisuke
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US12/897,411
• Publication Number:
  US 20110081084A1
• Time in Patent Office: 526 Days
• Field of Search: 382/182, 382/190, 382/310, 382/178, 382/294, 382/203, 382/204, 382/113, 382/195, 382/181, 382/177, 382/187, 382/176, 382/298, 382/229, 382/170, 455/557, 455/575.1, 455/414.1, 455/457, 235/462.25, 235/462.01, 235/472.01, 235/454
• US Class Current: 382/182
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Words possibly included in a scene image shot by a mobile camera can be efficiently extracted using a word dictionary or a map database. Positional information acquiring means 101 measures a current position of the device to acquire positional information. Directional information acquiring means 102 detects a direction of the device to acquire directional information. Character recognizing means 104 determines a range of shooting of a scene image based on the current positional information and the directional information. The character recognizing means 104 extracts from a map database 103 information such as store names, building names, and place names associated with positions in the shooting range. Then the character recognizing means 104 conducts character recognition using word knowledge such as the extracted store names, building names, and place names.

Claims

1. A mobile communication system in which a mobile terminal device and a fixed station device communicate via a wireless transmission path with each other, characterized in that:

the mobile terminal device comprises;

imaging means for shooting an image;

position measuring means for measuring a shooting position of an image to obtain shooting position information indicating the shooting position;

direction detecting means for detecting a shooting direction of an image to obtain shooting direction information indicating the shooting direction; and

information transmitting means on mobile terminal side for transmitting the shooting position information, the shooting direction information, and an image shot by the imaging means via a communication network to the fixed station device; and

the fixed station device comprises;

position corresponding information storage means for storing therein position corresponding information that is words associated with respective positional information indicating positions of respective places;

position corresponding information extracting means for determining, based on the shooting position information and the shooting direction information that are received from the information transmitting means on mobile terminal side, a range of a shooting object of the imaging means, and extracting from the position corresponding information storage means position corresponding information associated with positions included in the range;

character recognizing means for recognizing, using the position corresponding information extracted by the position corresponding information extracting means, a character or a character string included in the image that are received from the information transmitting means on mobile terminal side; and

information transmitting means on fixed station side for transmitting information of a character or a character string recognized by the character recognizing means via a communication network to the mobile terminal device.

2. The mobile communication system in accordance with claim 1, wherein the fixed station device comprises:

non-corresponding-to-position information storage means for storing therein non-corresponding-to-position information that is words not associated with positional information; and

non-corresponding-to-position information extracting means for extracting non-corresponding-to-position information from the non-corresponding-to-position information storage means according to the position corresponding information extracted by the position corresponding information extracting means, wherein

the character recognizing means recognizes a character or a character string included in the image using the position corresponding information extracted by the position corresponding information extracting means and the non-corresponding-to-position information extracted by the non-corresponding-to-position information extracting means.

3. The mobile communication system in accordance with claim 1, wherein the mobile terminal device comprises:

information receiving means on mobile terminal side for receiving information of a character or a character string recognized by the character recognizing means via a communication network from the information transmitting means on fixed station side; and

output means for outputting therefrom information of the character or the character string.

4. A mobile communication system in which a mobile terminal device and a fixed station device communicate via a wireless transmission path with each other, characterized in that:

the mobile terminal device comprises;

imaging means for shooting an image;

position measuring means for measuring a shooting position of an image to obtain shooting position information indicating the shooting position;

direction detecting means for detecting a shooting direction of an image to obtain shooting direction information indicating the shooting direction;

information transmitting means on mobile terminal side for transmitting the shooting position information and the shooting direction information via a communication network to the fixed station device; and

character recognizing means for recognizing a character or a character string included in the image shot by the imaging means; and

the fixed station device comprises;

position corresponding information storage means on fixed station side for storing therein position corresponding information that is words associated with respective positional information indicating positions of respective places;

position corresponding information extracting means on fixed station side for determining, based on the shooting position information and the shooting direction information that are received from the information transmitting means on mobile terminal side, a range of a shooting object of the imaging means, and extracting from the position corresponding information storage means on fixed station side position corresponding information associated with positions included in the range;

non-corresponding-to-position information storage means for storing therein non-corresponding-to-position information that is words not associated with positional information;

non-corresponding-to-position information extracting means for extracting non-corresponding-to-position information from the non-corresponding-to-position information storage means according to the position corresponding information extracted by the position corresponding information extracting means on fixed station side; and

information transmitting means on fixed station side for transmitting the non-corresponding-to-position information extracted by the non-corresponding-to-position information extracting means via a communication network to the mobile terminal device, wherein

the character recognizing means recognizes a character or a character string included in the image using the non-corresponding-to-position information received from the information transmitting means on fixed station side.

5. The mobile communication system in accordance with claim 4, wherein:

the mobile terminal device comprises;

position corresponding information storage means on mobile terminal side for storing position corresponding information that is words associated with respective positional information indicating positions of respective places; and

position corresponding information extracting means on mobile terminal side for determining, based on the shooting position information and the shooting direction information, a range of a shooting object of the imaging means, and extracting from the position corresponding information storage means on mobile terminal side position corresponding information associated with positions included in the range, wherein

the character recognizing means recognizes a character or a character string included in the image using the non-corresponding-to-position information received from the information transmitting means on fixed station side and the position corresponding information extracted by the position corresponding information extracting means on mobile terminal side.

6. The mobile communication system in accordance with claim 4, wherein:

the information transmitting means on fixed station side transmits the non-corresponding-to-position information extracted by the non-corresponding-to-position information extracting means and the position corresponding information extracted by the position corresponding information extracting means on fixed station side via a communication network to the mobile terminal device; and

the character recognizing means recognizes a character or a character string included in the image using the non-corresponding-to-position information and the position corresponding information that are received from the information transmitting means on fixed station side.

7. The mobile communication system in accordance with claim 4, wherein:

the non-corresponding-to-position information storage means stores non-corresponding-to-position information that is a set of words not associated with positional information and that includes at least one word equal to position corresponding information; and

the non-corresponding-to-position information extracting means extracts non-corresponding-to-position information from the non-corresponding-to-position information storage means using as a key the position corresponding information extracted by the position corresponding information extracting means on fixed station side.

8. A mobile terminal device for communicating information with a fixed station device that executes character recognition processing to recognize a character or a character string included in an image, comprising:

imaging means for shooting an image;

position measuring means for measuring a shooting position of an image to obtain shooting position information indicating the shooting position;

direction detecting means for detecting a shooting direction of an image to obtain shooting direction information indicating the shooting direction;

information transmitting means on mobile terminal side for transmitting the shooting position information, the shooting direction information, and an image imaged by the imaging means via a communication network to the fixed station device; and

information receiving means on mobile terminal side for receiving from the fixed station device information of a character or a character string obtained by the fixed station device through the character recognition processing for the image using the shooting position information and the shooting direction information.

9. A fixed station device for communicating information with a mobile terminal device, comprising:

information receiving means on fixed station side for receiving from the mobile terminal device an image shot by the mobile terminal device, the shooting position information indicating a shooting position of the image, and the shooting direction information indicating a shooting direction of the image;

position corresponding information storage means for storing position corresponding information that is words associated with respective positional information indicating positions of respective places;

position corresponding information extracting means for determining, based on the shooting position information and the shooting direction information that are received from the mobile terminal device, a range of an object shot by the mobile terminal device, and extracting from the position corresponding information storage means position corresponding information associated with positions included in the range;

character recognizing means for recognizing a character or a character string included in the image received from the mobile terminal device, using the position corresponding information extracted by the position corresponding information extracting means; and

information transmitting means on fixed station side for transmitting information of a character or a character string recognized by the character recognizing means via a communication network to the mobile terminal device.

10. The fixed station device in accordance with claim 9, comprising:

non-corresponding-to-position information storage means for storing non-corresponding-to-position information that is words not associated with positional information; and

non-corresponding-to-position information extracting means for extracting non-corresponding-to-position information from the non-corresponding-to-position information storage means according to the position corresponding information extracted by the position corresponding information extracting means, wherein

the character recognizing means recognizes a character or a character string included in the image using the position corresponding information extracted by the position corresponding information extracting means and the non-corresponding-to-position information extracted by the non-corresponding-to-position information extracting means.

11. A mobile terminal device for communicating information with a fixed station device and executing character recognition processing to recognize a character or a character string included in an image, comprising:

imaging means for shooting an image;

position measuring means for measuring a shooting position of an image to obtain shooting position information indicating the shooting position;

direction detecting means for detecting a shooting direction of an image to obtain shooting direction information indicating the shooting direction;

information transmitting means on mobile terminal side for transmitting the shooting position information and the shooting direction information via a communication network to the fixed station device;

information receiving means on mobile terminal side for receiving from the fixed station device non-corresponding-to-position information that is words not associated with respective positional information indicating positions of respective places and that is determined by the fixed station device using the shooting position information and the shooting direction information; and

character recognizing means for recognizing a character or a character string included in the image imaged by the imaging means, using the non-corresponding-to-position information received by the information receiving means on mobile terminal side.

12. The mobile terminal device in accordance with claim 11, comprising:

position corresponding information storage means on mobile terminal side for storing position corresponding information that is words associated with respective positional information indicating positions of respective places; and

position corresponding information extracting means on mobile terminal side for determining, based on the shooting position information and the shooting direction information, a range of a shooting object of the imaging means, and extracting from the position corresponding information storage means on mobile terminal side position corresponding information associated with positions included in the range, wherein

the character recognizing means recognizes a character or a character string included in the image using the non-corresponding-to-position information received by the information receiving means on mobile terminal side and the position corresponding information extracted by the position corresponding information extracting means on mobile terminal side.

13. A fixed station device for communicating information with a mobile terminal device that executes character recognition processing to recognize a character or a character string included in an image, comprising:

information receiving means on fixed station side for receiving from the mobile terminal device the shooting position information of an image that is shot by the mobile terminal device and that indicates a shooting position of the image and the shooting direction information indicating a shooting direction of the image;

position corresponding information storage means on fixed station side for storing therein position corresponding information that is words associated with respective positional information indicating positions of respective places;

position corresponding information extracting means on fixed station side for determining, based on the shooting position information and the shooting direction information that are received by the information receiving means on fixed station side, a range of an object shot by the mobile terminal device, and extracting from the position corresponding information storage means on fixed station side position corresponding information associated with positions included in the range;

non-corresponding-to-position information storage means for storing non-corresponding-to-position information that is words not associated with positional information;

non-corresponding-to-position information extracting means for extracting non-corresponding-to-position information from the non-corresponding-to-position information storage means according to the position corresponding information extracted by the position corresponding information extracting means on fixed station side; and

information transmitting means on fixed station side for transmitting the non-corresponding-to-position information extracted by the non-corresponding-to-position information extracting means via a communication network to the mobile terminal device.

14. The fixed station device in accordance with claim 13, wherein the information transmitting means on fixed station side transmits the non-corresponding-to-position information extracted by the non-corresponding-to-position information extracting means and the position corresponding information extracted by the position corresponding information extracting means on fixed station side.

15. A character recognition method of recognizing a character or a character string included in an image, for use in a mobile communication system in which a mobile terminal device and a fixed station device communicate via a wireless transmission path with each other, comprising:

the mobile terminal device shooting an image;

the mobile terminal device measuring a shooting position of an image to obtain shooting position information indicating the shooting position;

the mobile terminal device detecting a shooting direction of an image to obtain shooting direction information indicating the shooting direction;

the mobile terminal device transmitting the shooting position information, the shooting direction information, and an image imaged by the imaging means via a communication network to the fixed station device;

the fixed station device determining, based on the shooting position information and the shooting direction information that are received from the mobile terminal device, a range of a shooting object of the mobile terminal device;

the fixed station device extracting, from position corresponding information storage means storing position corresponding information that is words associated with respective positional information indicating positions of respective places, position corresponding information associated with positions included in the range;

the fixed station device recognizing, using the extracted position corresponding information, a character or a character string included in the image received from the mobile terminal device; and

the fixed station device transmits information of a character or a character string via a communication network to the mobile terminal device.

16. The character recognition method in accordance with claim 15, further comprising:

the mobile terminal device receiving information of a character or a character string recognized by the fixed station device, via a communication network from the fixed station device; and

the mobile terminal device outputting information of the character or the character string.

17. A character recognition method of recognizing a character or a character string included in an image, for use in a mobile communication system in which a mobile terminal device and a fixed station device communicate via a wireless transmission path with each other, comprising:

the mobile terminal device shooting an image;

the mobile terminal device measuring a shooting position of an image to obtain shooting position information indicating the shooting position;

the mobile terminal device detecting a shooting direction of an image to obtain shooting direction information indicating the shooting direction;

the mobile terminal device transmitting the shooting position information and the shooting direction information via a communication network to the fixed station device;

the fixed station device determining, based on the shooting position information and the shooting direction information that are received from the mobile terminal device, a range of a shooting object of the mobile terminal device;

the fixed station device extracting, from position corresponding information storage means on fixed station side storing position corresponding information that is words associated with respective positional information indicating positions of respective places, position corresponding information associated with positions included in the range;

the fixed station device extracting, from non-corresponding-to-position information storage means storing non-corresponding-to-position information that is words not associated with positional information, non-corresponding-to-position information according to the position corresponding information extracted from the position corresponding information storage means on fixed station side;

the fixed station device transmitting the extracted non-corresponding-to-position information via a wireless transmission network to the mobile terminal device; and

the mobile station device recognizing a character or a character string included in the image shot by the mobile terminal device, using the non-corresponding-to-position information received from the fixed station device.

18. A non-transitory computer readable medium storing a character recognition program, which causes a computer to execute:

determining, based on shooting position information indicating a shooting position of an image and shooting direction information indicating a shooting direction of an image, a range of an object of imaging;

extracting, from position corresponding information storage means storing position corresponding information that is words associated with respective positional information indicating positions of respective places, position corresponding information associated with positions included in the range; and

recognizing, using the extracted position corresponding information, a character or a character string included in the image thus shot.

19. The non-transitory computer readable medium of claim 18, wherein the character recognition program cases the computer to further execute:

extracting, from non-corresponding-to-position information storage means storing non-corresponding-to-position information that is words not associated with positional information, non-corresponding-to-position information according to the position corresponding information extracted from the position corresponding information storage means; and

recognizing a character or a character string included in the image using the position corresponding information extracted from the position corresponding information storage means and the non-corresponding-to-position information extracted from the non-corresponding-to-position information storage means.

20. An information communication program in a mobile terminal device communicating information with a fixed station device that recognizes a character or a character string included in an image, causing a computer to execute:

transmitting shooting position information indicating a shooting position of an image, shooting direction information indicating a shooting direction of an image, and an image via a wireless transmission network to the fixed station device; and

receiving from the fixed station device information of a character or a character string obtained by the fixed station device through the character recognition processing for the image using the shooting position information and the shooting direction information.

21. A character recognition program in a fixed station device for communicating information with a mobile terminal device, causing a computer to execute:

receiving from the mobile terminal device the image shot by the mobile terminal device, the shooting position information indicating a shooting position of the image, and the shooting direction information indicating a shooting direction of the image;

determining, based on the shooting position information and the shooting direction information that are received from the mobile terminal device, a range of an object shot by the mobile terminal device;

extracting, from position corresponding information storage means storing therein position corresponding information that is words associated with respective positional information indicating positions of respective places, position corresponding information associated with positions included in the range;

recognizing a character or a character string included in the image received from the mobile terminal device, using the extracted position corresponding information; and

transmitting information of a character or a character string via a communication network to the mobile terminal device.

22. A character recognition program in a mobile terminal device for communicating information with a fixed station device and recognizing a character or a character string included in an image, causing a computer to execute:

transmitting shooting position information indicating a shooting position of an image and shooting direction information indicating a shooting direction of an image via a communication network to the fixed station device;

receiving from the fixed station device non-corresponding-to-position information that is words not associated with respective positional information indicating positions of respective places and which is determined by the fixed station device using the shooting position information and the shooting direction information; and

recognizing a character or a character string included in the image shot by the mobile terminal device, using the non-corresponding-to-position information received from the fixed station device.

23. An information extraction program in a fixed station device for communicating information with a mobile terminal device which recognizes a character or a character string included in an image, causing a computer to execute:

receiving from the mobile terminal device shooting position information indicating a shooting position of an image shot by the mobile terminal device and shooting direction information indicating a shooting direction of the image;

determining, based on the shooting position information and the shooting direction information that are received from the mobile terminal device, a range of an object shot by the mobile terminal device;

extracting, from position corresponding information storage means on fixed station side storing position corresponding information that is words associated with respective positional information indicating positions of respective places, position corresponding information associated with positions included in the range;

extracting, from non-corresponding-to-position information storage means storing non-corresponding-to-position information that is words not associated with positional information, non-corresponding-to-position information according to the position corresponding information extracted from the position corresponding information storage means on fixed station side; and

transmitting the extracted non-corresponding-to-position information via a communication network to the mobile terminal device.