Device and system for labeling sight images

• US 6,222,583 B1
• Filed: 03/12/1998
• Issued: 04/24/2001
• Est. Priority Date: 03/27/1997
• Status: Expired due to Fees

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Matsumura, Takahiro, Katagiri, Masaji, Ikeda, Takeshi, Suzuki, Akira, Sugimura, Toshiaki, Nakano, Hirotaka
• Primary Examiner(s): Kelley, Chris
• Assistant Examiner(s): WONG, ALLEN C
• Application Number:
  US09/042,685
• Time in Patent Office: 1,139 Days
• Field of Search: 348/113, 348/116, 382/255, 382/154, 345/433
• US Class Current: 348/113
• CPC Class Codes: G01C 21/3635 Guidance using 3D or perspective road maps
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3673 Labelling using text of road map data items, e.g. road names, POI names

Abstract

The device and system for sight labeling according to the present invention comprises an image acquiring section for taking images, a position information acquiring section for recording a camera position while the images are being taken, a camera descriptive information acquiring section for acquiring the camera angle, focal distance and image size while the image are being taken, a map information management section for managing map information, determining a view space inside map information space based on the acquired position, camera angle, focal distance and image size, and capturing constructions present inside the view space, a label information preparation section for preparing label information including the names or descriptive information of constructions and transfer positions, a labeling information output section for overlaying the names or descriptive information from the map information at positions in the images corresponding to position information in the prepared label information, and a control section for controlling each of the sections described above.

Claims

1. A system for sight labeling comprising a sight labeling terminal and a sight labeling center;

said sight labeling terminal comprising

image acquiring means for taking images,

position information acquiring means for acquiring a camera position while the images are being taken,

camera descriptive information acquiring means for acquiring a camera angle, a focal distance and an image size while the images are being taken,

image processing means for dividing the taken images into a plurality of sub-domains,

communication control means for sending information relating to division of said images into sub-domains, the camera angle, the focal distance and the image size through the communication network to said sight labeling center, and receiving label information from said sight labeling center,

labeling information output means for overlaying names or descriptive information of constructions in said label information at predetermined positions in said image corresponding to said transfer positions inside said label information, positions in the images, and outputting the overlayed images to a visual device, and

terminal control means for controlling the means described above; and

said sight labeling center comprising

communication control means for receiving information relating to division of said images into sub-domains, the camera angle, the focal distance and the image size through the communication network from said sight labeling terminal, and sending said label information to said sight labeling terminal,

map information management means for managing map information containing constructions, determining a view of a camera based on the received position, camera angle, focal distance and image size, and calculating a view space formed by said view within a space of said map information and thereby capturing constructions present inside the view space,

label information preparation means for correlating the captured constructions with respect to said sub-domains of said images, specifying transfer positions of said captured constructions in said image, and preparing said label information including the transfer positions and the names or descriptive information of the correlated constructions, and

center control means for controlling the means described above.

2. A system for sight labeling in accordance with claim 1, wherein said label information preparation means prepares CG images which are computer graphics images based on the captured constructions, correlates said sub-domains in said images with sub-domains in said CG images by pattern matching, determines the constructions of the correlated sub-domains, and prepares label information including the names or descriptive information of the constructions and their transfer positions.

3. A system for sight labeling in accordance with claim 1, wherein said label information preparation means three-dimensional projection transforms the captured constructions onto the camera screen, prepares CG images by eliminating constructions which are not capable of being seen from the viewpoint, divides the CG images into sub-domains along profile lines of the sub-domains in the CG images, correlates said sub-domains in said images with sub-domains in said CG images by pattern matching, determines the constructions which are the basis of the sub-domains in the CG image correlated with the sub-domains of the images, and prepares label information including the names or descriptive information of the constructions and their transfer positions.

4. A system for sight labeling in accordance with claim 1, wherein

said sight labeling terminal further comprises distance information acquiring means for determining for each point in said images a depth value which is the distance to the points in said images from the camera position while the images are being taken by said image acquiring means;

in said sight labeling terminal, said label information output means sends information relating to division of said images into sub-domains, the camera angle, the focal distance, the image size and depth values through the communication network to said sight labeling center, and receives label information from said sight labeling center; and

in said sight labeling center, said communication control means receives information relating to division of said images into sub-domains, the camera angle, the focal distance, the image size and depth values through the communication network from said sight labeling terminal, and sends said label information to said sight labeling terminal, and

said label information preparation means, after preparing CG images which are computer graphics images based on constructions captured by said map information management means, determines depth values which are the distances to each point in the sub-domains of the CG images from said camera position while said images are being taken, correlates the sub-domains of said images with the sub-domains of said CG images by comparing groups of depth values for the points in each sub-domain of said CG images and groups of depth values for the points in each sub-domain of said images, determines the constructions of the correlated sub-domains, and prepares label information including the names or descriptive information of said constructions and their transfer positions.

5. A system for sight labeling in accordance with claim 4, wherein in said sight labeling center, said label information preparation means, after preparing CG images which are computer graphics images based on constructions captured by said map information management means, determines depth values which are the distances to each point in the sub-domains of the CG images from said camera position while said images are being taken, correlates the sub-domains of said images with the sub-domains of said CG images by pattern-matching using the ratios between groups of depth values for the points in each sub-domain of said CG images and groups of depth values for the points in each sub-domain of said images and overlap ratios between the sub-domains of said CG images and the sub-domains of said images, determines the constructions of the correlated sub-domains, and prepares label information including the names or descriptive information of said constructions and their transfer positions.

6. A device for sight labeling in accordance with either claim 4, wherein said label information preparation means three-dimensional projection transforms the captured constructions onto the camera screen, prepares CG images by eliminating constructions which are not capable of being viewed from the viewpoint, and divides the CG images into sub-domains by the profile lines of sub-domains in the CG images.

7. A device for sight labeling in accordance with either claim 4, comprising a plurality of each of said image acquiring means, said position information acquiring means and said camera descriptive information means.

8. A device for sight labeling comprising:

image acquiring means for taking images;

position information acquiring means for acquiring a camera position while the images are being taken;

camera descriptive information acquiring means for acquiring a camera angle, a focal distance and an image size while the images arc being taken;

image processing means for dividing images acquired by said image acquiring means into a plurality of sub-domains;

map information management means for managing map information containing constructions, determining a view of camera based on the received position, camera angle, focal distance and image size, and calculating a view space formed by said view within a space of said map information, and capturing constructions present inside the view space;

label information preparation means for correlating the captured constructions with respect to sub-domains of said images, specifying transfer positions of said captured constructions in said image, and preparing label information including the names of descriptive information of the correlated constructions and their transfer positions;

communication address memory means for storing communication addresses of communication devices relating to said constructions;

communication processing means for setting up communication paths with said communication addresses based on the received communication addresses;

labeling information output means for overlaying the names or descriptive information of constructions in said label information at predetermined positions within the images corresponding to the transfer positions inside the prepared label information, and outputting the overlayed images to a visual device;

screen position indicating means for allowing a user to indicate a position on the screen of the visual device, determining the construction in said label information corresponding to the indicated screen position when a position on the screen of said visual device is indicated by said user, determining the communication address of the communication device relating to the determined construction from said communication address memory means; and

control means for controlling the means described above.

9. A sight labeling device in accordance with claim 8, wherein said communication processing means is constructed such that when said user indicates a position on the screen of the visual device, the received communication address information is temporarily displayed on said visual device, and the communication path with said communication address is set up in response to a communication initiation request by said user.

10. A device for sight labeling in accordance with claim 8, further comprising:

image processing means for dividing images acquired by said image acquiring means into a plurality of sub-domains;

wherein

said label information preparation means correlates the captured constructions with respect to sub-domains of said images, and prepares label information including the names or descriptive information of the correlated constructions and their transfer positions;

said label information output means overlays the names or descriptive information in said label information at positions corresponding to their transfer positions in the images, displays the overlayed images on the visual device, and displays a communication address on the visual device upon receiving a communication address; and

said control means controls each means including said image acquiring means.

11. A device for sight labeling in accordance with any one of claim 8, further comprising:

target condition designating means for enabling a user to designate target conditions;

wherein

said label information preparation means appends target condition identifying information indicating that the target conditions designated by the user have been fulfilled to said label information; and

said label information output means displays that a construction matches the target conditions when target condition identifying information is appended to said label information.

12. A device for sight labeling in accordance with any one of claim 8, wherein said label information preparation means prepares CG images which are computer graphics images based on the captured constructions, correlates said sub-domains in said images with sub-domains in said CG images by pattern matching, determines the constructions of the correlated sub-domains, and prepares label information including the names or descriptive information of the constructions and their transfer positions.

13. A device for sight labeling in accordance with any one of claim 8, wherein said label information preparation means three-dimensional projection transforms the captured constructions onto the camera screen, prepares CG images by eliminating constructions which are not capable of being seen from the viewpoint, divides the CG images into sub-domains along profile lines of the sub-domains in the CG images, correlates said sub-domains in said images with sub-domains in said CG images by pattern matching, determines the constructions which are the basis of the sub-domains in the CG image correlated with the sub-domains of the images, and prepares label information including the names or descriptive information of the constructions and their transfer positions.

14. A system for sight labeling comprising a sight labeling terminal and a sight labeling center;

said sight labeling terminal comprising

image acquiring means for taking images,

position information acquiring means for acquiring a camera position while the images are being taken,

camera descriptive information acquiring means for acquiring a camera angle, a focal distance and an image size while the images are being taken,

image processing means for dividing the taken images into a plurality of sub-domains,

communication control means for sending information relating to division of said images into sub-domains, the camera angle, the focal distance and the image size through the communication network to said sight labeling center, and

receiving label information and communication addresses from said sight labeling center,

communication processing means for setting up paths to said communication addresses based on the received communication addresses,

labeling information output means for overlaying names or descriptive information of constructions in said label information at a predetermined position of said image corresponding to said transfer positions inside said label information and displaying the overlayed images on a visual device,

screen position indicating means for allowing a user to indicate a position on the screen of the visual device, determining the construction in said label information corresponding to the indicated screen position when a position on the screen of said visual device is indicated by said user, outputting said construction to said communication control means, and sending the communication address of the communication device relating to said construction received by said communication control means to said communication processing means;

said terminal control means for controlling the means described above; and

said sight labeling center comprising

communication control means for receiving information relating to division of said images into sub-domains, the camera angle, the focal distance and the image size through the communication network from said sight labeling terminal, and sending said label information and communication address to said sight labeling terminal,

map information management means for managing map information containing constructions, determining a view of a camera based on the received position, camera angle, focal distance and image size, calculating a view space formed by said view in a space of said map information, and thereby capturing constructions present inside the view space,

label information preparation means for correlating the captured constructions with respect to said sub-domains of said images, specifying transfer positions of said acquired constructions in said image, and preparing label information including the names or descriptive information of the correlated constructions and their transfer positions,

communication address memory means for storing communication addresses of communication devices relating to said constructions, and outputting communication addresses of constructions indicated by information of constructions in said label information, and

center control means for controlling the means described above.

15. A system for sight labeling in accordance with claim 14, wherein said communication processing means is constructed such that when said user indicates a position on the screen of the visual device, the received communication address information is temporarily displayed on said visual device, and the communication path with said communication address is set up in response to a communication initiation request by said user.

16. A system for sight labeling in accordance with any one of claim 14, said sight labeling terminal further comprising:

target condition designating means for enabling a user to designate target conditions;

wherein

said label information preparation means appends target condition identifying information indicating that the target conditions designated by the user have been fulfilled to said label information; and

said label information output means displays that a construction matches the target conditions when target condition identifying information is appended to said label information.

17. A system for sight labeling in accordance with any one of claim 14, wherein said label information preparation means prepares CG images which are computer graphics images based on the captured constructions, correlates said sub-domains in said images with sub-domains in said CG images by pattern matching, determines the constructions of the correlated sub-domains, and prepares label information including the names or descriptive information of the constructions and their transfer positions.

18. A system for sight labeling in accordance with any one of claim 14, wherein said label information preparation means three-dimensional projection transforms the captured constructions onto the camera screen, prepares CG images by eliminating constructions which are not capable of being seen from the viewpoint, divides the CG images into sub-domains along profile lines of the sub-domains in the CG images, correlates said sub-domains in said images with sub-domains in said CG images by pattern matching, determines the constructions which are the basis of the sub-domains in the CG image correlated with the sub-domains of the images, and prepares label information including the names or descriptive information of the constructions and their transfer positions.

19. A system for sight labeling comprising a sight labeling terminal and a sight labeling center;

said sight labeling terminal comprising

image acquiring means for taking images,

position information acquiring means for acquiring a camera position while the images are being taken,

camera descriptive information acquiring means for acquiring a camera angle, a focal distance and an image size while the images are being taken,

image processing means for dividing the taken images into a plurality of sub-domains,

communication control means for sending information relating to division of said images into sub-domains, the camera angle, the focal distance and the image size through the communication network to said sight labeling center, and receiving label information and communication addresses from said sight labeling center,

labeling information output means for overlaying names or descriptive information of constructions in said label information at predetermined position within said image corresponding to said transfer positions inside said label information at corresponding positions in the images, and displaying the overlayed images on a visual device, and for displaying a communication address on said visual device when said communication address is received,

screen position indicating means for allowing a user to indicate a position on the screen of the visual device, determining the construction in said label information corresponding to the indicated screen position when a position on the screen of said visual device is indicated by said user, outputting said construction to said communication control means, and sending the communication address of the communication device relating to said construction received by said communication control means to said label information output means; and

terminal control means for controlling the means described above; and

said sight labeling center comprising

communication control means for receiving information relating to division of said images into sub-domains, the camera angle, the focal distance and the image size through the communication network from said sight labeling terminal, and sending said label information and communication address to said sight labeling terminal,

map information management means for managing map information containing constructions, determining a view of a camera based on the received position, camera angle, focal distance and image size, calculating a view space formed by said view in a space of said map information, and thereby capturing constructions present inside the view space,

label information preparation means for correlating the captured constructions with respect to said sub-domains of said images, specifying a transfer position of said acquired constructions in said image, and preparing label information including the names or descriptive information of the correlated constructions and their transfer positions,

communication memory means for storing communication addresses of communication devices relating to said constructions, and outputting communication addresses of constructions indicated by information of constructions in said label information, and center control means for controlling the means described above.