Image information output method

• US 8,593,486 B2
• Filed: 11/18/2011
• Issued: 11/26/2013
• Est. Priority Date: 05/18/2009
• Status: Expired due to Fees

• Current Assignee: Kodaira Associates, Inc.
• Original Assignee: Kodaira Associates, Inc.
• Inventors: Kodaira, Takatoshi, Kodaira, Takamoto
• Primary Examiner(s): Wu, Xiao M.
• Assistant Examiner(s): RABURN, GREG A
• Application Number:
  US13/299,724
• Publication Number:
  US 20120120069A1
• Time in Patent Office: 739 Days
• Field of Search: None
• US Class Current: 345/629
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2210/44 Morphing
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Provided is a video image data generation system including a database for storing a plurality of image data photographed in various directions in various locations, correlating the directions and the locations with the stored image data, and correlating and storing a photographed sub-region when the image data is acquired, a route view point specifying device which specifies various locations and eye level directions arranged on a view point route, an image search engine which searches an image of an eye level direction specified from a location of a view point route specified by the route view point specifying device and outputs video data, wherein the image search engine searches image data stored in a database and the image data including a sub-region located in an eye level direction in each of a plurality of locations on a view point route by referencing photography direction data correlated with the sub-region.

Claims

1. A method for outputting information comprising:

photographing in a plurality of directions at each of a plurality of photographic locations a photographic region with a photographing device, obtaining a plurality of image data of the photographic region and storing in a memory, the plurality of image data of the photographic region at each of the plurality of photographic locations, the photographic region being divided into a plurality of sub-regions;

correlating and storing in the memory photographic location data and photographic direction data at the time when the plurality of image data is obtained with each of the plurality of image data when storing each of the plurality of image data in the memory;

specifying a photographed sub-region when the image data is obtained, by referencing the photographic location data and the photographic direction data correlated with image data stored in the memory;

correlating and storing in a database the image data, the photographic location data and photographic direction data with the specified sub-region;

specifying a plurality of locations arranged in order on a perspective route set above the photographic region and a direction of sight line towards the photographic region at each of the plurality of locations;

referencing the photographic location data correlated with the sub-region stored in the database and reading out in order image data with searching the image data including the sub-region, the sub-region being located in the direction of sight line at each of the plurality of locations on the perspective route; and

outputting the image data read out in order;

wherein the photographic device includes a plurality of cameras pointing in a plurality of directions, the plurality of cameras simultaneously photograph at each of the plurality of photographic locations and the plurality of image data is obtained; and

for each sub-region of the plurality of sub-regions, if a sub-region is photographed a plurality of times when seeing a plurality of photographic locations from an arbitrary geographical point in the sub-region, then an adjacent photographic location is included in a three dimensional angular range of 0.004 steradians or more and 0.54 steradians or less.

2. The method for outputting information according to claim 1, wherein video image data, which is generated, is video data connected by morphing the image data read out, when outputting.

3. The method for outputting information according to claim 1, wherein the perspective route and the direction of sight line are each given as P(t) and V(t) respectively as a function of time t, a trajectory T(t) of a point at which a direct line in a direction of V(t) from P(t) and a photographic sub-region intersect is calculated, and determining a sub-region which includes a location of T(t) at each time t as the sub-region located in the sight line direction in each of the plurality of locations on the perspective route.

4. The method for outputting information according to claim 1, wherein the photographic device is installed in a flying object, and the plurality of cameras photograph in a directions facing the ground including a vertical direction from the flying object.

5. The method for outputting information according to claim 1, wherein the flying object is an aircraft equipped with a user interface which displays a flight route which passes through the plurality of photograph locations.

6. The method for outputting information according to claim 1, wherein photographing is performed by the photographic device while the flying object is flying at an altitude of 200 m or more and 2500 m or less from the ground.

7. The method for outputting information according to claim 1, wherein photographing is performed by the photographic device while flying the flying object below the clouds in cloudy weather.

8. The method for outputting information according to claim 1, wherein when storing the image data in the database, dividing the image data into a plurality of sub image data in a horizontal direction within the image data, creating a histogram of the brightness of pixels for each of the plurality of sub image data, and image processing the image data so that characteristics of the plurality of created histograms become the same.

9. The method for outputting information according to claim 1, wherein the photographic device is installed on a vehicle which moves on the ground, and the plurality of cameras are pointing in the plurality of directions including a back, a forth, and side to side directions of the vehicle.

10. The method for outputting information according to claim 3, wherein image data is selected in which a photographic direction which shows photographic direction data correlated with image data is either the same or has the smallest difference with V(t) where the plurality of image data which includes a sub-region including a location of T(t) at time t exists.

11. The method for outputting information according to claim 3, wherein a sub-region is determined in which the distance between photographic location represented by location data which shows photographic direction data correlated with image data and T(t) is the smallest where a plurality of image data which includes a sub-region including a location of T (t) at time t exists.

12. The method for outputting information according to claim 3, wherein a sub-region is determined in which an obstacle doesn'"'"'t exist between P(t) and T(t) where a plurality of image data which includes a sub-region including a location of T(t) at time t exists.

13. The method for outputting information according to claim 9, wherein the plurality of cameras are pointing in the plurality of directions further including an upper oblique direction.

14. The method for outputting information according to claim 9, wherein the vehicle is equipped with an inclination sensor which detects an inclination direction of the vehicle, and the image data is corrected using a detection result of the inclination sensor.

15. The method for outputting information according to claim 9, wherein the vehicle is equipped with a user interface which displays a movement route which passes through the plurality of photographic locations, and it is possible to control a movement direction and a movement speed of the vehicle by an operation of the user interface.

16. The method for outputting information according to claim 9, wherein it is possible to control a direction of the plurality of cameras by an operation of a user interface.

17. An image data outputting system comprising:

a database which stores a plurality of image data obtained by photographing in a plurality of directions at each of a plurality of photographic locations a photographic region with a photographing device, the photographic region being divided into a plurality of sub-regions, the database correlating and storing photographic location data and photographic direction data at the time when the plurality of image data is obtained with each of the plurality of the image data;

a perspective route specifying device which specifies a plurality of locations arranged in order on a perspective route above the photographic region, and specifies a sight line direction pointing at the photographic region at each of the plurality of locations; and

an image search engine which searches and outputs image data in order, the image search engine searching image data in a direction specified by locations on the perspective route specified by the perspective route specifying device by referencing the photographic location data correlated with a sub-region stored in the database and reading out in order image data, which is stored in the database;

wherein the photographic device includes a plurality of cameras pointing in a plurality of directions, the plurality of cameras simultaneously photograph at each of the plurality of photographic locations and a plurality of image data is obtained; and

for each sub-region of the plurality of sub-regions, if a sub-region is photographed a plurality of times when seeing plurality of photographic locations from an arbitrary geographical point in the sub-region, then an adjacent photographic location is included in a three dimensional angular range of 0.004 steradians or more and 0.54 steradians or less.

18. The image data outputting system according to claim 17, wherein the photographic device photographs images of a square region of the photographic region, and the database correlates location data of the photographic region corresponding to four corners of image which the image data represents, distance data up to where an optical axis of the photographic device intersects with the photographic region, and angular data which shows that a frame of the image is rotating around the optical axis with respect to a horizontal surface and stores the location data, the distance data, and the angular data.

19. The image data outputting system according to claim 17, wherein the database stores an index which correlates the image data with an angle of direction, angle of elevation and a distance when viewing a photographic location at a representative point within a sub-region in which stored image data is photographed by the photographic device, and the image search engine performs a search using the index.