Spatial referenced photography

• US 6,195,122 B1
• Filed: 07/30/1997
• Issued: 02/27/2001
• Est. Priority Date: 01/31/1995
• Status: Expired due to Fees

• Current Assignee: Transcenic Incorporated
• Original Assignee: Robert S. Vincent
• Inventors: Vincent, Robert
• Primary Examiner(s): Garber, Wendy
• Assistant Examiner(s): Vu, Ngoc-Yen, VU, NGOC YEN T
• Application Number:
  US08/894,206
• Time in Patent Office: 1,308 Days
• Field of Search: 348/15, 348/135, 348/141, 348/142, 348/143, 348/169, 348/207, 348/211, 348/213, 348/222, 348/231, 348/232, 348/233, 348/333.01, 348/333.02, 348/333.12, 348/373, 348/375, 348/552, 358/906, 358/909.1, 345/326, 345/333, 345/334, 345/339, 396/50, 396/310, 396/321, 396/373, 396/374, 396/378, 396/383
• US Class Current: 348/169
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/9201 involving the multiplexing of an additional signal and the video signal
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

An image system captures, along with the images, information defining both the position and the orientation of a camera along with the distance to the subject. A video camera is attached to three accelerometers, two gyroscopes, and a rangefinder. Data gathered from these devices and defining the pitch, yaw, and roll of the camera, the camera'"'"'s acceleration, and the camera'"'"'s distance to the subject is captured and recorded along with the video images. The video images are later stored within a computer'"'"'s database along with data defining the position and the orientation of the camera, and the distance to the subject for each image. The images may then be presented to a user in a three-dimensional display in which the user can navigate through the images using a joystick or a mouse.

Claims

1. A spatially referenced photographic system comprising:

a data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

image presentation and navigation means for displaying the images to a user and for facilitating the user in navigating among said images by receiving spatial movement commands from the user, as indicated by said spatial movement commands;

a camera capable of capturing data representative of plural images of objects;

position sensing means connected to said camera for capturing data defining the position of said camera at times when images are captured;

orientation sensing means connected to said camera for capturing data defining the orientation of said camera at times when images are captured;

data capturing means for receiving image representation data from said camera, position defining data from said position sensing means, and orientation defining data from said orientation sensing means and for storing all of said data to form said data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position; and

wherein said orientation sensing means comprises gimbals associated with gyroscopes.

2. A spatially referenced photographic system in accordance with claim 1 wherein said gyroscopes are rigidly attached to said camera by means of gimbals, and where said orientation defining data is derived from the rotational position of said gimbals.

3. A spatially referenced photographic system comprising:

a data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

image presentation and navigation means for displaying the images to a user and for facilitating the user in navigating among said images by receiving spatial movement commands from the user, as indicated by said spatial movement commands;

a camera capable of capturing data representative of plural images of objects;

position sensing means connected to said camera for capturing data defining the position of said camera at times when images are captured;

orientation sensing means connected to said camera for capturing data defining the orientation of said camera at times when images are captured;

data capturing means for receiving image representation data from said camera, position defining data from said position sensing means, and orientation defining data from said orientation sensing means and for storing all of said data to form said data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

wherein said image presentation and navigation means includes means for displaying, along with an image, a view of the camera path and an indication of the camera position and orientation when the image was recorded;

wherein camera position and orientation is indicated by a mark on the path oriented as the camera is oriented to point where the camera was pointing; and

wherein the view is a plan view and wherein the mark bears an indication thereon of the yaw angle of the camera.

4. A spatially referenced photographic system comprising:

a data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

image presentation and navigation means for displaying the images to a user and for facilitating the user in navigating among said images by receiving spatial movement commands from the user, as indicated by said spatial movement commands;

a camera capable of capturing data representative of plural images of objects;

position sensing means connected to said camera for capturing data defining the position of said camera at times when images are captured;

orientation sensing means connected to said camera for capturing data defining the orientation of said camera at times when images are captured;

data capturing means for receiving image representation data from said camera, position defining data from said position sensing means, and orientation defining data from said orientation sensing means and for storing all of said data to form said data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

wherein said image presentation and navigation means includes means for displaying, along with an image, a view of the camera path and an indication of the camera position and orientation when the image was recorded; and

wherein a mark appears in said image of a location associated with another image such that the user may signal a desire to navigate forward to view said another image in a simple manner.

5. A spatially referenced photographic system in accordance with claim 4 wherein the path also bears an indication of the location of said another image.

6. A spatially referenced photographic system comprising:

a data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

image presentation and navigation means for displaying the images to a user and for facilitating the user in navigating among said images by receiving spatial movement commands from the user, as indicated by said spatial movement commands;

a camera capable of capturing data representative of plural images of objects;

position sensing means connected to said camera for capturing data defining the position of said camera at times when images are captured;

orientation sensing means connected to said camera for capturing data defining the orientation of said camera at times when images are captured;

data capturing means for receiving image representation data from said camera, position defining data from said position sensing means, and orientation defining data from said orientation sensing means and for storing all of said data to form said data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position; and

wherein said image presentation and navigation means provides the user with navigation controls for moving backward, in response to the actuation of which controls said presentation and navigation means selects an image captured at a generally backward camera position having an orientation similar to that of an image the user is currently viewing.

7. A spatially referenced photographic system comprising:

a data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

image presentation and navigation means for displaying the images to a user and for facilitating the user in navigating among said images by receiving spatial movement commands from the user, as indicated by said spatial movement commands;

a camera capable of capturing data representative of plural images of objects;

position sensing means connected to said camera for capturing data defining the position of said camera at times when images are captured;

orientation sensing means connected to said camera for capturing data defining the orientation of said camera at times when images are captured;

data capturing means for receiving image representation data from said camera, position defining data from said position sensing means, and orientation defining data from said orientation sensing means and for storing all of said data to form said data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position; and

wherein said image presentation and navigation means provides the user with navigation controls for rotating left or right, in response to the actuation of which controls said presentation and navigation means selects an image captured at a generally left-rotated or right-rotated camera position having a position fore-, aft-, and side-to-side similar to that of an image the user is currently viewing.

8. A spatially referenced photographic system in accordance with claim 7 wherein the image which the user is currently viewing bears marks indicating left and right possible rotations which thereby indicate the general location of the viewpoint of said image captured at said generally left-rotated or right-rotated camera positions.

9. A spatially referenced photographic system in accordance with claim 8 wherein the user signals a desire to move generally left or right by mouse clicking on said indicating marks.

10. A spatially referenced photographic system comprising:

a data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

image presentation and navigation means for displaying the images to a user and for facilitating the user in navigating among said images by receiving spatial movement commands from the user, as indicated by said spatial movement commands;

a camera capable of capturing data representative of plural images of objects;

position sensing means connected to said camera for capturing data defining the position of said camera at times when images are captured;

orientation sensing means connected to said camera for capturing data defining the orientation of said camera at times when images are captured;

data capturing means for receiving image representation data from said camera, position defining data from said position sensing means, and orientation defining data from said orientation sensing means and for storing all of said data to form said data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position; and

wherein said image representation and navigation means provides the user with navigation controls for moving forward and simultaneously rotating to the left or to the right, in response to the actuation of which controls said presentation and navigation means selects an image captured at a generally forward camera position having an angular orientation rotated to the left or right of that of an image the user is currently viewing.

11. A spatially referenced photographic system comprising:

a data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position;

image presentation and navigation means for displaying the images to a user and for facilitating the user in navigating among said images by receiving spatial movement commands from the user, as indicated by said spatial movement commands;

a camera capable of capturing data representative of plural images of objects;

position sensing means connected to said camera for capturing data defining the position of said camera at times when images are captured;

orientation sensing means connected to said camera for capturing data defining the orientation of said camera at times when images are captured;

data capturing means for receiving image representation data from said camera, position defining data from said position sensing means, and orientation defining data from said orientation sensing means and for storing all of said data to form said data base containing plural images of objects and also containing information corresponding to said images defining the position at which each image was originally viewed and the orientation of the image with respect to that position; and

said image presentation and navigation means providing the user with navigation controls, in response to the actuation of which controls said presentation and navigation means selects an image captured at a position generally shifted from that of an image the user is currently viewing, as indicated by the user actuation of said controls.