Apparatus for referencing and sorting images in a three-dimensional system

• US 6,292,215 B1
• Filed: 07/24/1997
• Issued: 09/18/2001
• Est. Priority Date: 01/31/1995
• Status: Expired due to Term

• Current Assignee: Transcenic Incorporated
• Original Assignee: Transcenic Incorporated
• Inventors: Vincent, Robert S.
• Primary Examiner(s): Garber, Wendy
• Assistant Examiner(s): Vu, Ngoc-Yen, VU, NGOC YEN T
• Application Number:
  US08/900,046
• Time in Patent Office: 1,517 Days
• Field of Search: 348/61, 348/13, 348/15, 348/135, 348/142, 348/143, 348/169, 348/207, 348/211, 348/213, 348/231-233, 348/333, 348/334, 348/373-375, 348/552, 358/906, 358/909.1, 396/13, 396/429, 396/419, 396/427, 386/95, 386/107, 386/108, 318/649, 381/74, 600/587
• US Class Current: 348/169
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/9201 involving the multiplexing of an additional signal and the video signal
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

An image system which captures, along with the images, information defining both the position and the orientation of the camera along with the distance to the subject. A video camera is attached to three accelerometers, two gyroscopes, and a rangefinder. Data gathered from these devices and defining the pitch, yaw, and roll of the camera, the camera'"'"'s acceleration, and the distance to the subject is captured and recorded along with video images. The video images are later stored within a computer'"'"'s data base along with data defining the position and orientation of the camera and the distance to the subject for each image, this latter data being computed from the captured data. The images may then be presented to the user in a three-dimensional display in which the user can navigate through the images using a joystick device, with the images located in positions corresponding to the positions in space of the objects that were imaged.

Claims

1. A spatially referenced camera comprising:

a camera capable of capturing data representative of plural images of objects;

a camera orientation sensor connected to said camera and generating data defining the orientation of said camera;

a position sensor which includes a plurality of accelerometers rigidly attached to said camera and generating data defining the position of said camera when each image is captured, said position defining data needing adjustment relative to said orientation defining data to compensate for camera orientation during motion; and

a data capture device that captures image representation data from said camera, position defining data from said position sensor, and orientation defining data from said orientation sensor and that stores said data.

2. A spatially referenced camera in accordance with claim 1 wherein the camera is a video camera.

3. A spatially referenced camera in accordance with claim 2 wherein the data capture device utilizes a magnetic storage medium having the capacity to store video images.

4. A spatially referenced camera in accordance with claim 1 wherein said spatially referenced camera, position sensor, and orientation sensor are integrated into a single compact, portable unit.

5. A spatially referenced camera in accordance with claim 1 wherein the data capture device utilizes magnetic storage medium, and wherein said image representation data from said camera and said position and orientation defining data are stored on portions of said magnetic storage medium with the position and orientation defining data associated with the corresponding image representation data.

6. A spatially referenced camera in accordance with claim 1 wherein said accelerometer data must be integrated twice and corrected for camera orientation during motion to convert it into data indicating camera position.

7. A spatially referenced camera in accordance with claim 1 wherein said orientation sensor comprises gyroscopes.

8. A spatially referenced camera in accordance with claim 7 wherein said gyroscopes are rigidly attached to said camera, and where said camera orientation defining data is derived from rotational forces applied to and sensed by said gyroscopes.

9. A spatially referenced camera in accordance with claim 1 wherein said orientation and position defining data is captured by a microprocessor-based tracking data acquisition unit, bundled into digital packets suitable for transmission, and transmitted to said data capture device.

10. A spatially referenced camera in accordance with claim 1 which further includes a range finder attached to said camera for generating range data indicating the distance to said objects, and wherein said range data is also captured by said data capture device.

11. A spatially referenced camera in accordance with claim 10 wherein object position computational means are provided to compute the position of said objects from said range, orientation defining, and position defining data.

12. A spatially referenced camera in accordance with claim 1 to which is added a computer which accepts all the data from said data capture device and which includes image retrieval and display means for facilitating the retrieval of the data representative of any of said images, and the display of the images, based on the position of the camera when the image data was captured, said retrieval and display means utilizing data provided by said data capture device to control the display of visual navigation aids to one viewing the images on the display means.

13. A spatially referenced camera comprising:

a video camera and data recorder capable of capturing video data representative of plural images of objects and storing said video data on a magnetic storage medium;

orientation sensing gyroscopes rigidly attached to said camera by means of gimbals and generating gimbal rotational position data to serve as camera orientation defining data;

position sensing accelerometers rigidly attached to said camera to define the position of said camera when each image is captured,

a processor programmed to use said orientation defining data to adjust accelerometer-generated position defining data to compensate for camera orientation during motion; and

a tracking data acquisition unit attached to said camera and arranged to receive said position and orientation defining data and to present said data for storage on said magnetic storage medium.

14. A spatially referenced camera in accordance with claim 13 which further includes processor means for double integrating said position de fining data in to data indicating camera position.

15. A spatially referenced camera in accordance with claim 13 to which is added a computer which accepts all the data from said data acquisition unit and which includes image retrieval and display means for facilitating the retrieval of the data representative of any of said images, and the display of the image, based on the position of the camera when the image data was captured, said retrieval and display means utilizing data provided by said data acquisition unit to control the display of visual navigation aids to one viewing the images on the display means.

16. A spatially referenced camera in accordance with claim 15 which further includes a range finder rigidly attached to said camera and generating range data defining the range to objects being photographed by said camera, which data is also received by said tracking data acquisition unit.

17. A spatially referenced camera in accordance with claim 16 which further includes processor means for adjusting said accelerometer-generated position defining data to compensate for camera orientation using said orientation defining data, for double integrating said position defining data into data indicating camera position, and for computing from said data indicating camera position, said orientation defining data, and said range data the position of said photographed objects.

18. A spatially referenced camera in accordance with claim 17 to which is added a computer that accepts video data from said data acquisition unit and object position data computed by said processor means and which includes image retrieval and display means for facilitating the retrieval of the data representative of any of said images, and the display of the images, based on the position of the object when the image data was captured.

19. A spatially referenced camera comprising:

a cameral capable of capturing data representative of plural images of objects;

an orientation sensor connected to said camera for capturing data defining the orientation of said camera, said orientation of said camera including rotation about three axes of rotation;

a position sensor which includes a plurality of accelerometers rigidly attached to said camera to define the position of said camera when images are captured, said position sensor adapted to be adjusted relative to said orientation defining data to compensate for camera orientation during motion; and

a data capturing device that receives image representation data from said camera, position defining data from said position sensor, and orientation defining data from said orientation sensor and for storing all of said data.