Camera system with both a wide angle view and a high resolution view

• US 6,738,073 B2
• Filed: 04/30/2002
• Issued: 05/18/2004
• Est. Priority Date: 05/12/1999
• Status: Expired due to Term

• Current Assignee: Immersive Licensing, Inc.
• Original Assignee: iMove Inc. (Digital Domain Holdings Ltd.)
• Inventors: Park, Michael C., Ripley, G. David
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Santiago, Enrique L
• Application Number:
  US10/136,659
• Publication Number:
  US 20020180759A1
• Time in Patent Office: 749 Days
• Field of Search: 345/418, 345/629, 345/474, 345/660, 348/38, 348/39, 348/36, 141/94
• US Class Current: 345/629
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/8047 using transform coding
• CPC Class Codes: H04N 9/806 with processing of the sound signal
• CPC Class Codes: H04N 9/8211 the additional signal being a sound signal H04N9/835 takes precedence
• CPC Class Codes: H04N 9/8227 the additional signal being at least another television signal
• CPC Class Codes: H04N 9/8233 the additional signal being a character code signal

Abstract

An improved surveillance system which includes a multi-lens camera system and a viewer. The camera system includes a plurality of single lens cameras each of which have a relatively wide angle lens. These single lens cameras simultaneously capture images that can be seamed into a panorama. The camera system also includes a high resolution camera (i.e. a camera with a telephoto lens) that can be pointed in a selected direction that is within the field of view of the other cameras. The system displays a view window into the panorama that is created from images captured by the wide angle lenses. The image from the high resolution camera is either superimposed on top of the panoramic image or it is displayed on a separate monitor. If it is displayed in the panorama, the higher solution image is positioned at the point in the panorama which is displaying the same area in space at a lower resolution. Thus an operator sees a relatively low resolution panorama; however, a selected portion of the panorama is displayed at a high resolution. An operator can point the high resolution camera toward any desired location, thereby providing an output which shows more detail in a selected area of the panorama.

Claims

1. An image capture and rendering system, comprising:

a first image capture device for capturing a first image at a first resolution, the first image containing an object;

a second image capture device for capturing a second image at a second resolution higher than the first resolution, the second image containing at least a portion of the object; and

a rendering system for superimposing the second image on the first image, wherein the second image is oriented within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

2. The system of claim 1, wherein the first image capture device and the second image capture device are mounted on a platform that moves relative to the object, and the rendering system uses metadata indicative of the movement to compensate at least one of the first image and the second image.

3. The system of claim 1, wherein the first image is a panoramic image generated by seaming together a plurality of images captured by the first image capture device.

4. The system of claim 1, wherein the rendering system superimposes the second image on the first image using metadata associated with at least one of the first image and the second image, wherein metadata is used to orient the second image within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

5. The system of claim 1, wherein the second image capture device captures a third image having an image resolution that is higher than the resolution of the first image and lower than the resolution of the second image, and the rendering system superimposes the second image over the third image, which is superimposed over the first image.

6. The system of claim 1, wherein the first image capture device comprises a plurality of single lens cameras for capturing a plurality of images which are seamed together into a panoramic image, and the rendering system blends the images along seam lines between the images to minimize artifacts around the seam lines.

7. The system of claim 1, wherein the position of the second image capture device is controlled by a sensor system and the rendering system superimposes sensor data associated with the object on at least one of the first and second images.

8. The system of claim 1, wherein calibration data is used to orient the second image within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

9. An image capture and rendering system, comprising:

a first image capture device for capturing a first image of a scene containing an object;

a second image capture device for capturing a number of images of the object, wherein each image contains at least a portion of the object; and

a rendering system for seaming together the number of images of the object into a composite object image and superimposing the composite object image on the first image, wherein the composite object image is oriented within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

10. An image capture and rendering system, comprising:

a first image capture device for capturing a first image at a first resolution, the first image containing an object;

a second image capture device for capturing a second image at a second resolution higher than the first resolution, the second image containing at least a portion of the object; and

a rendering system for superimposing the second image on the first image, wherein pattern matching is used to orient the second image within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

11. A method of capturing and rendering images, comprising:

capturing a first image at a first image resolution, the first image containing an object;

capturing a second image at a second image resolution higher than the first image resolution, the second image containing at least a portion of the object; and

superimposing the second image on the first image using metadata associated with at least one of the first image and the second image, wherein the second image is oriented within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

12. The method of claim 11, further comprising:

capturing a third image having an image resolution that is higher than the resolution of the first image and lower than the resolution of the second image;

superimposing the second image over the third image; and

superimposing the third image over the first image.

13. The method of claim 11, further comprising:

capturing a plurality of images;

seaming together the plurality of images into a panoramic image; and

blending seam lines between the images to minimize artifacts around the seam lines.

14. The method of claim 11, further comprising:

superimposing sensor data associated with the object on at least one of the first and second images.

15. The method of claim 11, further comprising:

using calibration data to orient the second image within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

16. A method of capturing and rendering images, comprising:

capturing a first image of a scene containing an object to be imaged;

capturing a number of images of the object, wherein each image contains a portion of the object;

seaming together the number of images of the object into a composite object image; and

superimposing the composite object image on the first image, wherein the composite object image is oriented within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

17. A method of capturing and rendering images, comprising:

capturing a first image at a first resolution, the first image containing an object;

capturing a second image at a second resolution higher than the first resolution, the second image containing a portion of the object; and

superimposing the second image on the first image, wherein pattern matching is used to orient the second image within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

18. A computer readable medium having instructions contained thereon, which, when executed by a processor in an image rendering system, cause the processor to perform the operations of:

receiving a first image at a first resolution, the first image containing an object;

receiving a second image at a second resolution higher than the first image resolution, the second image containing at least a portion of the object; and

superimposing the second image on the first image, wherein the second image is oriented within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

19. A computer readable medium having contained thereon instructions, which, when executed by a processor in a image rendering system, cause the processor to perform the operations of:

receiving a first image of a scene containing an object to be imaged from a first image capture device;

receiving a number of images of the object from a second image capture device, wherein each image contains at least a portion of the object;

seaming together the number of images of the object into a composite object image; and

superimposing the composite object image on the first image using metadata associated with at least one of the first image and the composite object image, wherein the composite object image is oriented within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

20. A computer readable medium containing instructions thereon, which, when executed by a processor in an image rendering system, cause the processor to perform the operations of:

receiving a first image at a first resolution from a first image captured device, the first image containing an object;

receiving a second image from a second image capture device, the second image having a second resolution higher than the first resolution and containing at least a portion of the object; and

superimposing the second image on the first image, wherein pattern matching is used to orient the second image within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.

21. An image capture and rendering system, comprising:

means for capturing a first image at a first resolution, the first image containing an object;

means for capturing a second image at a second resolution higher than the first image resolution, the second image containing at least a portion of the object; and

means for superimposing the second image on the first image, wherein the second image is oriented within the first image such that portions of the object are substantially continuous across boundaries between the first image and the second image.