Image processing system for integrating multi-resolution images

• US 7,719,568 B2
• Filed: 12/16/2006
• Issued: 05/18/2010
• Est. Priority Date: 12/16/2006
• Status: Active Grant

• Current Assignee: National Chiao Tung University (Government of The Republic of China)
• Original Assignee: National Chiao Tung University (Government of The Republic of China)
• Inventors: Yeh, Wei-Shian, Hung, Yi-Ping
• Primary Examiner(s): Ometz; David L
• Assistant Examiner(s): Bemben; Richard M
• Application Number:
  US11/611,851
• Publication Number:
  US 20080143821A1
• Time in Patent Office: 1,249 Days
• Field of Search: 348143-160, 348/218.1, 348/222.1, 382/284
• US Class Current: 348/159
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The image processing system for integrating multi-resolution images mainly applies several different multi-resolution image capture devices to extract the images from the observed scene and using the multi-resolution imaging devices to present the scene images relative to the observed scenes, respectively. In order to present a seamless image according to two different-resolution image sources, the coordinate conversions among the image capture devices, and among the imaging devices are mainly applied to extract and present the entire image to be able to resemble having a single image capturing source and a single imaging source.

Claims

1. An image processing system for integrating multi-resolution images using a plurality of multi-resolution image capture devices to extract images from an observed scene, and using a plurality of multi-resolution imaging devices to present a scene image relative to the observed scene, comprising:

an imaging plane;

a wide-angle image capture device, for capturing the observed scene according to a first resolution and producing a wide-angle image;

a Pan-Tilt-Zoom (PTZ) image capture device, for capturing the observed scene according to a second resolution, a designated direction, and a focal distance and producing a Pan-Tilt-Zoom image, wherein the second resolution is higher than the first resolution;

a fixed imaging device, for presenting a processed wide-angle image on the imaging plane according to a third resolution, wherein the processed wide-angle image has at least one reserved region without image information;

a steerable imaging device, for presenting a processed Pan-Tilt-Zoom image on the imaging plane according to a fourth resolution, wherein the processed Pan-Tilt-Zoom image is only presented in a region corresponding to the reserved region of the processed wide-angle image, wherein the fourth resolution is higher than the third resolution;

a conversion module, for converting the Pan-Tilt-Zoom image to a conversion image which is in a coordinate system of the wide-angle image capture device based on an image capture device conversion matrix (homography;

H_wide^PTZ); and

an imaging module, for applying the fixed imaging device to present the processed wide-angle image on the imaging plane according to the third resolution, and applying the coordinate system of the steerable imaging device to present the processed Pan-Tilt-Zoom image on the imaging plane according to the fourth resolution based on an imaging device conversion matrix (H_steer^fixed), wherein the scene images relative to the observed scenes are presented;

wherein said image processing system further comprises;

an imaging corrected pattern module for the fixed imaging device and the steerable imaging device to respectively present a corrected pattern;

a corrected pattern acquisition module for the Pan-Tilt-Zoom (PTZ) image capture device to capture the corrected patterns presented in the fixed imaging device and the steerable imaging device and to respectively obtain a corrected image;

a conversion matrix module, for calculating a first extraction imaging conversion matrix (H_PTZ^fixed) according to the corrected image from the steerable imaging device, and calculating a second extraction imaging conversion matrix (H_steer^PTZ) according to the corrected image from the fixed imaging device; and

an imaging device conversion module, for obtaining the imaging device conversion matrix (H_steer^fixed) according to the first extraction imaging conversion matrix (H_PTZ^fixed) and the second extraction imaging conversion matrix (H_steer^PTZ).

2. The image processing system for integrating multi-resolution images as claimed in claim 1, wherein the image capture device conversion matrix (homography;

H_wide^PTZ) is a coordinate conversion matrix between the wide-angle image capture device and the Pan-Tilt-Zoom (PTZ) image capture device.

3. The image processing system for integrating multi-resolution images as claimed in claim 1, wherein the imaging device conversion matrix (H_steer^fixed) is a coordinate conversion matrix between the fixed imaging device and the steerable imaging device.

4. The image processing system for integrating multi-resolution images as claimed in claim 1, further comprising:

a laser pen, used by a user to generate a specified point on the imaging plane including the wide-angle image; and

an image capture device, for at least extracting one Pan-Tilt-Zoom image including the specified point from the imaging plane.