Integrated image surveillance system and image synthesis method thereof

• US 8,547,434 B2
• Filed: 02/13/2009
• Issued: 10/01/2013
• Est. Priority Date: 04/10/2008
• Status: Active Grant

• Current Assignee: National Chiao Tung University (Government of The Republic of China)
• Original Assignee: National Chiao Tung University (Government of The Republic of China)
• Inventors: Chen, Yong-Sheng, Cheng, Yung-Cheng, Lin, Kai-Ying
• Primary Examiner(s): Feild, Lynn
• Assistant Examiner(s): Liu, Lin
• Application Number:
  US12/370,668
• Publication Number:
  US 20090256908A1
• Time in Patent Office: 1,691 Days
• Field of Search: 348/143, 348/159
• US Class Current: 348/143
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Disclosed is an integrated multi-view surveillance system which integrates multiple surveillance camera images of an area into a large-coverage image of the area in order to monitor the area. The system includes a first camera, a second camera, a third camera, an image processing device, and a display device. The image processing device includes a first defining component, a first synthesis component, a second defining component, an adjusting component, a transforming component, a third defining component, and a second synthesis component.

Claims

1. An integrated image surveillance system, comprising:

a first camera for capturing a first image of an area from a first viewpoint, wherein the first image is a static, large-coverage image of the area;

a second camera for capturing a second image of the area from a second viewpoint displaced from the first viewpoint, wherein the second image is a dynamic, small-coverage image of the area, and the second image and the first image have a first overlap region;

an image processing device, comprising;

a first defining component, configured to define a first set of feature points in the first image and a second set of feature points in the second image, wherein the first set of feature points and the second set of feature points have a first registration relation, and the first registration relation further defines a first registration function for the first image and the second image;

a first synthesis component, configured to integrate the first image and the second image according to the first registration function to form a third image, wherein, upon overlapping of the first image and the second image, data of the second image are shown in the third image; and

a second defining component, configured to define the first image into first set of image points each having a first image chromaticity, the second image into second set of image points each having a second image chromaticity, and the third image into third set of image points each having a third image chromaticity;

an adjusting component, configured to adjust discontinuous image chromaticity in the third image on two sides of a seam between the first image and the second image;

a transforming component, configured to transform the third image synthesized from the first image and the second image from a two-dimensional image to a three-dimensional image for better spatial perception; and

a display device for displaying the third image.

2. The integrated image surveillance system of claim 1, wherein the adjusting component extracts an appropriate range around the seam as a stitching area, the stitching area being at least part of the first overlap region of the first set of image points and the second set of image points;

and mixes the first image chromaticities of the first set of image points with the second image chromaticities of the second set of image points at an appropriate ratio to produce corrected third image chromaticities for the third set of image points in the stitching area.

3. The integrated image surveillance system of claim 1, further comprising a third camera for capturing a fourth image of the area, the fourth image being a dynamic, small-coverage image of the area, wherein the fourth image and the second image have a second overlap region, and the second overlap region and the first overlap region have a third overlap region.

4. The integrated image surveillance system of claim 3, wherein the image processing device further comprises a third defining component, configured to define fourth set of feature points in a portion of the fourth image that is in the third overlap region, and third set of feature points in a portion of the second image that is in the third overlap region, in which the fourth set of feature points and the third set of feature points have a second registration relation, and the second registration relation further defines a second registration function for the fourth image and the second image.

5. The integrated image surveillance system of claim 4, wherein the image processing device further comprises a second synthesis component, configured to integrate the second image and the fourth image according to the second registration function so as to form a fifth image.

6. The integrated image surveillance system of claim 5, wherein, upon overlapping of the fourth image and the second image, the data of the second image or the data of the fourth image are shown in the fifth image;

and the display device displays the fifth image.

7. An image synthesis method for use in a surveillance system, comprising steps of:

capturing a first image of an area by a first camera at a first viewpoint, wherein the first image is a static, large-coverage image of the area;

capturing a second image of the area by a second camera at a second viewpoint displaced from the first viewpoint, wherein the second image is a dynamic, small-coverage image of the area, and the second image and the first image have a first overlap region;

providing an image processing step comprising;

defining a first set of feature points in the first image and a second set of feature points in the second image, wherein the first set of feature points of the first image and the second set of feature points in the second image have a first registration relation, and the first registration relation further defines a first registration function for the first image and the second image;

integrating the first image and the second image according to the first registration function to form a third image, wherein, upon overlapping of the first image and the second image, data of the second image are shown in the third image; and

defining the first image into first set of image points each having a first image chromaticity, the second image into second set of image points each having a second image chromaticity, and the third image into third set of image points each having a third image chromaticity;

providing an adjusting procedure for adjusting discontinuous image chromaticity on two sides of a seam between the first image and the second image;

transforming the third image synthesized from the first image and the second image from a two-dimensional image to a three-dimensional image for better spatial perception; and

displaying the third image.

8. The image synthesis method of claim 7, wherein the adjusting procedure comprises extracting an appropriate range around the seam as a stitching area, the stitching area being at least part of the first overlap region of the first set of image points and the second set of image points;

and mixing the first image chromaticities of the first set of image points with the second image chromaticities of the second set of image points at an appropriate ratio to produce corrected third image chromaticities for the third set of image points in the stitching area.

9. The image synthesis method of claim 7, further comprising a step of capturing a fourth image of the area by a third camera, the fourth image being a dynamic, small-coverage image of the area, wherein the fourth image and the second image have a second overlap region, and the second overlap region and the first overlap region have a third overlap region.

10. The image synthesis method of claim 9, wherein the image processing step further comprises defining fourth set of feature points in a portion of the fourth image that is in the third overlap region, and third set of feature points in a portion of the second image that is in the third overlap region, in which the fourth set of feature points and the third set of feature points have a second registration relation, and the second registration relation further defines a second registration function for the fourth image and the second image.

11. The image synthesis method of claim 10, wherein the image processing step further comprises integrating the second image and the fourth image according to the second registration function so as to form a fifth image.

12. The image synthesis method of claim 11, wherein, upon overlapping of the fourth image and the second image, the data of the second image or the data of the fourth image are shown in the fifth image;

and the fifth image is displayed.

13. An integrated image surveillance system, comprising:

a first camera for capturing a first image of an area from a first viewpoint, wherein the first image is a low-resolution image of the area;

a second camera for capturing a second image of the area from a second viewpoint displaced from the first viewpoint, wherein the second image is a high-resolution image of the area, and the second image and the first image have a first overlap region;

an image processing device, comprising;

a first defining component, configured to define a first set of feature points in the first image and a second set of feature points in the second image, wherein the first set of feature points and the second set of feature points have a first registration relation, and the first registration relation further defines a first registration function for the first image and the second image;

a first synthesis component, configured to integrate the first image and the second image according to the first registration function to form a third image, wherein, upon overlapping of the first image and the second image, data of the second image are shown in the third image; and

a second defining component, configured to define the first image into first set of image points each having a first image chromaticity, the second image into second set of image points each having a second image chromaticity, and the third image into third set of image points each having a third image chromaticity;

an adjusting component, configured to adjust discontinuous image chromaticity in the third image on two sides of a seam between the first image and the second image;

a transforming component, configured to transform the third image synthesized from the first image and the second image from a two-dimensional image to a three-dimensional image for better spatial perception; and

a display device for displaying the third image.

14. An image synthesis method for use in a surveillance system, comprising steps of:

capturing a first image of an area by a first camera at a first viewpoint, wherein the first image is a low-resolution image of the area;

capturing a second image of the area by a second camera at a second viewpoint displaced from the first viewpoint, wherein the second image is a high-resolution image of the area, and the second image and the first image have a first overlap region;

providing an image processing step comprising;

defining a first set of feature points in the first image and a second set of feature points in the second image, wherein the first set of feature points of the first image and the second set of feature points in the second image have a first registration relation, and the first registration relation further defines a first registration function for the first image and the second image;

integrating the first image and the second image according to the first registration function to form a third image, wherein, upon overlapping of the first image and the second image, data of the second image are shown in the third image; and

defining the first image into first set of image points each having a first image chromaticity, the second image into second set of image points each having a second image chromaticity, and the third image into third set of image points each having a third image chromaticity;

providing an adjusting procedure for adjusting discontinuous image chromaticity on two sides of a seam between the first image and the second image;

transforming the third image synthesized from the first image and the second image from a two-dimensional image to a three-dimensional image for better spatial perception; and

displaying the third image.