Sensor array module with wide angle, and image calibration method, operation method and application for the same

• US 8,384,789 B2
• Filed: 07/21/2009
• Issued: 02/26/2013
• Est. Priority Date: 07/23/2008
• Status: Active Grant

• Current Assignee: PixArt Imaging Incorporated
• Original Assignee: PixArt Imaging Incorporated
• Inventors: Chao, Tzu Yi, Chen, Hsin Chia, Lin, Chih Hsin, Tsai, Chang Che, Gu, Ren Hau
• Primary Examiner(s): Fan, Chieh M
• Assistant Examiner(s): Berhan, Ahmed A
• Application Number:
  US12/506,714
• Publication Number:
  US 20100020201A1
• Time in Patent Office: 1,316 Days
• Field of Search: 348/36, 348/37, 348/39, 348/72, 348/146, 348/153, 348/159, 348/218.1, 348/211.11, 348/211.12, 348/211.3, 348/208.14, 348/147
• US Class Current: 348/218.1
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen

Abstract

A sensor array module with wide angle for creating a composite image includes a first image sensor, a second image sensor, a storage unit and a processor. The first image sensor is for capturing a first image. The second image sensor is for capturing a second image and has a relative spatial relationship with respective to the first image sensor. The storage unit stores at least one transformation matrix obtained according to the relative spatial relationship between the first and the second image sensors. The processor combines the first image and the second image by using the transformation matrix to create the composite image. The present invention also provides an image calibration method, an operation method and an application for a sensor array module with wide angle.

Claims

1. An image calibration method for a sensor array module with wide angle, the sensor array module comprising a first image sensor and a second image sensor, the image calibration method comprising:

providing at least three reference points, wherein the reference points define a reference coordinate system and each reference point has a reference coordinate;

capturing a first image including at least three reference points with the first image sensor, wherein the first image defines a first image coordinate system and each reference point included in the first image has a first image coordinate;

obtaining a first transformation matrix according to the reference coordinate system and the first image coordinate system;

capturing a second image including at least three reference points with the second image sensor, wherein the second image defines, a second image coordinate system and each reference point included in the second image has a second image coordinate;

obtaining a second transformation matrix according to the reference coordinate system and the second image coordinate system; and

transforming the first image and/or the second image to a virtual plane by using the first transformation matrix and/or the second transformation matrix.

2. The image calibration method as claimed in claim 1, wherein the reference coordinate system, the first image coordinate system and the second image coordinate system are three-dimensional Cartesian coordinate systems.

3. The image calibration method us claimed in claim 1, wherein the virtual plane is based on one of the reference coordinate system, the first image coordinate system and the second image coordinate system.

4. The image calibration method as claimed in claim 3, wherein when the virtual plane is based on the reference coordinate system, transform the first image to the virtual plane by using the first transformation matrix and transform the second image to the virtual plane by using the second transformation matrix.

5. The image calibration method as claimed in claim 3, wherein when the virtual plane is based on the first image coordinate system, transform the second image to the virtual plane by using the first and the second transformation matrices.

6. The image calibration method as claimed in claim 3, wherein when the virtual plane is based on the second image coordinate system, transform the first image to the virtual plane by using the first and the second transformation matrices.

7. A sensor array module with wide angle for creating a composite image, the sensor array module comprising:

a first image sensor for capturing a first image;

a second image sensor for capturing a second image and having a transverse angle with respect to the first image sensor;

a storage unit storing at least one transformation matrix, wherein the at least one transformation matrix is obtained according to the transverse angle between the first image sensor and the second image sensor; and

a processor combining the first image and the second image by using the at least one transformation matrix to create the composite image;

wherein the transverse angle is 2Φ and

the at least one transformation matrix is

8. The sensor array module as claimed in claim 7, further comprising a frame for mounting the first and the second image sensors, such that the transverse angle is defined between the first image sensor and the second image sensor by the frame.

9. The sensor array module as claimed in claim 7, further comprising:

a third image sensor having a first transverse angle with respect to the first image sensor and a fourth image sensor having a second transverse angle with respect to the first image sensor, wherein a first transformation matrix obtained according to the first transverse angle and a second transformation matrix obtained according to the second transverse angle are stored in the storage unit, and the processor combines images captured by the first and second, third and fourth image sensors by using the at least one transformation matrix, the first transformation matrix and second transformation matrix to create the composite image.

10. A sensor array module with wide angle for creating a composite image, the sensor array module comprising:

a first image sensor for capturing a first image;

a second image sensor for capturing a second image and having a relative spatial relationship with respect to the first image sensor, a storage unit storing at least one transformation matrix, wherein the at least one transformation matrix is obtained according to the relative spatial relationship between the first image sensor and the second image sensor; and

a processor combining the first image and the second image by using the at least one transformation matrix to create the composite image;

wherein the relative spatial relationship is a transverse angle 2Φ and

a longitudinal angle θ

between the first image sensor and the second image sensor, and the at least one transformation matrix is

11. The sensor array module as claimed in claim 10, further comprising a frame for mounting the first and the second image sensors, such that the relative spatial relationship is defined between the first image sensor and the second image sensor by the frame.

12. The sensor array module as claimed in claim 10, further comprising:

a third image sensor having a first relative spatial relationship with respect to the first image sensor, and a fourth image sensor having a second relative spatial relationship with respect to the first image sensor,

wherein a first transformation matrix is obtained according to the first relative spatial relationship and a second transformation matrix is obtained according to the second relative spatial relationship are stored in the storage unlit, and the processor combines images captured by the first, second, third and fourth image sensors by using the at least one transformation matrix, the first transformation matrix and second transformation matrix to create the composite image.