METHOD AND SYSTEM OF LENS SHIFT CORRECTION FOR A CAMERA ARRAY

• US 20160191759A1
• Filed: 03/17/2015
• Published: 06/30/2016
• Est. Priority Date: 12/29/2014
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Somanath, Gowri, Nestares, Oscar, Seshadrinathan, Kalpana, Wu, Yi
• Granted Patent: US 9,628,695 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

A system, article, and method of lens shift correction for a camera array.

Claims

1. A computer-implemented method of lens shift correction for a camera array comprising:

obtaining image data of multiple cameras wherein at least one of the cameras is to perform autofocus;

determining shift parameters to shift the position of misplaced points of individual images of the autofocus camera and to correct positions determined by using corresponding points on images of non-autofocus cameras of the multiple cameras; and

warping the images from the autofocus camera by using the parameters to modify the positions of the misplaced points on the images of the autofocus camera.

2. The method of claim 1 wherein the shift parameters indicate a correction to the misplaced points as if the misplaced points were positioned by using a focal length, principal point, or both that were previously used for a calibration.

3. The method of claim 1 wherein the parameters are different from image to image of the autofocus camera.

4. The method of claim 1 wherein the parameters are determined by using an over-determined linear system.

5. The method of claim 1 wherein the parameters are determined by minimizing a rectification error equation.

6. The method of claim 5 wherein the error equations are placed in matrix multiplication form with the parameter variables to be determined.

7. The method of claim 1 comprising:

establishing at least two epipolar lines each extending from a corresponding point of a misplaced point and from an image of a non-autofocus camera, and extending to the image of the autofocus camera, wherein the intersection of the epipolar lines on the image of the autofocus camera is the correct position of the misplaced point.

8. The method of claim 7 comprising determining a distance from the misplaced point to each of the epipolar lines on the image of the autofocus camera;

and

using the distances to compute at least one of the shift parameters.

9. The method of claim 1 wherein the cameras are in a non-collinear arrangement.

10. The method of claim 1 wherein the shift parameters include a 2D displacement in a direction substantially parallel to a plane of a sensor of the autofocus camera, or scale substantially in a direction of the optical axis of the autofocus camera.

11. The method of claim 1 comprising warping rectified images of the autofocus camera with the parameters.

12. The method of claim 1 comprising warping unrectified high resolution images of the autofocus camera with the parameters.

13. The method of claim 1 wherein the shift parameters indicate a correction to the misplaced points as if the misplaced points were positioned by using a focal length, principal point, or both that were previously used for a calibration;

wherein the parameters are different from image to image of the autofocus camera;

wherein the parameters are determined by using an over-determined linear system;

wherein the parameters are determined by minimizing a rectification error equation;

wherein the error equations are placed in matrix multiplication form with the parameter variables to be determined;

the method of comprising;

establishing at least two epipolar lines each extending from a corresponding point of a misplaced point and from an image of a non-autofocus camera, and extending to the image of the autofocus camera, wherein the intersection of the epipolar lines on the image of the autofocus camera is the correct position of the misplaced point;

determining a distance from the misplaced point to each of the epipolar lines on the image of the autofocus camera; and

using the distances to compute at least one of the shift parameters;

wherein the cameras are in a non-collinear arrangement;

wherein the shift parameters include a 2D displacement in a direction substantially parallel to a plane of a sensor of the autofocus camera, or scale substantially in a direction of the optical axis of the autofocus camera;

the method comprising at least one of;

warping rectified images of the autofocus camera with the parameters; and

warping unrectified high resolution images of the autofocus camera with the parameters.

14. A computer-implemented system of lens shift correction, comprising:

a display;

at least one processor communicatively coupled to the display;

at least one memory communicatively coupled to at least one processor; and

a lens shift correction unit operated by the processor and to;

obtain image data of multiple cameras wherein at least one of the cameras is to perform autofocus;

determine shift parameters to shift the position of misplaced points of individual images of the autofocus camera and to correct positions determined by using corresponding points on images of non-autofocus cameras of the multiple cameras; and

warp the images from the autofocus camera by using the parameters to modify the positions of the misplaced points on the images of the autofocus camera.

15. The system of claim 14 wherein the shift parameters indicate a correction to the misplaced points as if the misplaced points were positioned by using a focal length, principal point, or both that were previously used for a calibration.

16. The system of claim 14 wherein the parameters are different from image to image of the autofocus camera.

17. The system of claim 14 wherein the parameters are determined by using an over-determined linear system.

18. The system of claim 14 wherein the parameters are determined by minimizing a rectification error equation.

19. The system of claim 14 wherein the lens shift correction unit is to:

establish at least two epipolar lines each extending from a corresponding point of a misplaced point and from an image of a non-autofocus camera, and extending to the image of the autofocus camera, wherein the intersection of the epipolar lines on the image of the autofocus camera is the correct position of the misplaced point.

20. The system of claim 19 wherein the lens shift correction unit is to determine a distance from the misplaced point to each of the epipolar lines on the image of the autofocus camera;

and

use the distances to compute at least one of the shift parameters.

21. The system of claim 19 wherein the cameras are in a non-collinear arrangement.

22. The system of claim 19 wherein the shift parameters include a 2D displacement in a direction substantially parallel to a plane of a sensor of the autofocus camera, or scale substantially in a direction of the optical axis of the autofocus camera.

23. The system of claim 14 wherein the shift parameters indicate a correction to the misplaced points as if the misplaced points were positioned by using a focal length, principal point, or both that were previously used for a calibration;

wherein the parameters are different from image to image of the autofocus camera;

wherein the parameters are determined by using an over-determined linear system;

wherein the parameters are determined by minimizing a rectification error equation;

wherein the error equations are placed in matrix multiplication form with the parameter variables to be determined;

wherein the lens shift correction unit is to;

establish at least two epipolar lines each extending from a corresponding point of a misplaced point and from an image of a non-autofocus camera, and extending to the image of the autofocus camera, wherein the intersection of the epipolar lines on the image of the autofocus camera is the correct position of the misplaced point;

determine a distance from the misplaced point to each of the epipolar lines on the image of the autofocus camera; and

use the distances to compute at least one of the shift parameters;

wherein the cameras are in a non-collinear arrangement;

wherein the shift parameters include a 2D displacement in a direction substantially parallel to a plane of a sensor of the autofocus camera, or scale substantially in a direction of the optical axis of the autofocus camera;

wherein the lens shift correction unit is to at least one of;

warp rectified images of the autofocus camera with the parameters; and

warp unrectified high resolution images of the autofocus camera with the parameters.

24. A computer-readable medium having stored thereon instructions that when executed cause a computing device to:

obtain image data of multiple cameras wherein at least one of the cameras is to perform autofocus;

determine shift parameters to shift the position of misplaced points of individual images of the autofocus camera and to correct positions determined by using corresponding points on images of non-autofocus cameras of the multiple cameras; and

warp the images from the autofocus camera by using the parameters to modify the positions of the misplaced points on the images of the autofocus camera.

25. The medium of claim 24 wherein the shift parameters indicate a correction to the misplaced points as if the misplaced points were positioned by using a focal length, principal point, or both that were previously used for a calibration;

wherein the parameters are different from image to image of the autofocus camera;

wherein the parameters are determined by using an over-determined linear system;

wherein the parameters are determined by minimizing a rectification error equation;

wherein the error equations are placed in matrix multiplication form with the parameter variables to be determined;

wherein the instructions cause the computing device to;

establish at least two epipolar lines each extending from a corresponding point of a misplaced point and from an image of a non-autofocus camera, and extending to the image of the autofocus camera, wherein the intersection of the epipolar lines on the image of the autofocus camera is the correct position of the misplaced point;

determine a distance from the misplaced point to each of the epipolar lines on the image of the autofocus camera; and

use the distances to compute at least one of the shift parameters;

wherein the cameras are in a non-collinear arrangement;

wherein the shift parameters include a 2D displacement in a direction substantially parallel to a plane of a sensor of the autofocus camera, or scale substantially in a direction of the optical axis of the autofocus camera;

wherein the instructions cause the computing device to at least one of;

warp rectified images of the autofocus camera with the parameters; and

warp unrectified high resolution images of the autofocus camera with the parameters.