Stereo yaw correction using autofocus feedback

• US 10,178,373 B2
• Filed: 04/11/2014
• Issued: 01/08/2019
• Est. Priority Date: 08/16/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Ramachandra, Vikas, Atanassov, Kalin Mitkov, Velarde, Ruben Manuel
• Primary Examiner(s): Vazquez Colon, Maria E
• Application Number:
  US14/250,798
• Publication Number:
  US 20150049172A1
• Time in Patent Office: 1,733 Days
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras

Abstract

Systems and methods for correcting stereo yaw of a stereoscopic image sensor pair using autofocus feedback are disclosed. A stereo depth of an object in an image is estimated from the disparity of the object between the images captured by each sensor of the image sensor pair. An autofocus depth to the object is found from the autofocus lens position. If the difference between the stereo depth and the autofocus depth is non zero, one of the images is warped and the disparity is recalculated until the stereo depth and the autofocus depth to the object is substantially the same.

Claims

1. A system for digitally correcting for a physical misalignment between a first imaging sensor and a second imaging sensor, comprising:

at least one processor; and

a memory having stored thereon instructions that, when executed, cause the at least one processor to;

obtain a first image data of an object from the first imaging sensor;

obtain a second image data of the object from the second imaging sensor;

determine a stereoscopic distance to the object based upon a determined disparity of the object within the first image data and the second image data;

determine an autofocus distance to the object based upon autofocus lens positions of the first imaging sensor and the second imaging sensor using a high frequency map;

compare the stereoscopic distance and the autofocus distance to determine a misalignment difference; and

correct for the physical misalignment between the first imaging sensor and the second imaging sensor based upon the determined misalignment difference.

2. The system of claim 1, the memory further having stored thereon instructions that, when executed, cause the processor to determine the disparity of the object by comparing the first image data and the second image data.

3. The system of claim 1, the memory further having stored thereon instructions that, when executed, cause the processor to determine the stereoscopic distance using the first image data and the second image data based on comparing keypoints of the object located in both the first image data and the second image data.

4. The system of claim 1, the memory further having stored thereon instructions that, when executed, cause the processor to perform autofocus functions to determine and set the autofocus lens positions of the first imaging sensor and the second imaging sensor.

5. The system of claim 1, the memory further having stored thereon instructions that, when executed, cause the processor to warp one of the first image data and the second image data to reduce the disparity of the object if a difference between the stereoscopic distance and the autofocus distance is not zero.

6. A method for digitally correcting for a physical misalignment between a first imaging sensor and a second imaging sensor using autofocus feedback, comprising:

obtaining a first image data of an object from the first imaging sensor;

obtaining a second image data of the object from the second imaging sensor;

determining a stereoscopic distance to the object based upon a determined disparity of the object within the first image data and the second image data;

determining an autofocus distance to the object based upon autofocus lens positions of the first imaging sensor and the second imaging sensor using a high frequency map;

comparing the stereoscopic distance and the autofocus distance to determine a misalignment difference; and

correcting for the physical misalignment between the first imaging sensor and the second imaging sensor based upon the determined misalignment difference.

7. The method of claim 6, further comprising determining the disparity of the object by comparing the first image data and the second image data.

8. The method of claim 6, wherein determining the stereoscopic distance to the object using the first image data and the second image data comprises comparing keypoints of the object located in both the first image data and the second image data.

9. The method of claim 6, further comprising performing autofocus functions to determine and set the autofocus lens positions of the first imaging sensor and the second imaging sensor.

10. The method of claim 6, further comprising warping one of the first image data and the second image data to reduce the disparity of the object if a difference between the stereoscopic distance and the autofocus distance is not zero.

11. A method for correcting for an autofocus lens position of an imaging device having a first imaging sensor and a second imaging sensor using autofocus feedback, comprising:

obtaining a first image data of an object from the first imaging sensor;

obtaining a second image data of the object from the second imaging sensor;

determining a stereoscopic distance to the object based upon a determined disparity of the object within the first image data and the second image data;

determining an autofocus distance to the object based upon autofocus lens positions of the first imaging sensor and the second imaging sensor using a high frequency map;

comparing the stereoscopic distance and the autofocus distance to determine an autofocus correction; and

correcting for the autofocus lens position of the imaging device based upon the determined autofocus correction.

12. The method of claim 11, wherein determining the stereoscopic distance to the object further comprises determining the disparity of the object by comparing the first image data and the second image data.

13. The method of claim 11, wherein determining the autofocus distance comprises performing autofocus functions on the object to determine and set the autofocus lens positions of the first imaging sensor and the second imaging sensor.

14. The method of claim 11, wherein correcting for the autofocus lens position of the imaging device further comprises correcting for a distance estimate of the autofocus lens position of the imaging device if a difference between the autofocus distance and the stereoscopic distance is not zero.

15. An apparatus for correcting for an autofocus lens position of an imaging device and correcting for a physical misalignment between a first imaging sensor and a second imaging sensor using autofocus feedback, comprising:

means for obtaining a first image data of an object from the first imaging sensor;

means for obtaining a second image data of the object from the second imaging sensor;

means for determining a stereoscopic distance to the object using the first image data and the second image data by determining a disparity of the object from the first image data and the second image data;

means for determining an autofocus distance to the object using the autofocus lens position of the imaging device by performing autofocus functions on the object to determine and set autofocus lens positions of the first imaging sensor and the second imaging sensor using a high frequency map;

means for comparing the stereoscopic distance and the autofocus distance to determine an autofocus correction; and

means for correcting for the autofocus lens position of the imaging device based upon the determined autofocus correction if a difference between the autofocus distance and the stereoscopic distance is not zero.

16. The apparatus of claim 15, further comprising means for determining the disparity of the object by comparing the first image data and the second image data.

17. The apparatus of claim 15, further comprising means for performing autofocus functions to determine and set the autofocus lens positions of the first imaging sensor and the second imaging sensor.

18. The apparatus of claim 15, further comprising means for warping one of the first image data and the second image data to reduce the disparity of the object if a difference between the stereoscopic depth and the autofocus depth is not zero.

19. The apparatus of claim 15, further comprising means for correcting for the physical misalignment between the first imaging sensor and the second imaging sensor using a difference between the stereoscopic distance and the autofocus distance.

20. A non-transitory computer-readable medium storing instructions that, when executed, cause at least one physical computer processor to perform a method of digitally correcting for a physical misalignment between a first imaging sensor and a second imaging sensor using autofocus feedback, the method comprising:

obtaining a first image data of an object from the first imaging sensor;

obtaining a second image data of the object from the second imaging sensor;

determining a stereoscopic distance to the object based upon a determined disparity of the object within the first image data and the second image data;

determining an autofocus distance to the object based upon autofocus lens positions of the first imaging sensor and the second imaging sensor;

comparing the stereoscopic distance and the autofocus distance to determine a misalignment difference; and

correcting for the physical misalignment between the first imaging sensor and the second imaging sensor based upon the determined misalignment difference.

21. The non-transitory computer-readable medium of claim 20, further comprising determining the disparity of the object by comparing the first image data and the second image data.

22. The non-transitory computer-readable medium of claim 20, wherein determining the stereoscopic distance comprises comparing keypoints of the object located in both the first image data and the second image data.

23. The non-transitory computer-readable medium of claim 20, further comprising performing autofocus functions to determine and set the autofocus lens positions of the first imaging sensor and the second imaging sensor.

24. The non-transitory computer-readable medium of claim 20, further comprising warping one of the first image data and the second image data to reduce the disparity of the object if a difference between the stereoscopic distance and the autofocus distance is not zero.