Method and apparatus for a computer vision camera unit

• US 10,645,277 B1
• Filed: 04/23/2018
• Issued: 05/05/2020
• Est. Priority Date: 03/17/2016
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Herman Herman
• Inventors: Herman, Herman
• Primary Examiner(s): Perkey, William B
• Application Number:
  US15/960,540
• Time in Patent Office: 743 Days
• Field of Search: None
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/617 Upgrading or updating of programs or applications for camera control

Abstract

A computer vision camera unit comprising first and second pairs, of image capture assemblies, both of which have an approximately same orientation to a first axis. The first and second pairs are optically focused to capture first and second fields of view that, at least, overlap each other. Providing the first and second pairs with imager chips of differing light sensitivity permits high dynamic range to be realized. Having the second field of view within the first field of view permits specialization, between the first and second pairs, similar to the roles of peripheral and foveal vision, found in biological stereo vision systems. The image capture assemblies can be organized in a rectangular configuration that implies third and fourth pairs, orthogonal to the first and second pairs. The orthogonal but interlocking nature, of the first and second pairs in relation to the third and fourth pairs, enables an autocalibration technique.

Claims

1. A method for a first computer vision camera unit, comprising:

identifying a same first feature, of unknown depth and spatial location, with each of a first, second, third, and fourth image capture assemblies, each image capture assembly having a lens assembly embedded in a first frame, wherein each embedding locates a lens assembly at a different vertex of a rectangular configuration of a first type;

identifying the first feature at a first, second, third, and fourth location of, respectively, a first, second, third, and fourth imager chip, wherein each of the first, second, third, and fourth imager chips is part of, respectively, the first, second, third, and fourth image capture assemblies;

determining, performed at least in part with a configuration of computing hardware and programmable memory, if, with respect to a first axis, there is a first misalignment between the first and second locations;

determining, performed at least in part with a configuration of computing hardware and programmable memory, if, with respect to the first axis, there is second misalignment between the third and fourth locations;

determining, performed at least in part with a configuration of computing hardware and programmable memory, if, with respect to a second axis, there is third misalignment between the first and third locations;

determining, performed at least in part with a configuration of computing hardware and programmable memory, if, with respect to the second axis, there is a fourth misalignment between the second and fourth locations; and

generating, performed at least in part with a configuration of computing hardware and programmable memory, a loss of calibration signal, if at least one of the first, second, third, or fourth misalignments is detected.

2. The method of claim 1, further comprising:

randomly shifting a focal point, of one or more of the embedded lens assemblies, relative to its corresponding imager chip.

3. The method of claim 1, further comprising:

maintaining accurate and consistent placing of the image capture assemblies with a front plate of the first frame that is machined from a uniform block of material.

4. The method of claim 1, further comprising:

mounting each of the first, second, third, and fourth imager chips with sub-micron accuracy on a same printed circuit board.

5. The method of claim 1, further comprising:

generating, performed at least in part with a configuration of computing hardware and programmable memory, a loss of calibration signal when no misalignment is detected, but the first, second, third, and fourth locations define a rectangular configuration that is not of the first type.

6. The method of claim 1, further comprising:

generating, in response to the loss of calibration signal, a first rectangular configuration, that has no misalignments, is of the first type, and is comprised of fifth, sixth, seventh, and eighth locations;

determining a first score, according to a first metric, by comparing the first, second, third, and fourth locations to, respectively, the fifth, sixth, seventh, and eighth locations;

repeating the steps of generating rectangular configurations and determining their scores, in accordance with a first strategy for exploring a first space of potential solutions;

selecting, from the first space, a rectangular configuration that, according to a first metric, appears to be at least a good fit to the first, second, third, and fourth locations.

7. The method of claim 6, wherein the first metric is least absolute distance.

8. The method of claim 6, wherein the first strategy is an exhaustive exploration the first space.

9. The method of claim 8, wherein the exploration is limited by a first error window, applied to each of the first, second, third, and fourth locations.

10. The method of claim 1, further comprising:

determining there is a misalignment between the first and second locations;

determining there is a misalignment between the second and fourth locations;

determining there is no misalignment between the third and fourth locations;

determining there is no misalignment between the first and third locations;

identifying a fifth location, completing a first rectangular configuration defined by the first, third, and fourth locations;

determining the first rectangular configuration is of the first type;

selecting the first rectangular configuration as the best fit to the first, second, third, and fourth locations.

11. The method of claim 6, further comprising:

determining a first average disparity, from a first disparity between the first and second locations, and a second disparity between the third and fourth locations;

generating the first rectangular configuration with a first dimension equal to the first average disparity.

12. The method of claim 6, further comprising:

determining a first average location, from the first and third locations; and

locating a first side of the first rectangular configuration at the first average location.

13. The method of claim 1, further comprising:

identifying, with each of the first, second, third, and fourth image capture assemblies, a same calibration target of a known depth and spatial location, relative to the first computer vision camera unit;

identifying the calibration target at a first, second, third, and fourth calibration imaging element of, respectively, the first, second, third, and fourth imager chips;

identifying the first, second, third, and fourth calibration imaging elements relative to, respectively, a first, second, third, and fourth origin imaging element; and

adjusting at least one of the first, second, third, and fourth origin imaging elements until the first, second, third, and fourth calibration imaging elements are each at a different vertex of a rectangular configuration of the first type.

14. The method of claim 13, further comprising:

illuminating each of the first, second, third, and fourth image capture assemblies with substantially parallel light rays; and

finding a first, second, third, and fourth focal-point imaging element, for each of, respectively, the first, second, third, and fourth imager chips, as, respectively, the first, second, third, and fourth origin imaging element.

15. The method of claim 4, further comprising:

providing high speed connections, with the same printed circuit board, between each imager chip and hardware, also located on the printed circuit board, for processing, with an auto-alignment algorithm, images produced by the imager chips.