Method and apparatus of lens alignment for capsule

• US 10,638,920 B2
• Filed: 06/01/2017
• Issued: 05/05/2020
• Est. Priority Date: 06/13/2016
• Status: Active Grant

• Current Assignee: CapsoVision, Inc.
• Original Assignee: CapsoVision, Inc.
• Inventors: Lu, Ganyu, Wilson, Gordon C., Wang, Kang-Huai
• Primary Examiner(s): Hance, Robert J
• Application Number:
  US16/092,301
• Publication Number:
  US 20190150717A1
• Time in Patent Office: 1,069 Days
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: A61B 5/0031 Implanted circuitry
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 13/16 for use in conjunction with image converters or intensifiers , or for use with projectors, e.g. objectives for projection TV
• CPC Class Codes: G03B 2215/0503 Built-in units
• CPC Class Codes: G03B 37/005 Photographing internal surfaces, e.g. of pipe, G03B37/02 - G03B37/06 take precedence; instruments for performing medical examinations of cavities or tubes of the body by photographic inspection A61B1/00
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H04N 23/40 Circuit details for pick-up tubes
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors

Abstract

A method and apparatus of aligning a lens module with respect to an image sensor device for a capsule camera are disclosed. The image sensor device comprises multiple pixel arrays and the lens module comprises multiple lens sets to form multiple images corresponding to multiple fields of view associated with the multiple lens sets, and each lens set forms one image for one corresponding pixel array associated with one field of view. A method according to the present invention present invention, one or more test images are presented in the multiple fields of view associated with the lens module. Multiple images in the multiple fields of view are captured using the multiple pixel arrays. Metric measurement is derived based on the multiple images captured by the multiple pixel arrays. Lens alignment between the lens module and the image sensor device is then adjusted based on the metric measurement.

Claims

1. A method of aligning a lens module with respect to an image sensor device for a capsule camera, wherein the image sensor device comprises multiple pixel arrays and the lens module comprises multiple lens sets to form multiple images corresponding to multiple fields of view associated with the multiple lens sets, and each lens set forms one image for one corresponding pixel array associated with one field of view, the method comprising:

presenting one or more test images in the multiple fields of view associated with the multiple lens sets;

capturing multiple images in the multiple fields of view using the multiple pixel arrays;

deriving metric measurement comprising back focus distances (BFDs) corresponding to substantially parallel optical paths associated with the multiple pixel arrays based on the multiple images captured by the multiple pixel arrays, wherein each back focus distance (BFD) is related to a distance from one image plane of one pixel array to a last lens surface of one corresponding lens set; and

adjusting lens alignment between the lens module and the image sensor device based on the metric measurement of the BFDs for the multiple lens sets; and

wherein the lens module consists of four lens sets arranged laterally to provide a combined fields-of-view of 360 degrees and the image sensor device consists of four pixel arrays, four BFDs designated as Z1, Z2, Z3 and Z4 associated with four neighboring lens sets are determined, and wherein each of pixel array and one corresponding lens set is used for capturing images in one quadrant; and

wherein rotational alignments about x-axis and y-axis are adjusted to minimize (Z2−

Z4)/2d and (Z1−

Z3)/2d respectively, and wherein d is a lateral distance between two corresponding pixel arrays on two opposite sides and the x-axis and the y-axis are in a plane parallel to the image sensor device.

2. The method of claim 1, wherein the lens alignment between the lens module and the image sensor device is performed in multiple degrees of freedom and second lens alignment in second one or more degrees of freedom is performed after first lens alignment in first one or more degrees of freedom.

3. The method of claim 1, wherein said adjusting the lens alignment between the lens module and the image sensor device is performed in the first one or more degrees of freedom and the second lens alignment iteratively until a criterion is met.

4. The method of claim 3, wherein the criterion corresponds to no further improving image quality by said adjusting the lens alignment.

5. The method of claim 1, wherein said one or more test images are selected from a group comprising “

cross hairs”

, “

slant edges”

, “

circles”

, “

dots”

, “

grids”

, “

checker board”

, “

grating lines”

, and “

sinusoidal waves”

.

6. The method of claim 1, the metric measurement corresponds to modulation transfer function (MTF), point spread function (PSF), spot size of a pin hole, contrast or maximum achievable spatial resolution.

7. The method of claim 1 further comprising fixing adjustable positions of the lens module after adjusting the lens alignment is completed.

8. The method of claim 7, wherein the adjustable positions of the lens module are fixed using UV-cured adhesive.

9. A method of aligning a lens module with respect to an image sensor device for a capsule camera, wherein the image sensor device comprises multiple pixel arrays and the lens module comprises multiple lens sets to form multiple images corresponding to multiple fields of view associated with the multiple lens sets, and each lens set forms one image for one corresponding pixel array associated with one field of view, the method comprising:

presenting one or more test images in the multiple fields of view associated with the multiple lens sets;

capturing multiple images in the multiple fields of view using the multiple pixel arrays;

deriving metric measurement comprising back focus distances (BFDs) corresponding to substantially parallel optical paths associated with the multiple pixel arrays based on the multiple images captured by the multiple pixel arrays, wherein each back focus distance (BFD) is related to a distance from one image plane of one pixel array to a last lens surface of one corresponding lens set; and

adjusting lens alignment between the lens module and the image sensor device based on the metric measurement of the BFDs for the multiple lens sets; and

wherein the lens module consists of four lens sets arranged laterally to provide a combined fields-of-view of 360 degrees and the image sensor device consists of four pixel arrays, four BFDs designated as Z1, Z2, Z3 and Z4 associated with four neighboring lens sets are determined, and wherein each of pixel array and one corresponding lens set is used for capturing images in one quadrant; and

wherein the lens alignment is adjusted to minimize alignment error for all quadrants of the lens module, wherein the alignment error is determined according to |Z1−

Z2+Z3-Z4|/4.

10. A method of aligning a lens module with respect to an image sensor device for a capsule camera, wherein the image sensor device comprises multiple pixel arrays and the lens module comprises multiple lens sets to form multiple images corresponding to multiple fields of view associated with the multiple lens sets, and each lens set forms one image for one corresponding pixel array associated with one field of view, the method comprising:

presenting one or more test images in the multiple fields of view associated with the multiple lens sets;

capturing multiple images in the multiple fields of view using the multiple pixel arrays;

deriving metric measurement comprising back focus distances (BFDs) corresponding to substantially parallel optical paths associated with the multiple pixel arrays based on the multiple images captured by the multiple pixel arrays, wherein each back focus distance (BFD) is related to a distance from one image plane of one pixel array to a last lens surface of one corresponding lens set; and

adjusting lens alignment between the lens module and the image sensor device based on the metric measurement, wherein said adjusting the lens alignment comprising adjusting translational offset (Tx, Ty) between an optical center of the multiple lens set and a center of the pixel arrays to minimize the translational offset; and

wherein the translational offset is determined by using a white cylindrical target as a test image to find a center of each quadrant of the lens module with four lens sets and the center of each quadrant of the lens module is determined from a center of contours of an image of the white cylindrical target.

11. The method of claim 10, wherein the image of the white cylindrical target is filtered prior to finding the center of each quadrant of the lens module.

12. The method of claim 10, wherein said adjusting the lens alignment comprising adjusting rotation value in z-direction, Rz to minimize rotational error, and wherein the z-direction is perpendicular to a plane of the pixel arrays.

13. The method of claim 12, wherein the rotation value is determined contours of the image of the white cylindrical target.

14. The method of claim 12, wherein least square error, a min-max criterion or minimum of largest error in (Tx, Ty) or Rz associated with each quadrant of the lens module is used as a performance criterion.

15. The method of claim 10, wherein a target shape, pattern or color is used as a test image to find a center of each quadrant of the lens module with four lens sets and light associated with test image enters a pupil over a field of view of one lens set so that at least a portion of image footprint of the test image is visible on a corresponding pixel array associated with one field of view.

16. An apparatus for aligning a lens module with respect to an image sensor device for a capsule camera, wherein the image sensor device comprises multiple pixel arrays and the lens module comprises multiple lens sets to form multiple images corresponding to multiple fields of view associated with the multiple lens sets, and each lens set forms one image for one corresponding pixel array associated with one field of view, the apparatus comprising one or more electronic circuits, controllers or processors arranged to:

present one or more test images in the multiple fields of view associated with the multiple lens sets;

capture multiple images in the multiple fields of view using the multiple pixel arrays;

derive metric measurement comprising back focus distances (BFDs) corresponding to substantially parallel optical paths associated with the multiple pixel arrays based on the multiple images captured by the multiple pixel arrays, wherein each back focus distance (BFD) is related to a distance from one image plane of one pixel array to a last lens surface of one corresponding lens set; and

adjust lens alignment between the lens module and the image sensor device based on the metric measurement of the BFDs for the multiple lens sets; and

wherein the lens module consists of four lens sets arranged laterally to provide a combined fields-of-view of 360 degrees and the image sensor device consists of four pixel arrays, four BFDs designated as Z1, Z2, Z3 and Z4 associated with four neighboring lens sets are determined, and wherein each of pixel array and one corresponding lens set is used for capturing images in one quadrant; and

wherein rotational alignments about x-axis and y-axis are adjusted to minimize (Z2−

Z4)/2d and (Z1−

Z3)/2d respectively, and wherein d is a lateral distance between two corresponding pixel arrays on two opposite sides and the x-axis and the y-axis are in a plane parallel to the image sensor device.