METHODS TO COMPENSATE MANUFACTURING VARIATIONS AND DESIGN IMPERFECTIONS IN A CAPSULE CAMERA

• US 20080165248A1
• Filed: 10/29/2007
• Published: 07/10/2008
• Est. Priority Date: 01/09/2007
• Status: Active Grant

• Current Assignee: CapsoVision, Inc.
• Original Assignee: CapsoVision, Inc.
• Inventors: Wilson, Gordon, Wang, Kang-Huai
• Granted Patent: US 8,405,711 B2
• Time in Patent Office: Days
• US Class Current: 348/76
• CPC Class Codes: A61B 1/00057 provided with means for testing or calibration
• CPC Class Codes: A61B 1/00059 provided with identification means for the endoscope
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 25/63 applied to dark current
• CPC Class Codes: H04N 25/671 for non-uniformity detection or correction
• CPC Class Codes: H04N 25/673 by using reference sources

Abstract

A method is provided for characterizing manufacturing variations in a camera and imperfections in its operating environment to allow images captured by the camera to be compensated for these defects. In one embodiment, a method for characterizing a camera includes: (a) illuminating a field of view of the optical elements under a controlled condition; (b) exposing multiple images onto the image sensor under the controlled condition; (c) extracting parameter values of a model for the image provided on the image sensor from the multiple images; and (d) compensating images taken subsequently in the camera using the parameter values. The objects in the field of view may have a predetermined color, contrast or pattern. In one instance, the controlled condition includes an external light source for illuminating the field of view, and the image sensor is sensitive to a plurality of color components. The field of view may be illuminated at a predetermined light intensity for each of the color components. The camera may include an archival memory for storing the images taken. Alternatively, the camera may include a transmitter for sending images taken to an external depository. The camera may also be associated with an identification, and the transmitter sends the images together with the identification to identify the images sent.

Claims

1. A method for characterizing an image sensor, comprising:

Illuminating the image sensor under a controlled condition;

exposing multiple images onto the image sensor under the controlled condition;

extracting from the multiple images parameter values of a model for images exposed onto the image sensor; and

compensating images taken subsequently with the image sensor using the parameter values.

2. A method as in claim 1, further comprises exposing to the image sensor objects having a predetermined color, contrast or pattern.

3. A method as in claim 1, further comprises preparing the controlled condition by providing an external light source for illuminating the image sensor.

4. A method as in claim 3, wherein the image sensor is sensitive to a plurality of color components.

5. A method as in claim 4, wherein the exposing comprises providing the illumination at a predetermined light intensity for each of the color components.

6. A method as in claim 5, wherein objects provide predetermined patterns in the multiple images.

7. A method as in claim 1, further comprising storing the images taken under the controlled condition in an archival memory.

8. A method as in claim 1, further comprising storing parameters calculated in an archival memory.

9. A method as in claim 1, wherein the parameter values are expressed in a form selected from the group consisting of:

an equation, a curve, a piecewise curve and a lookup table.

10. A method as in claim 9, further comprising measuring signals from one or more dark pixels at various temperatures.

11. A method as in claim 9, wherein compensating the images further comprises measuring the temperature at which said images are captured and using temperature measurements as input to the model.

12. A method as in claim 9, wherein compensating the images further comprises measuring one or more dark pixels within a predetermined time before or after an image is captured at a temperature substantially equal to the temperature at which the image was captured and using the measurement of the dark pixels as input to the model.

13. A method as in claim 1, wherein the model of the image comprises a function for compensating pixel-to-pixel variations.

14. A method as in claim 13, wherein the function comprises a product of a multiplicative factor and a measured value of an image.

15. A method as in claim 13, wherein the function compensates an average pixel value across a plurality of pixels.

16. A method as in claim 13, wherein the function compensates for a non-uniform pixel responsivity.

17. A method as in claim 13, wherein the function compensates for an average pixel value.

18. A method as in claim 13, wherein the function is based on expected pixel values obtained from controlled conditions.

19. A method as in claim 1, wherein the model of the image comprises an additive factor for compensating a leakage current in a pixel of the image sensor.

20. A method as in claim 19, wherein the leakage current is integrated over a time of exposure.

21. A method as in claim 19, wherein the additive factor is temperature dependent.

22. A method as in claim 1, wherein the parameter values are calculated for each pixel for compensation for that pixel.

23. A method as in claim 1, wherein the parameter values are calculated based on a group of pixels and wherein the calculated parameter values are then applied to each pixel in the group of pixels for compensation.

24. A method as in claim 23, wherein the group of pixels are selected from pixels in a rectangular area.

25. A method as in claim 1, further comprising compressing the compensated images.

26. A method as in claim 25, wherein further comprises transmitting the compressed images taken to an external depository.

27. A method as in claim 26, wherein the image sensor is associated with an identifier, and wherein the images are transmitted together with the identifier to identify the compressed images sent.

28. A method as in claim 26, wherein the compressed images are stored in an archival memory.

29. A method as in claim 1, further comprising transmitting images taken to an external depository.

30. A method as in claim 29, wherein the image sensor is associated with an identifier, and wherein the images are transmitted together with the identifier to identify the images sent.

31. A method as in claim 1, further comprises preparing the controlled condition by providing an illumination system from within the camera.

32. A method as in claim 31, wherein the model comprises a correction for reflected light.

33. A method as in claim 32, wherein the correction is a function of a current in a light emitting diode.

34. A method as in claim 33, wherein the correction is further a function of a time of exposure.

35. A method as in claim 34, wherein the image sensor is provided in a camera having a plurality of optical elements, the method further comprising characterizing the optical elements independently of the image sensor, and wherein the images are compensated using both the characterization of the optical elements and the parameter values.

70. A camera as in claim 35, wherein the model comprises a multiplicative factor for compensating an average pixel value across a plurality of pixels.

71. A camera as in claim 35, wherein the characterization data is obtained by taking exposure based on an external light source of predetermined characteristics.

36. A camera, comprising:

an image sensor; and

means for preserving a record of characterization data created based on a model for images to be taken with the camera, the characterization data being derived from multiple images projected onto the image sensor.

37. A camera as in claim 36, wherein the means for preserving a record comprises a storage device.

38. A camera as in claim 36, wherein the means for preserving a record comprises a transmitter for transmitting the record for an external recording device.

39. A camera as in claim 36, wherein the camera is associated with an identifier and wherein the record is subsequently retrieved from the external recording device using the identifier.

40. A camera as in claim 36, wherein the model comprises a multiplicative factor for compensating pixel-to-pixel variations.

41. A camera as in claim 36, further comprising means for compressing an image, after compensating the pixel-to-pixel variations in the image.

42. A camera as in claim 36, wherein the projected images are of objects provided in the field of view of a predetermined color, contrast or pattern.

43. A camera as in claim 36, wherein the image sensor is sensitive to a plurality of color components.

44. A camera as in claim 36, wherein the camera further comprises an archival memory for storing the images taken.

45. A camera as in claim 36, further comprising an archival memory for storing parameters calculated.

46. A camera as in claim 36, wherein the model is temperature dependent.

47. A camera as in claim 36, wherein the model of the image comprises a function for compensating pixel-to-pixel variations.

48. A camera as in claim 36, wherein the function comprises a product of a multiplicative factor and a measured value of an image.

49. A camera as in claim 36, wherein the function compensates an average pixel value across a plurality of pixels.

50. A camera as in claim 36, wherein the function compensates for a non-uniform pixel responsivity.

51. A camera as in claim 36, wherein the function compensates for an average pixel value.

52. A camera as in claim 36, wherein the function is based on expected pixel values obtained from controlled conditions.

53. A camera as in claim 36, wherein the model of the image comprises an additive factor for compensating a leakage current in a pixel of the image sensor.

54. A camera as in claim 53, wherein the leakage current is integrated over a time of exposure.

55. A camera as in claim 53, wherein the additive factor is temperature dependent.

56. A camera as in claim 36, wherein the characterization data are calculated for each pixel for compensation for that pixel.

57. A camera as in claim 36, wherein the characterization data are calculated based on a group of pixels and wherein the calculated parameter values are then applied to each pixel in the group of pixels for compensation.

58. A camera as in claim 57, wherein the group of pixels are selected from pixels in a rectangular area.

59. A camera as in claim 36, wherein the camera stores compressed compensated images.

60. A camera as in claim 59, wherein the camera further comprises a transmitter for sending the compressed images taken to an external depository.

61. A camera as in claim 60, wherein the camera is associated with an identifier, and wherein the transmitter sends the images together with the identifier to identify the compressed images sent.

62. A camera as in claim 61, wherein the compressed images are stored in an archival memory.

63. A method as in claim 36, wherein the camera further comprises a transmitter for sending images taken to an external depository.

64. A camera as in claim 63, wherein the camera is associated with an identifier, and wherein the transmitter sends the images together with the identifier to identify the images sent.

65. A camera as in claim 36, further comprising a transparent housing encapsulating the image sensor;

and a light source within the transparent housing for preparing controlled condition for preparing the characterization data.

66. A camera as in claim 65, further comprising means for providing a light absorption environment for preparing the controlled condition.

67. A camera as in claim 65, wherein the model comprises a correction for light reflected from the transparent housing.

68. A camera as in claim 67, wherein the correction is a function of a current in a light emitting diode.

69. A camera as in claim 68, wherein the correction is further a function of a time of exposure.

72. A method for characterizing a camera with a transparent housing and a light source within the transparent housing, comprising:

Placing the camera inside a light absorption environment, and taking a first plurality of exposures;

Placing the camera within a field-dependent reflectance environment, and taking a second plurality of exposures; and

Processing the first and second pluralities of exposures to extract parameter values of a model for images to be taken with the camera.

73. A method as in claim 72, wherein the first plurality of exposures are taken using an illumination system within the camera.

74. A method as in claim 72, wherein the first plurality of exposures are taken for different relative illuminations.

75. A method as in claim 72, wherein the second plurality of exposures are taken for different exposure times.

76. A method as in claim 72, wherein some of the first and second pluralities of exposures are taken in a liquid environment.

77. A method as in claim 72, further comprising compensating images taken by the camera based on the parameter values.

78. A method as in claim 72, wherein the model is based on the first and second pluralities of exposures as sensed in the image sensor.

79. A method as in claim 78, wherein the model comprises a function for compensating pixel-to-pixel variations.

80. A method as in claim 79, wherein the function is temperature dependent.

81. A method as in claim 79, wherein the function comprises a term for compensating a non-uniform pixel responsivity across a plurality of pixels.

82. A method as in claim 79, wherein the function comprises a term for compensating a leakage current in a pixel of the image sensor.

83. A method as in claim 82, wherein the leakage current is integrated over a time of exposure.

84. A method as in claim 79, wherein the function is based on a current in a light emitting diode.

85. A method as in claim 84, wherein the current is a function of a time of exposure.

86. A method as in claim 79, wherein the extracted parameter values are calculated based on a group of pixels and wherein the calculated parameter values are then applied to each pixel in the group of pixels for compensation.

87. A method as in claim 79, wherein the extracted parameter values are calculated for each pixel and wherein the calculated parameter values are applied to that pixel in subsequent exposures.

88. A method as in claim 74, wherein the first and second plurality of exposures are stored in an archival memory in the camera.

89. A method as in claim 74, wherein extracted coefficients or parameters are stored in an archival memory in the camera.

90. A method as in claim 74, wherein the first and second pluralities of exposures are transmitted to an external receiver for storage.

91. A method as in claim 74, wherein extracted parameter values are transmitted to an external receiver for storage.

92. A method as in claim 74, wherein the camera is associated with an identification, and wherein the first and second pluralities of images are transmitted with the identification to allow subsequent retrieval for use in the compensating.

93. A method for efficiently operating a camera having a sensor array organized as a two-dimensional array of picture elements (pixels), comprising:

Providing an image on the sensor array;

Measuring the responses of the pixels along a first dimension of the two-dimensional array to detect a first edge of the image and a second edge of the image;

Measuring the responses of the pixels along a second dimension of the two-dimensional array to detect a third edge of the image and a fourth edge of the image;

recording the area bounded by the first, second, third and fourth edges of the image as an active area of the image sensor; and

processing and storing or transmitting only the responses of the pixels in the active area of the image sensor to subsequent image exposures.

94. A method as in claim 93, wherein the recorded active area is specified by two vertices of a rectangular area.

95. A method as in claim 93, further comprising mapping the centers of the subsequent image exposures to the center of the active area.

96. A method as in claim 95, wherein the mapping is achieved using a displacement vector representing a displacement joining the center of the sensor array to the center of the active area.

97. A method as in claim 93, further comprising scaling subsequent image exposures by a scaling factor representing a ratio between an expected size of the active area and the actual size of the active area.

98. A method for correcting a distortion in an optical element of a camera, comprising:

Taking an image of a known pattern including a plurality of edges with the camera;

Determining distortions at predetermined locations on the image relative to the known pattern by identifying the edges of the known pattern and detecting corresponding edges in the image;

Taking an image of a subject with the camera; and

using the determined distortions at the predetermined locations, undoing the determined distortions from the image of the subject.

99. A method as in claim 98, further comprising:

Sending the image of the subject after it is taken to a workstation with a display; and

Correcting the image of the subject by undoing of the determined distortions on the workstation and displaying the corrected image of the subject on the display.

100. A method for determining a boundary of an image based on frequency-domain information of the image, comprising:

(a) selecting an area of the image M pixels by M+n pixels into a plurality of n×

n blocks, wherein M is divisible by n;

(b) computing a discrete cosine transform over each of the blocks;

(c) identifying, along the direction of the M+n pixels, one of the n×

n blocks where the n² DCT values increase over a previous block;

(d) reforming the n×

n blocks of the image beginning at the identified block by including into the identified block the last column of the previous block; and

(e) repeating steps (b), (c) and (d) until the block with maximum DCT values is found.