Image sensing apparatus and method capable of merging function for obtaining high-precision image by synthesizing images and image stabilization function

• US 6,429,895 B1
• Filed: 12/22/1997
• Issued: 08/06/2002
• Est. Priority Date: 12/27/1996
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Onuki, Ichiro
• Primary Examiner(s): HO, TUAN V
• Application Number:
  US08/996,287
• Time in Patent Office: 1,688 Days
• Field of Search: 348/207, 348/218, 348/219, 348/200, 348/222, 348/208, 348/358, 348/335, 382/284, 382/299, 382/312, 396/54, 396/55
• US Class Current: 348/208.99
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/663 for controlling interchangeable camera parts based on electronic image sensor signals
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 25/48 Increasing resolution by shifting the sensor relative to the scene

Abstract

An image sensing apparatus, which performs pixel shifting operation for shifting an image formation position of an image on an image sensing device using a lens capable shifting a light path and senses images at respective image formation positions, has a vibration sensor for detecting vibration. The lens is moved so as to cancel the vibration detected by the vibration sensor when performing pixel shifting operation. Since the shifting characteristics of the lens slightly changes depending upon states of zooming and focusing lenses, the lens is moved by a shift amount which is adjusted in accordance with the states of the zooming and focusing lens.

Claims

1. An image sensing apparatus comprising:

image sensing means for converting an optical image into electric signals and outputting the electric signals as image signals;

shifting means for shifting an image formation position of the optical image formed on said image sensing means to a plurality of different positions;

first vibration detection means for detecting vibration of the image sensing apparatus and outputting vibration information;

optical system state detection means for detecting a state of at least one of a zooming optical system and a focusing optical system,

control means for controlling said shifting means on the basis of the vibration information outputted by said first vibration detection means, said control means comprises;

pixel shifting control data generation means for generating pixel shifting control data for controlling said shifting means in pixel shifting operation;

vibration compensation data generation means for generating vibration compensation data for compensating the vibration on the basis of the vibration information outputted by said first vibration detection means;

control data synthesis means for synthesizing the pixel shifting control data and the vibration compensation data to generate a control signal for controlling said shifting means;

data correction means for correcting at least one of the pixel shifting control data and the vibration compensation data in accordance with the state detected by said optical state detection means;

image signal synthesis means for synthesizing image signals of a plurality of images outputted by said image sensing means to generate a single image; and

storage means for storing first correction data for correcting the pixel shifting control data and second correction data for correcting the vibration compensation data,

wherein said image sensing means converts the optical image into electric signals at each of the plurality of different image formation positions shifted by said shifting means and said data correction means reads at least one of the first and second correction data, corresponding to the state detected by said optical state detection means, stored in said storage means, and calculates the pixel shifting control data or the vibration compensation data on the basis of at least one of the read first and second data.

2. The image sensing apparatus according to claim 1, wherein said shifting means is a movable optical means.

3. The image sensing apparatus according to claim 1, wherein said shifting means is a plane parallel plate.

4. The image sensing apparatus according to claim 1, wherein said shifting means is a variable apical angle prism.

5. The image sensing apparatus according to claim 1, wherein said control means comprises change-over means for changing between a plurality of pixel shifting modes, and pixel shifting control data generation means for generating pixel shifting control data for controlling said shifting means in pixel shifting operation,

wherein said change-over means changes the pixel shifting modes on the basis of the vibration information outputted by said first vibration detection means.

6. The image sensing apparatus according to claim 5, wherein the pixel shifting modes indicate number of the image formation positions to be shifted in pixel shifting operation, and said change-over means selects a first number of image formation positions when a degree of vibration indicated by the vibration information is equal or less than a first predetermined value;

and selects a second number of image formation positions which is smaller than the first number when the degree of vibration is larger than the first predetermined value.

7. The image sensing apparatus according to claim 6, wherein said change-over means selects a third number of image formation positions which is smaller than the second number when the degree of vibration is larger than the second predetermined value which is larger than the first predetermined value.

8. The image sensing apparatus according to claim 5, further comprising optical system state detection means for detecting a state of at least one of a zooming optical system and a focusing optical system,

wherein said control means further comprises data correction means for correcting the pixel shifting control data in accordance with the state detected by said optical state detection means.

9. The image sensing apparatus according to claim 8 further comprising storage means for storing correction data for correcting the pixel shifting control data,

wherein said data correction means reads the correction data, corresponding to the state detected by said optical state detection means, stored in said storage means, and corrects the pixel shifting control data on the basis of the read correction data.

10. The image sensing apparatus according to claim 1, wherein said control means comprises change-over means for changing between a plurality of pixel shifting modes, and pixel shifting control data generation means for generating pixel shifting control data for controlling said shifting means in pixel shifting operation,

wherein said change-over means changes between the pixel shifting modes in accordance with a user designation.

11. The image sensing apparatus according to claim 10 further comprising optical system state detection means for detecting a state of at least one of a zooming optical system and a focusing optical system,

wherein said control means further comprises data correction means for correcting the pixel shifting control data in accordance with the state detected by said optical state detection means.

12. The image sensing apparatus according to claim 11 further comprising storage means for storing correction data for correcting the pixel shifting control data,

wherein said data correction means reads the correction data, corresponding to the state detected by said optical state detection means, stored in said storage means, and corrects the pixel shifting control data on the basis of the read correction data.

13. The image sensing apparatus according to claim 1 further comprising second vibration detection means for detecting blurring in an image on the basis of the image signals outputted by said image sensing means and outputting blurring information,

wherein said control means controls said shifting means on the basis of the vibration information outputted by said first vibration detection means and the blurring information outputted by said second vibration detection means.

14. The image sensing apparatus according to claim 13, wherein said first vibration detection means is a vibration-type gyro and said second vibration detection means is movement vector detection means.

15. The image sensing apparatus according to claim 1 further comprising photometry means and luminous exposure adjustment means for determining a luminous exposure on the basis of a result of the photometry performed by said photometry means and controlling said image sensing means to use the determined luminous exposure.

16. The image sensing apparatus according to claim 15, wherein, when the image formation position is shifted by said shifting means, said luminous exposure adjustment means determines a different luminous exposure and controls said image sensing means to use the different luminous exposure.

17. The image sensing apparatus according to claim 16, wherein said photometry means performs photometry on each of a plurality of divided areas of an image sensed by said image sensing means.

18. The image sensing apparatus according to claim 17, further comprising main object determination means for determining a divided area which includes a main object among the plurality of divided areas,

wherein said luminous exposure adjustment means controls said image sensing means to use a first luminous exposure which is suitable for the divided area including the main object, which is determined by said main object determination means, and to use at least one of second and third luminous exposures where the second luminous exposure is larger than the first luminous exposure and the third exposure is smaller than the first luminous exposure.

19. The image sensing apparatus according to claim 18, wherein said control means comprises change-over means for changing between a plurality of pixel shifting modes, and pixel shifting control data generation means for generating pixel shifting control data for controlling said shifting means in pixel shifting operation,

wherein said change-over means changes the pixel shifting modes on the basis of the vibration information outputted by said first vibration detection means, and said luminous exposure adjustment means determines the luminous exposure to be used in said image sensing device on the basis of the pixel shifting mode selected by said change-over means.

20. The image sensing apparatus according to claim 19, wherein the pixel shifting modes indicate number of the image formation positions to be shifted in pixel shifting operation, and said change-over means selects a first number of image formation positions when a degree of vibration indicated by the vibration information is equal or less than a first predetermined value;

selects a second number of image formation positions which is smaller than the first number when the degree of vibration is larger than the first predetermined value and equal or less than a second predetermined value which is larger than the first predetermined value; and

selects a third number of image formation positions which is smaller than the second number when the degree of vibration is larger than the second predetermined value.

21. The image sensing apparatus according to claim 18, wherein said control means comprises change-over means for changing between a plurality of pixel shifting modes, and pixel shifting control data generation means for generating pixel shifting control data for controlling said shifting means in pixel shifting operation,

wherein said change-over means changes between the pixel shifting modes in accordance with a user designation and said luminous exposure adjustment means determines the luminous exposure to be used in said image sensing device on the basis of the pixel shifting mode selected by said change-over means.

22. The image sensing apparatus according to claim 15, wherein said luminous exposure adjustment means adjusts the luminous exposure by changing exposure time.

23. The image sensing apparatus according to claim 15, wherein said luminous exposure adjustment means adjusts the luminous exposure by changing aperture of an iris diaphragm.

24. The image sensing apparatus according to claim 15, wherein said luminous exposure adjustment means adjusts the luminous exposure using an electrochromic element.

25. The image sensing apparatus according to claim 1 further comprising blurring detection means for detecting relative blurring amount between two images out of a plurality of images obtained at the plurality of image formation positions shifted by said shifting means by comparing image signals of the two images, and outputting the blurring amount.

26. The image sensing apparatus according to claim 25 wherein said blurring detection means detects the relative blurring amount by comparing the image signals of the two images which are obtained in series out of the plurality of images obtained at the plurality of image formation positions.

27. The image sensing apparatus according to claim 25, wherein said blurring detection means detects the relative blurring amount by comparing the image signals of the two images which are obtained at the same image formation position in series out of the plurality of images obtained at the plurality of image formation positions.

28. The image sensing apparatus according to claim 25 further comprising disabling means for disabling the image signal synthesis means in a case where the blurring amount detected by said blurring detection means is greater than a predetermined value.

29. The image sensing apparatus according to claim 28 further comprising notification means for notifying a user of image synthesis being disabled in a case where said disabling means disabled said image signal synthesis means.

30. The image sensing apparatus according to claim 28, wherein said image signal synthesis means has a plurality of image synthesis modes, and selects one of the plurality of image synthesis modes in accordance with the blurring amount.

31. The image sensing apparatus according to claim 30, wherein the image synthesis modes indicate numbers of images to be synthesized out of the plurality of images sensed by said image sensing means, and said image signal synthesis means selects a first number of images when the blurring amount is equal or less than a first predetermined value;

and selects a second number of images which is smaller than the first number when the blurring amount is larger than the first predetermined value.

32. The image sensing apparatus according to claim 31, wherein said image signal synthesis means selects a third number of images which is smaller than the second number when the blurring amount is larger than the second predetermined value which is larger than the first predetermined value.

33. The image sensing apparatus according to claim 31 further comprising notification means for notifying a user of the image sensing mode selected by said image signal synthesis means.

34. The image sensing apparatus according to claim 25, wherein said image signal synthesis means synthesizes the image signals of the plurality of images on the basis of the blurring amount detected by said blurring detection means so that blurring is compensated.

35. The image sensing apparatus according to claim 34, wherein said image signal synthesis means synthesizes the plurality of images after changing relative positions between the plurality of images on the basis of the blurring amounts between respective pairs of the plurality of images.

36. An image sensing apparatus comprising:

an image sensing device for converting an optical image into electronic signals and outputting the electric signals as image signals;

a shifting unit for shifting an image formation position of the optical image formed on said image sensing device to a plurality of different positions;

an optical system state detection unit for detecting a state of a focusing optical system;

a control unit for controlling said shifting unit on the basis of the state of the focusing optical system detected by said optical system state detection unit; and

an image signal synthesis unit for synthesizing image signals of a plurality of images outputted by said image sensing device to generate a single image;

wherein said image sensing device converts the optical image into electric signals at each of the plurality of different image formation positions shifted by said shifting unit.

37. The apparatus according to claim 36, wherein said focusing optical system is a rear focusing system.

38. The apparatus according to claim 36, wherein said control unit controls a shift amount of an image formation position by said shifting unit.

39. The apparatus according to claim 36, wherein said shifting unit moves in a substantially vertical direction with respect to an optical axis.

40. The apparatus according to claim 36, wherein said shifting unit is a plane parallel plate.

41. The apparatus according to claim 36, wherein said shifting unit is a variable apical angle prism.

42. An image sensing method comprising:

an optical system state detection step of detecting a state of a focusing optical system;

an image sensing step of, while shifting an image formation position of an optical image on an image sensing device to a plurality of different positions by a shifting unit on the basis of the state of the focusing optical system detected in said optical system state detection step, converting the optical image into electronic signals and outputting the electric signals as image signals at each image formation position; and

an image signal synthesis step of synthesizing image signals of a plurality of images outputted by said image sensing device to generate a single image.

43. The method according to claim 42, wherein said focusing optical system is a rear focusing system.

44. The method according to claim 42, wherein a shift amount of the shifting unit is controlled upon shifting the image formation position.

45. The method according to claim 42, wherein the shifting unit moves in a substantially vertical direction with respect to an optical axis.

46. The method according to claim 42, wherein the shifting unit is a plane parallel plate.

47. The method according to claim 42, wherein the shifting unit is a variable apical angle prism.