Image processing device, image projection apparatus, image processing method, recording medium, and computer data signal

• US 8,111,949 B2
• Filed: 02/25/2005
• Issued: 02/07/2012
• Est. Priority Date: 02/27/2004
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Sakurai, Keiichi
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US11/067,087
• Publication Number:
  US 20050190986A1
• Time in Patent Office: 2,538 Days
• Field of Search: 382/255, 382/254, 382/299, 382/307, 382/318, 382/319, 382/168, 348/218.1, 348/672
• US Class Current: 382/299
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: H04N 1/00283 with a television apparatus
• CPC Class Codes: H04N 1/195 the array comprising a two-dimensional array or a combination of two-dimensional arrays
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/40068 Modification of image resolution, i.e. determining the values of picture elements at new relative positions H04N1/3935 takes precedence
• CPC Class Codes: H04N 2201/0436 Scanning a picture-bearing surface lying face up on a support H04N2201/0434 takes precedence; using cylindrical picture bearing surfaces H04N1/06; with manual scanning H04N1/107

Abstract

A captured image projection apparatus obtains projection parameters and image effect correction parameters from a moving image captured at a low resolution in a task different from a task in which capturing of a high-resolution still image is to be performed, when a change ceases to occur in the moving image and before the capturing of the high-resolution still image is started. The apparatus changes the scale of the obtained projection parameters from one for low-resolution image to one for high-resolution image. The apparatus applies an image conversion process and an image effect process to the captured high-resolution still image by using the projection parameters and image effect correction parameters obtained before the still image is captured. Thereby, the captured image projection apparatus shortens the time required from the capturing of the image to projection thereof.

Claims

1. An image processing device comprising:

a camera which is operable at a low resolution, and at a high resolution;

an initializing unit which sets the camera to initially operate at the low resolution to capture a plurality of images of an object at the low resolution;

a change determination section which determines whether there is any change in the images captured by the camera at the low resolution;

a resolution setting section which maintains the camera operating at the low resolution when the change determination section determines that there is a change in the images captured by the camera at the low resolution, and which sets the camera to operate at the high resolution to capture an image of the object at the high resolution when the change determination section determines that there is no change in the images captured by the camera at the low resolution;

a correction parameter extraction section which extracts parameters for performing image correction of the image captured by the camera at the high resolution, from a last-captured image captured by the camera at the low resolution just before the resolution setting section sets the camera to operate at the high resolution;

a correction parameter conversion section which converts a low scale of the parameters extracted from the last-captured image captured by the camera at the low resolution into a high scale corresponding to the high resolution; and

an image correction section which corrects a luminance and color of the image captured by the camera at the high resolution by using the parameters with the high scale converted by the correction parameter conversion section;

wherein the correction parameter extraction section includes;

a luminance histogram generation section which generates a luminance histogram showing distribution of luminance in the last-captured image captured by the camera at the low resolution, by counting a number of pixels included in the last-captured image for each luminance value,

a color difference histogram generation section which generates a color difference histogram showing distribution of color differences in the last-captured image captured by the camera at the low resolution, by counting a number of pixels included in the last-captured image for each color difference, and

an image effect correction parameter obtaining section which obtains a maximum value, a minimum value, and a peak value of luminance from the luminance histogram generated by the luminance histogram generation section, and a peak value and an average value of color difference from the color difference histogram generated by the color difference histogram generation section, thereby obtaining image effect correction parameters including the maximum value, the minimum value, and the peak value of luminance, and the peak value and the average value of color difference, and

wherein the image correction section includes a luminance/color difference correction section which corrects luminance and color differences of the image captured by the camera at the high resolution, by adjusting luminance and color differences of each pixel included in the image by using the image effect correction parameters obtained by the image effect correction parameter obtaining section.

2. The image processing device according to claim 1, wherein the correction parameter extraction section further includes:

a polygon detection section which detects from the last-captured image captured by the camera at the low resolution, a polygon shaping a contour of the image, and

an affine parameter obtaining section which obtains affine parameters from coordinates of vertexes of the polygon detected by the polygon detection section, and

wherein the image correction section further includes an inclination correction section which corrects an inclination of the image captured by the camera at the high resolution by performing affine transformation of coordinates of each pixel included in the image by using the affine parameters obtained by the affine parameter obtaining section.

3. The image processing device according to claim 2, wherein the correction parameter conversion section changes a scale of the affine parameters obtained by the affine parameter obtaining section from a scale for the last-captured image captured at the low resolution to a scale for the image captured at the high resolution, by multiplying each element of the affine parameters by a predetermined multiple.

4. The image processing device according to claim 1, wherein the correction parameter extraction section performs extraction of the parameters in a different task from a task in which change determination by the change determination section, resolution setting by the resolution setting section, and image correction by the image correction section are performed.

5. The image processing device according to claim 1, wherein the change determination section includes an image change amount calculation section which calculates an amount of change in the images captured by the camera at the low resolution, and

wherein the change determination section determines that there is a change in the images when the amount of change calculated by the image change amount calculation section is greater than or equal to a predetermined threshold, and determines that there is no change in the images when the calculated amount of change calculated by the image change amount calculation section is smaller than the predetermined threshold.

6. The image processing device according to claim 5, wherein the image change amount calculation section calculates a total of absolute values of differences between pixel values of respective pixels included in a previously captured image and pixel values of respective pixels included in a currently captured image, as the amount of change in the images captured by the camera at the low resolution.

7. An image projection apparatus comprising:

an image sensor which is operable at a low resolution, and at a high resolution;

an initializing section which sets the image sensor to initially operate at the low resolution to capture a plurality of images of an object at the low resolution;

a change determination section which determines whether there is any change in the images captured by the image sensor at the low resolution;

a resolution setting section which maintains the image sensor operating at the low resolution when the change determination section determines that there is a change in the images captured by the image sensor at the low resolution, and which sets the image sensor to operate at the high resolution to capture an image of the object at the high resolution when the change determination section determines that there is no change in the images captured by the image sensor at the low resolution;

a correction parameter extraction section which extracts parameters for performing image correction of the image captured by the image sensor at the high resolution, from a last-captured image captured by the image sensor at the low resolution just before the resolution setting section sets the image sensor to operate at the high resolution;

a correction parameter conversion section which converts a low scale of the parameters extracted from the last-captured image captured by the image sensor at the low resolution into a high scale corresponding to the high resolution;

an image correction section which corrects a luminance and color of the image captured by the image sensor at the high resolution by using the parameters with the high scale converted by the correction parameter conversion section; and

a projection section which projects the image corrected by the image correction section on a screen,

wherein the correction parameter extraction section includes;

a luminance histogram generation section which generates a luminance histogram showing distribution of luminance in the last-captured image captured by the image sensor at the low resolution, by counting a number of pixels included in the last-captured image for each luminance value,

a color difference histogram generation section which generates a color difference histogram showing distribution of color differences in the last-captured image captured by the image sensor at the low resolution, by counting a number of pixels included in the last-captured image for each color difference, and

an image effect correction parameter obtaining section which obtains a maximum value, a minimum value, and a peak value of luminance from the luminance histogram generated by the luminance histogram generation section, and a peak value and an average value of color difference from the color difference histogram generated by the color difference histogram generation section, thereby obtaining image effect correction parameters including the maximum value, the minimum value, and the peak value of luminance, and the peak value and the average value of color difference, and

wherein the image correction section includes a luminance/color difference correction section which corrects luminance and color differences of the image captured by the image sensor at the high resolution, by adjusting luminance and color differences of each pixel included in the image by using the image effect correction parameters obtained by the image effect correction parameter obtaining section.

8. An image processing method comprising:

initially operating a camera to capture a plurality of images of an object at a low resolution;

determining whether there is any change in the images captured by the camera at the low resolution;

maintaining the camera to operate at the low resolution when it is determined that there is a change in the images captured by the camera at the low resolution, and setting the camera to operate at a high resolution to capture an image of the object at the high resolution when it is determined that there is no change in the images captured by the camera at the low resolution;

extracting parameters for performing image correction of the image captured by the camera at the high resolution from a last-captured image captured by the camera at the low resolution just before the camera is set to operate at the high resolution;

converting a low scale of the parameters extracted from the last-captured image captured by the camera at the low resolution into a high scale corresponding to the high resolution; and

correcting a luminance and color of the image captured by the camera at the high resolution by using the parameters with the converted high scale,

wherein the extracting includes;

generating a luminance histogram showing distribution of luminance in the last-captured image captured by the camera at the low resolution, by counting a number of pixels included in the last-captured image for each luminance value,

generating a color difference histogram showing distribution of color differences in the last-captured image captured by the camera at the low resolution, by counting a number of pixels included in the last-captured image for each color difference, and

obtaining a maximum value, a minimum value, and a peak value of luminance from the generated luminance histogram, and a peak value and an average value of color difference from the generated color difference histogram, thereby obtaining image effect correction parameters including the maximum value, the minimum value, and the peak value of luminance, and the peak value and the average value of color difference, and

wherein the correcting includes correcting luminance and color differences of the image captured by the camera at the high resolution, by adjusting luminance and color differences of each pixel included in the image by using the obtained image effect correction parameters.

9. A non-transitory computer readable recording medium having stored thereon a computer readable program for controlling a computer to perform functions comprising:

initially operating a camera to capture a plurality of images of an object at a low resolution;

determining whether there is any change in the images captured by the camera at the low resolution;

maintaining the camera to operate at the low resolution when it is determined that there is a change in the images captured by the camera at the low resolution, and setting the camera to operate at a high resolution to capture an image of the object at the high resolution when it is determined that there is no change in the images captured by the camera at the low resolution;

extracting parameters for performing image correction of the image captured by the camera at the high resolution from a last-captured image captured by the camera at the low resolution just before the camera is set to operate at the high resolution;

converting a low scale of the parameters extracted from the last-captured image captured by the camera at the low resolution into a high scale corresponding to the high resolution; and

correcting a luminance and color of the image captured by the camera at the high resolution by using the parameters with the converted high scale,

wherein the extracting includes;

generating a luminance histogram showing distribution of luminance in the last-captured image captured by the camera at the low resolution, by counting a number of pixels included in the last-captured image for each luminance value,

generating a color difference histogram showing distribution of color differences in the last-captured image captured by the camera at the low resolution, by counting a number of pixels included in the last-captured image for each color difference, and

obtaining a maximum value, a minimum value, and a peak value of luminance from the generated luminance histogram, and a peak value and an average value of color difference from the generated color difference histogram, thereby obtaining image effect correction parameters including the maximum value, the minimum value, and the peak value of luminance, and the peak value and the average value of color difference, and

wherein the correcting includes correcting luminance and color differences of the image captured by the camera at the high resolution, by adjusting luminance and color differences of each pixel included in the image by using the obtained image effect correction parameters.