Generation of still image

• US 20050157949A1
• Filed: 09/28/2004
• Published: 07/21/2005
• Est. Priority Date: 09/30/2003
• Status: Abandoned Application

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Aiso, Seiji, Matsuzaka, Kenji
• Application Number:
  US10/954,027
• Publication Number:
  US 20050157949A1
• Time in Patent Office: Days
• US Class Current: 382/299
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images

Abstract

A still image generation apparatus of the invention includes an image acquisition module that obtains multiple first image data arrayed in a time series among multiple lower-resolution image data, an image storage module that stores the multiple first image data obtained by the image acquisition module, and a correction rate estimation module that estimates correction rates for eliminating positional shifts between images of the respective first image data, based on the multiple first image data stored in the image storage module. The still image generation apparatus further includes an image composition module that corrects the multiple first image data with the estimated correction rates to eliminate the positional shifts between the images of the respective first image data, and combines the multiple corrected first image data to generate higher-resolution second image data as resulting still image data. This arrangement of the invention desirably shortens the total processing time in the process of combining multiple image data.

Claims

1. A still image generation method of generating higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the still image generation method comprising the steps of:

(a) correcting the multiple first image data to eliminate a positional shift between images of the multiple first image data;

(b) detecting a motion in each of the images of the multiple first image data, based on comparison of the multiple corrected first image data; and

(c) selecting one resolution enhancement process among multiple available resolution enhancement processes according to a result of the detection.

2. A still image generation method in accordance with claim 1, the still image generation method further comprising the step of:

(d) executing the selected resolution enhancement process to generate the higher-resolution second image data from the multiple corrected lower-resolution first image data.

3. A still image generation method in accordance with claim 1, the still image generation method further comprising the step of:

(d) notifying a user of the selected resolution enhancement process as a recommendation of resolution enhancement process.

4. A still image generation method in accordance with claim 1, wherein the multiple first image data are multiple image data that are extracted from moving picture data and are arrayed in a time series.

5. A still image generation method of generating higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the still image generation method comprising the steps of:

(a) correcting the multiple first image data to eliminate a positional shift between images of the multiple first image data;

(b) comparing base image data set as a standard with at least one subject image data other than the base image data among the multiple corrected first image data, detecting each localized motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data, and calculating a motion rate as a total sum of localized motions over the whole subject image; and

(c) selecting one resolution enhancement process among multiple available resolution enhancement processes according to the calculated motion rate.

6. A still image generation method in accordance with claim 5, the still image generation method further comprising the step of:

(d) executing the selected resolution enhancement process to generate the higher-resolution second image data from the multiple corrected lower-resolution first image data.

7. A still image generation method in accordance with claim 5, the still image generation method further comprising the step of:

(d) notifying a user of the selected resolution enhancement process as a recommendation of resolution enhancement process.

8. A still image generation method in accordance with claim 5, wherein the multiple first image data are multiple image data that are extracted from moving picture data and are arrayed in a time series.

9. A still image generation method in accordance with claim 5, wherein the step (b) detects a motion or no motion of each pixel included in the subject image relative to the base image, and calculates the motion rate from a total number of pixels detected as a pixel with motion.

10. A still image generation method in accordance with claim 9, wherein the step (b) sequentially sets each pixel in the subject image as an object pixel or an object of motion detection in the subject image relative to the base image, sets an object range of the motion detection based on a pixel value of a nearby pixel in the base image that is located near to the object pixel, and detects the object pixel as the pixel with motion when a pixel value of the object pixel is within the object range, while detecting the object pixel as a pixel with no motion when the pixel value of the object pixel is out of the object range.

11. A still image generation method in accordance with claim 9, wherein the step (b) sequentially sets each pixel in the subject image as an object pixel or an object of motion detection in the subject image relative to the base image, estimates an assumed pixel to have an identical pixel value with a pixel value of the object pixel based on a pixel value of a nearby pixel in the base image that is located near to the object pixel, and detects the object pixel as the pixel with motion when a distance between the object pixel and the assumed pixel is greater than a preset threshold value, while detecting the object pixel as a pixel with no motion when the distance is not greater than the preset threshold value.

12. A still image generation method in accordance with claim 5, wherein the step (b) computes a motion value of each pixel in the subject image, which represents a degree of motion of the pixel in the subject image relative to the base image, and calculates the motion rate from a total sum of the computed motion values.

13. A still image generation method in accordance with claim 12, wherein the step (b) sequentially sets each pixel in the subject image as an object pixel or an object of motion detection in the subject image relative to the base image, sets a reference pixel value based on a pixel value of a nearby pixel in the base image that is located near to the object pixel, and computes a difference between a pixel value of the object pixel and the reference pixel value as the motion value of the object pixel.

14. A still image generation method in accordance with claim 12, wherein the step (b) sequentially sets each pixel in the subject image as an object pixel or an object of motion detection in the subject image relative to the base image, estimates an assumed pixel to have an identical pixel value with a pixel value of the object pixel based on a pixel value of a nearby pixel in the base image that is located near to the object pixel, and computes a distance between the object pixel and the assumed pixel as the motion value of the object pixel.

15. A still image generation method of generating higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the still image generation method comprising the steps of:

(a) comparing base image data set as a standard with at least one subject image data other than the base image data among the multiple first image data, detecting a motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data with regard to each of multiple blocks obtained by dividing the subject image, and determining a motion rate, which represents a degree of motion in the whole subject image relative to the base image, based on a result of the motion detection; and

(b) selecting one resolution enhancement process among multiple available resolution enhancement processes according to the determined motion rate.

16. A still image generation method in accordance with claim 15, the still image generation method further comprising the step of:

(c) executing the selected resolution enhancement process to generate the second image data from the multiple first image data.

17. A still image generation method in accordance with claim 15, the still image generation method further comprising the step of:

(c) notifying a user of the selected resolution enhancement process as a recommendation of resolution enhancement process.

18. A still image generation method in accordance with claim 15, the still image generation method further comprising the step of:

(c) detecting a first positional shift of the whole subject image relative to the base image and second positional shifts of respective blocks included in the subject image relative to corresponding blocks of the base image,

wherein the step (a) detects a motion in a specified block, based on the detected first positional shift of the whole subject image and the detected second positional shift of the specified block.

19. A still image generation method in accordance with claim 15, wherein the step (a) detects a motion or no motion of each pixel included in a specified block of the subject image relative to a corresponding block of the base image, and detects a motion in the specified block, based on a total number of pixels detected as a pixel with motion.

20. A still image generation method in accordance with claim 15, wherein the step (a) computes a motion value of each pixel in a specified block of the subject image, which represents a magnitude of motion of the subject image relative to the base image, and detects a motion in the specified block, based on a total sum of the computed motion values.

21. A still image generation method in accordance with claim 15, wherein the step (a) calculates the motion rate from a total number of blocks detected as a block with motion.

22. A still image generation method in accordance with claim 15, wherein the step (a) calculates the motion rate from a total sum of magnitudes of motions detected in respective blocks.

23. A still image generation method in accordance with claim 15, wherein the multiple first image data are multiple image data that are extracted from moving picture data and are arrayed in a time series.

24. A still image generation method of generating higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the still image generation method comprising the steps of:

(a) comparing base image data set as a standard with at least one subject image data other than the base image data among the multiple first image data, detecting a motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data with regard to each of multiple blocks obtained by dividing the subject image, and determining an in-block motion rate of each block of the subject image, which represents a degree of motion in the block of the subject image relative to a corresponding block of the base image, based on a result of the motion detection;

(b) selecting one resolution enhancement process for each block among multiple available resolution enhancement processes according to the determined in-block motion rate; and

(c) executing the resolution enhancement process selected for each block, so as to generate the second image data representing the block of the resulting still image from the multiple first image data.

25. A still image generation method in accordance with claim 24, the still image generation method further comprising the step of:

(d) detecting a first positional shift of the whole subject image relative to the base image and second positional shifts of respective blocks included in the subject image relative to corresponding blocks of the base image,

wherein the step (a) detects a motion in a specified block, based on the detected first positional shift of the whole subject image and the detected second positional shift of the specified block.

26. A still image generation method in accordance with claim 24, wherein the step (a) detects a motion or no motion of each pixel included in a specified block of the subject image relative to a corresponding block of the base image, and detects a motion in the specified block, based on a total number of pixels detected as a pixel with motion.

27. A still image generation method in accordance with claim 24, wherein the step (a) computes a motion value of each pixel in a specified block of the subject image, which represents a magnitude of motion of the subject image relative to the base image, and detects a motion in the specified block, based on a total sum of the computed motion values.

28. A still image generation method in accordance with claim 24, wherein the multiple first image data are multiple image data that are extracted from moving picture data and are arrayed in a time series.

29. A still image generation apparatus that generates higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the still image generation apparatus comprising:

a shift correction module that corrects the multiple first image data to eliminate a positional shift between images of the multiple first image data;

a motion detection module that detects a motion in each of the images of the multiple first image data, based on comparison of the multiple corrected first image data; and

a resolution enhancement process selection module that selects one resolution enhancement process among multiple available resolution enhancement processes according to a result of the detection.

30. A still image generation apparatus that generates higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the still image generation apparatus comprising:

a shift correction module that corrects the multiple first image data to eliminate a positional shift between images of the multiple first image data;

a motion detection module that compares base image data set as a standard with at least one subject image data other than the base image data among the multiple corrected first image data, detects each localized motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data, and calculates a motion rate as a total sum of localized motions over the whole subject image; and

a resolution enhancement process selection module that selects one resolution enhancement process among multiple available resolution enhancement processes according to the calculated motion rate.

31. A still image generation apparatus that generates higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the still image generation apparatus comprising:

a motion detection module that compares base image data set as a standard with at least one subject image data other than the base image data among the multiple first image data, detects a motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data with regard to each of multiple blocks obtained by dividing the subject image, and determines a motion rate, which represents a degree of motion in the whole subject image relative to the base image, based on a result of the motion detection; and

a resolution enhancement process selection module that selects one resolution enhancement process among multiple available resolution enhancement processes according to the determined motion rate.

32. A still image generation apparatus that generates higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the still image generation apparatus comprising:

a motion detection module that compares base image data set as a standard with at least one subject image data other than the base image data among the multiple first image data, detects a motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data with regard to each of multiple blocks obtained by dividing the subject image, and determines an in-block motion rate of each block of the subject image, which represents a degree of motion in the block of the subject image relative to a corresponding block of the base image, based on a result of the motion detection;

a resolution enhancement process selection module that selects one resolution enhancement process for each block among multiple available resolution enhancement processes according to the determined in-block motion rate; and

a resolution enhancement module that executes the resolution enhancement process selected for each block, so as to generate the second image data representing the block of the resulting still image from the multiple first image data.

33. A computer program product used to generate higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the computer program product comprising:

a first program code of correcting the multiple first image data to eliminate a positional shift between images of the multiple first image data;

a second program code of detecting a motion in each of the images of the multiple first image data, based on comparison of the multiple corrected first image data;

a third program code of selecting one resolution enhancement process among multiple available resolution enhancement processes according to a result of the detection; and

a computer readable medium to store the first through the third program codes.

34. A computer program product used to generate higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the computer program product comprising:

a first program code of correcting the multiple first image data to eliminate a positional shift between images of the multiple first image data;

a second program code of comparing base image data set as a standard with at least one subject image data other than the base image data among the multiple corrected first image data, detecting each localized motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data, and calculating a motion rate as a total sum of localized motions over the whole subject image;

a third program code of selecting one resolution enhancement process among multiple available resolution enhancement processes according to the calculated motion rate; and

a computer readable medium to store the first through the third program codes.

35. A computer program product used to generate higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the computer program product comprising:

a first program code of comparing base image data set as a standard with at least one subject image data other than the base image data among the multiple first image data, detecting a motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data with regard to each of multiple blocks obtained by dividing the subject image, and determining a motion rate, which represents a degree of motion in the whole subject image relative to the base image, based on a result of the motion detection;

a second program code of selecting one resolution enhancement process among multiple available resolution enhancement processes according to the determined motion rate; and

a computer readable medium to store the first and second program codes.

36. A computer program product used to generate higher-resolution second image data, which represents a resulting still image, from multiple lower-resolution first image data, the computer program product comprising:

a first program code of comparing base image data set as a standard with at least one subject image data other than the base image data among the multiple first image data, detecting a motion in a subject image expressed by the at least one subject image data relative to a base image expressed by the base image data with regard to each of multiple blocks obtained by dividing the subject image, and determining an in-block motion rate of each block of the subject image, which represents a degree of motion in the block of the subject image relative to a corresponding block of the base image, based on a result of the motion detection;

a second program code of selecting one resolution enhancement process for each block among multiple available resolution enhancement processes according to the determined in-block motion rate;

a third program code of executing the resolution enhancement process selected for each block, so as to generate the second image data representing the block of the resulting still image from the multiple first image data; and

a computer readable medium to store the first through the third program codes.