Image processing apparatus and method, and image pickup apparatus

• US 20040061795A1
• Filed: 09/08/2003
• Published: 04/01/2004
• Est. Priority Date: 04/10/2001
• Status: Active Grant

• Current Assignee: Plug Power Inc., Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ishibashii, Junichi, Sawao, Takashi, Kondo, Tetsujiro, Nagano, Takahiro, Wada, Seiji, Miyake, Toru, Fujiwara, Naoki
• Granted Patent: US 8,134,613 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

The present invention particularly relates to an image processing apparatus in which motion blur contained in a blurred image can be eliminated. An area specifying unit 103 specifies a non-mixed area formed of a foreground area consisting of foreground object components which form a foreground object and a background area consisting of background object components which form a background object, or a mixed area in which the foreground object components and the background object components are mixed. A separating/blur-eliminating unit 1503 simultaneously performs processing for separating the foreground object components and the background object components from the pixel data of the mixed area and processing for eliminating motion blur from the separated foreground object components based on a result obtained by specifying the area. The present invention is applicable to an image processing apparatus in which a difference between a signal detected by an image-capturing device and the real world is considered.

Claims

1. An image processing apparatus for performing processing on image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels and having a time integrating function, said image processing apparatus comprising:

area specifying means for specifying a non-mixed area formed of a foreground area consisting of foreground object components which form a foreground object of the image data and a background area consisting of background object components which form a background object of the image data, or a mixed area in which the foreground object components and the background object components area mixed; and

processing execution means for simultaneously performing, based on a result obtained by specifying the area by said area specifying means, processing for separating the foreground object components and the background object components from the pixel data of the mixed area, and processing for eliminating motion blur from the separated foreground object components.

2. An image processing apparatus according to claim 1, further comprising equal-portion detection means for detecting an equal portion consisting of adjacent pixel data of the foreground area whose values are substantially equal to each other, wherein said processing execution means simultaneously performs at least the processing for separating the foreground object components and the background object components from the pixel data of the mixed area and the processing for eliminating motion blur from the separated foreground object components based on the detected equal portion and the result obtained by specifying the area by said area specifying means.

3. An image processing apparatus according to claim 2, further comprising unit-of-processing determining means for determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein said processing execution means simultaneously performs, for each unit of processing, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components.

4. An image processing apparatus according to claim 3, wherein said unit-of-processing determining means determines the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion.

5. An image processing apparatus according to claim 2, wherein said equal-portion detection means detects the equal portion by comparing a difference of the pixel data with a threshold.

6. An image processing apparatus according to claim 2, wherein said equal-portion detection means detects the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object.

7. An image processing apparatus according to claim 1, wherein said processing execution means simultaneously performs the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components by applying a calculation corresponding to a motion vector.

8. An image processing apparatus according to claim 1, wherein said processing execution means comprises:

model acquiring means for acquiring a model corresponding to the unit of processing and a motion vector;

equation generating means for generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

calculation means for calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

9. An image processing method for performing processing on image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels and having a time integrating function, said image processing method comprising:

an area specifying step of specifying a non-mixed area formed of a foreground area consisting of foreground object components which form a foreground object of the image data and a background area consisting of background object components which form a background object of the image data, or a mixed area in which the foreground object components and the background object components area mixed; and

a processing execution step of simultaneously performing, based on a result obtained by specifying the area by the processing of said area specifying step, processing for separating the foreground object components and the background object components from the pixel data of the mixed area, and processing for eliminating motion blur from the separated foreground object components.

10. An image processing method according to claim 9, further comprising an equal-portion detection step of detecting an equal portion consisting of adjacent pixel data of the foreground area whose values are substantially equal to each other, wherein, in the processing of said processing execution step, at least the processing for separating the foreground object components and the background object components from the pixel data of the mixed area and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed based on the detected equal portion and the result obtained by specifying the area by said area specifying step.

11. An image processing method according to claim 10, further comprising a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed for each unit of processing.

12. An image processing method according to claim 11, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

13. An image processing method according to claim 10, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

14. An image processing method according to claim 10, wherein, in the processing of said equal-portion detection step, the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

15. An image processing method according to claim 9, wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed by applying a calculation corresponding to a motion vector.

16. An image processing method according to claim 9, wherein the processing of said processing execution step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

a calculation step of calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

17. A storage medium in which a computer-readable program for performing processing on image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels and having a time integrating function is stored, said computer-readable program comprising:

an area specifying step of specifying a non-mixed area formed of a foreground area consisting of foreground object components which form a foreground object of the image data and a background area consisting of background object components which form a background object of the image data, or a mixed area in which the foreground object components and the background object components area mixed; and

a processing execution step of simultaneously performing, based on a result obtained by specifying the area by the processing of said area specifying step, processing for separating the foreground object components and the background object components from the pixel data of the mixed area, and processing for eliminating motion blur from the separated foreground object components.

18. A storage medium according to claim 17, wherein said program further comprises an equal-portion detection step of detecting an equal portion consisting of adjacent pixel data of the foreground area whose values are substantially equal to each other, and wherein, in the processing of said processing execution step, at least the processing for separating the foreground object components and the background object components from the pixel data of the mixed area and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed based on the detected equal portion and the result obtained by specifying the area by said area specifying step.

19. A storage medium according to claim 18, wherein said program further comprises a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, and wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed for each unit of processing.

20. A storage medium according to claim 19, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

21. A storage medium according to claim 18, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

22. A storage medium according to claim 18, wherein, in the processing of said equal-portion detection step, the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

23. A storage medium according to claim 17, wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed by applying a calculation corresponding to a motion vector.

24. A storage medium according to claim 17, wherein the processing of said processing execution step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

a calculation step of calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

25. A program for allowing a computer for performing processing on image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels and having a time integrating function to execute:

an area specifying step of specifying a non-mixed area formed of a foreground area consisting of foreground object components which form a foreground object of the image data and a background area consisting of background object components which form a background object of the image data, or a mixed area in which the foreground object components and the background object components area mixed; and

a processing execution step of simultaneously performing, based on a result obtained by specifying the area by the processing of said area specifying step, processing for separating the foreground object components and the background object components from the pixel data of the mixed area, and processing for eliminating motion blur from the separated foreground object components.

26. A program according to claim 25, further comprising an equal-portion detection step of detecting an equal portion consisting of adjacent pixel data of the foreground area whose values are substantially equal to each other, wherein, in the processing of said processing execution step, at least the processing for separating the foreground object components and the background object components from the pixel data of the mixed area and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed based on the detected equal portion and the result obtained by specifying the area by the processing of said area specifying step.

27. A program according to claim 26, further comprising a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed for each unit of processing.

28. A program according to claim 27, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

29. A program according to claim 26, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

30. A program according to claim 26, wherein, in the processing of said equal-portion detection step, the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

31. A program according to claim 25, wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed by applying a calculation corresponding to a motion vector.

32. A program according to claim 25, wherein the processing of said processing execution step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

a calculation step of calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

33. An image processing apparatus for performing processing on image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels and having a time integrating function, said image processing apparatus comprising:

input means for inputting the image data having an object area consisting of object components which form an object; and

motion-blur eliminating means for eliminating motion blur occurring in the object area by assuming that values of part of the pixel data in the object area of the image data input by said input means are substantially equal.

34. An image processing apparatus according to claim 33, wherein:

said input means inputs the image data having a foreground area consisting of foreground object components which form the object, a background area consisting of background object components which form a background object, and a mixed area in which the foreground object components and the background object components are mixed; and

said motion-blur eliminating means eliminates motion blur occurring in the foreground area by assuming that values of part of the pixel data in the foreground area of the image data input by said input means are substantially equal.

35. An image processing apparatus according to claim 34, further comprising equal-portion detection means for detecting an equal portion in which the values of the pixel data in the foreground area of the image data are substantially equal, wherein said motion-blur eliminating means eliminates motion blur occurring in the foreground area based on the equal portion detected by said equal-portion detection means.

36. An image processing apparatus according to claim 35, further comprising unit-of-processing determining means for determining a unit of processing consisting of a plurality of the foreground object components based on a position of the equal portion, wherein said motion-blur eliminating means eliminates motion blur of the foreground area for each unit of processing.

37. An image processing apparatus according to claim 36, wherein said unit-of-processing determining means determines the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion.

38. An image processing apparatus according to claim 35, further comprising area specifying means for specifying the foreground area, the background area, or the mixed area.

39. An image processing apparatus according to claim 35, wherein said equal-portion detection means detects the equal portion by comparing a difference of the pixel data with a threshold.

40. An image processing apparatus according to claim 35, wherein said equal-portion detection means detects the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object.

41. An image processing apparatus according to claim 35, wherein said motion-blur eliminating means eliminates motion blur occurring in the foreground area by applying a calculation corresponding to a motion vector.

42. An image processing apparatus according to claim 35, wherein said motion-blur eliminating means comprises:

model acquiring means for acquiring a model corresponding to the unit of processing and a motion vector;

equation generating means for generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components contained in the unit of processing; and

calculation means for calculating the foreground object components contained in the unit of processing based on the generated equation.

43. An image processing apparatus according to claim 35, wherein said motion-blur eliminating means simultaneously performs processing for separating the pixel data of the mixed area into the foreground object components and the background object components and processing for eliminating motion blur from the separated foreground object components based on area information indicating a non-mixed area consisting of the foreground area and the background area or the mixed area and based on the equal portion.

44. An image processing apparatus according to claim 43, further comprising unit-of-processing determining means for determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein said processing execution means simultaneously performs, for each unit of processing, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components.

45. An image processing apparatus according to claim 44, wherein said unit-of-processing determining means determines the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion.

46. An image processing apparatus according to claim 43, further comprising area specifying means for specifying the foreground area, the background area, or the mixed area.

47. An image processing apparatus according to claim 43, wherein said equal-portion detection means detects the equal portion by comparing a difference of the pixel data with a threshold.

48. An image processing apparatus according to claim 43, wherein said equal-portion detection means detects the equal portion consisting of adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object.

49. An image processing apparatus according to claim 43, wherein said unit-or-processing determining means simultaneously performs the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components by applying a calculation corresponding to a motion vector.

50. An image processing apparatus according to claim 43, wherein said unit-of-processing determining means comprises:

model acquiring means for acquiring a model corresponding to the unit of processing and a motion vector;

equation generating means for generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

calculation means for calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

51. An image processing method for performing processing on image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels and having a time integrating function, said image processing method comprising:

an input step of inputting the image data having an object area consisting of object components which form an object; and

a motion-blur eliminating step of eliminating motion blur occurring in the object area by assuming that values of part of the pixel data in the object area of the image data input by the processing of said input step are substantially equal.

52. An image processing method according to claim 51, wherein:

in the processing of said input step, the image data having a foreground area consisting of foreground object components which form the object, a background area consisting of background object components which form a background object, and a mixed area in which the foreground object components and the background object components are mixed are input; and

in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated by assuming that values of part of the pixel data in the foreground area of the image data input by the processing of said input step are substantially equal.

53. An image processing method according to claim 52, further comprising an equal-portion detection step of detecting an equal portion in which the values of the pixel data in the foreground area of the image data are substantially equal, wherein in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated based on the equal portion detected by the processing of said equal-portion detection step.

54. An image processing method according to claim 53, further comprising a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components based on a position of the equal portion, wherein, in the processing of said motion-blur eliminating step, motion blur of the foreground area is eliminated for each unit of processing.

55. An image processing method according to claim 54, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

56. An image processing method according to claim 53, further comprising an area specifying step of specifying the foreground area, the background area, or the mixed area.

57. An image processing method according to claim 53, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

58. An image processing method according to claim 53, wherein, in the processing of said equal-portion detection step, the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

59. An image processing method according to claim 53, wherein, in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated by applying a calculation corresponding to a motion vector.

60. An image processing method according to claim 53, wherein the processing of said motion-blur eliminating step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components contained in the unit of processing; and

a calculation step of calculating the foreground object components contained in the unit of processing based on the generated equation.

61. An image processing method according to claim 53, wherein, in the processing of said motion-blur eliminating step, processing for separating the pixel data of the mixed area into the foreground object components and the background object components and processing for eliminating motion blur from the separated foreground object components are simultaneously performed based on area information indicating a non-mixed area consisting of the foreground area and the background area or the mixed area and based on the equal portion.

62. An image processing method according to claim 61, further comprising a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed for each unit of processing.

63. An image processing method according to claim 62, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

64. An image processing method according to claim 61, further comprising an area specifying step of specifying the foreground area, the background area, or the mixed area.

65. An image processing method according to claim 61, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

66. An image processing method according to claim 61, wherein, in the processing of said equal-portion detection step, the equal portion consisting of adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

67. An image processing method according to claim 61, wherein, in the processing of said unit-or-processing determining step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed by applying a calculation corresponding to a motion vector.

68. An image processing method according to claim 61, wherein the processing of said unit-of-processing determining step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

a calculation step of calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

69. A storage medium in which a computer-readable program for performing processing on image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels and having a time integrating function is stored, said computer-readable program comprising:

an input step of inputting the image data having an object area consisting of object components which form an object; and

a motion-blur eliminating step of eliminating motion blur occurring in the object area by assuming that values of part of the pixel data in the object area of the image data input by the processing of said input step are substantially equal.

70. A storage medium according to claim 69, wherein:

in the processing of said input step, the image data having a foreground area consisting of foreground object components which form the object, a background area consisting of background object components which form a background object, and a mixed area in which the foreground object components and the background object components are mixed are input; and

in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated by assuming that values of part of the pixel data in the foreground area of the image data input by the processing of said input step are substantially equal.

71. A storage medium according to claim 70, further comprising an equal-portion detection step of detecting an equal portion in which the values of the pixel data in the foreground area of the image data are substantially equal, wherein in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated based on the equal portion detected by the processing of said equal-portion detection step.

72. A storage medium according to claim 71, further comprising a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components based on a position of the equal portion, wherein, in the processing of said motion-blur eliminating step, motion blur of the foreground area is eliminated for each unit of processing.

73. A storage medium according to claim 72, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

74. A storage medium according to claim 71, further comprising an area specifying step of specifying the foreground area, the background area, or the mixed area.

75. A storage medium according to claim 71, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

76. A storage medium according to claim 71, wherein, in the processing of said equal-portion detection step, the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

77. A storage medium according to claim 71, wherein, in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated by applying a calculation corresponding to a motion vector.

78. A storage medium according to claim 71, wherein the processing of said motion-blur eliminating step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components contained in the unit of processing; and

a calculation step of calculating the foreground object components contained in the unit of processing based on the generated equation.

79. A storage medium according to claim 71, wherein, in the processing of said motion-blur eliminating step, processing for separating the pixel data of the mixed area into the foreground object components and the background object components and processing for eliminating motion blur from the separated foreground object components are simultaneously performed based on area information indicating a non-mixed area consisting of the foreground area and the background area or the mixed area and based on the equal portion.

80. A storage medium according to claim 79, further comprising a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed for each unit of processing.

81. A storage medium according to claim 80, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

82. A storage medium according to claim 79, further comprising an area specifying step of specifying the foreground area, the background area, or the mixed area.

83. A storage medium according to claim 79, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

84. A storage medium according to claim 79, wherein, in the processing of said equal-portion detection step, the equal portion consisting of adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

85. A storage medium according to claim 79, wherein, in the processing of said unit-or-processing determining step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed by applying a calculation corresponding to a motion vector.

86. A storage medium according to claim 79, wherein the processing of said unit-of-processing determining step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

a calculation step of calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

87. A program for allowing a computer for performing processing on image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels and having a time integrating function to execute:

an input step of inputting the image data having an object area consisting of object components which form an object; and

a motion-blur eliminating step of eliminating motion blur occurring in the object area by assuming that values of part of the pixel data in the object area of the image data input by the processing of said input step are substantially equal.

88. A program according to claim 87, wherein:

in the processing of said input step, the image data having a foreground area consisting of foreground object components which form the object, a background area consisting of background object components which form a background object, and a mixed area in which the foreground object components and the background object components are mixed are input; and

in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated by assuming that values of part of the pixel data in the foreground area of the image data input by the processing of said input step are substantially equal.

89. A program according to claim 88, further comprising an equal-portion detection step of detecting an equal portion in which the values of the pixel data in the foreground area of the image data are substantially equal, wherein in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated based on the equal portion detected by the processing of said equal-portion detection step.

90. A program according to claim 89, further comprising a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components based on a position of the equal portion, wherein, in the processing of said motion-blur eliminating step, motion blur of the foreground area is eliminated for each unit of processing.

91. A program according to claim 90, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

92. A program according to claim 89, further comprising an area specifying step of specifying the foreground area, the background area, or the mixed area.

93. A program according to claim 89, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

94. A program according to claim 89, wherein, in the processing of said equal-portion detection step, the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

95. A program according to claim 89, wherein, in the processing of said motion-blur eliminating step, motion blur occurring in the foreground area is eliminated by applying a calculation corresponding to a motion vector.

96. A program according to claim 89, wherein the processing of said motion-blur eliminating step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components contained in the unit of processing; and

a calculation step of calculating the foreground object components contained in the unit of processing based on the generated equation.

97. A program according to claim 89, wherein, in the processing of said motion-blur eliminating step, processing for separating the pixel data of the mixed area into the foreground object components and the background object components and processing for eliminating motion blur from the separated foreground object components are simultaneously performed based on area information indicating a non-mixed area consisting of the foreground area and the background area or the mixed area and based on the equal portion.

98. A program according to claim 97, further comprising a unit-of-processing determining step of determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein, in the processing of said processing execution step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed for each unit of processing.

99. A program according to claim 98, wherein, in the processing of said unit-of-processing determining step, the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion is determined.

100. A program according to claim 97, further comprising an area specifying step of specifying the foreground area, the background area, or the mixed area.

101. A program according to claim 97, wherein, in the processing of said equal-portion detection step, the equal portion is detected by comparing a difference of the pixel data with a threshold.

102. A program according to claim 97, wherein, in the processing of said equal-portion detection step, the equal portion consisting of adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object is detected.

103. A program according to claim 97, wherein, in the processing of said unit-or-processing determining step, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components are simultaneously performed by applying a calculation corresponding to a motion vector.

104. A program according to claim 97, wherein the processing of said unit-of-processing determining step comprises:

a model acquiring step of acquiring a model corresponding to the unit of processing and a motion vector;

an equation generating step of generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

a calculation step of calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

105. An image-capturing apparatus comprising:

image-capturing means for outputting a subject image captured by an image-capturing device including a predetermined number of pixels and having a time integrating function as image data formed of a predetermined number of pixel data;

area specifying means for specifying a non-mixed area formed of a foreground area consisting of foreground object components which form a foreground object of the image data and a background area consisting of background object components which form a background object of the image data, or a mixed area in which the foreground object components and the background object components area mixed; and

processing execution means for simultaneously performing, based on a result obtained by specifying the area by said area specifying means, processing for separating the foreground object components and the background object components from the pixel data of the mixed area, and processing for eliminating motion blur from the separated foreground object components.

106. An image-capturing apparatus according to claim 105, further comprising equal-portion detection means for detecting an equal portion consisting of adjacent pixel data of the foreground area whose values are substantially equal to each other, wherein said processing execution means simultaneously performs at least the processing for separating the foreground object components and the background object components from the pixel data of the mixed area and the processing for eliminating motion blur from the separated foreground object components based on the detected equal portion and the result obtained by specifying the area by said area specifying means.

107. An image-capturing apparatus according to claim 106, further comprising unit-of-processing determining means for determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein said processing execution means simultaneously performs, for each unit of processing, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components.

108. An image-capturing apparatus according to claim 107, wherein said unit-of-processing determining means determines the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion.

109. An image-capturing apparatus according to claim 106, wherein said equal-portion detection means detects the equal portion by comparing a difference of the pixel data with a threshold.

110. An image-capturing apparatus according to claim 106, wherein said equal-portion detection means detects the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object.

111. An image-capturing apparatus according to claim 105, wherein said processing execution means simultaneously performs the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components by applying a calculation corresponding to a motion vector.

112. An image-capturing apparatus according to claim 105, wherein said processing execution means comprises:

model acquiring means for acquiring a model corresponding to the unit of processing and a motion vector;

equation generating means for generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

calculation means for calculating the foreground object components and the background object components contained in the unit of processing based on the generated equation.

113. An image-capturing apparatus comprising:

image-capturing means for outputting a subject image captured by an image-capturing device including a predetermined number of pixels and having a time integrating function as image data formed of a predetermined number of pixel data and having an object area consisting of object components which form an object; and

motion-blur eliminating means for eliminating motion blur occurring in the object area by assuming that values of part of the pixel data in the object area of the image data are substantially equal.

114. An image-capturing apparatus according to claim 113, wherein:

said input means inputs the image data having a foreground area consisting of foreground object components which form the object, a background area consisting of background object components which form a background object, and a mixed area in which the foreground object components and the background object components are mixed; and

said motion-blur eliminating means eliminates motion blur occurring in the foreground area by assuming that values of part of the pixel data in the foreground area of the image data input by said input means are substantially equal.

115. An image-capturing apparatus according to claim 114, further comprising equal-portion detection means for detecting an equal portion in which the values of the pixel data in the foreground area of the image data are substantially equal, wherein said motion-blur eliminating means eliminates motion blur occurring in the foreground area based on the equal portion detected by said equal-portion detection means.

116. An image-capturing apparatus according to claim 115, further comprising unit-of-processing determining means for determining a unit of processing consisting of a plurality of the foreground object components based on a position of the equal portion, wherein said motion-blur eliminating means eliminates motion blur of the foreground area for each unit of processing.

117. An image-capturing apparatus according to claim 116, wherein said unit-of-processing determining means determines the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion.

118. An image-capturing apparatus according to claim 115, further comprising area specifying means for specifying the foreground area, the background area, or the mixed area.

119. An image-capturing apparatus according to claim 115, wherein said equal-portion detection means detects the equal portion by comparing a difference of the pixel data with a threshold.

120. An image-capturing apparatus according to claim 115, wherein said equal-portion detection means detects the equal portion consisting of the adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object.

121. An image-capturing apparatus according to claim 115, wherein said motion-blur eliminating means eliminates motion blur occurring in the foreground area by applying a calculation corresponding to a motion vector.

122. An image-capturing apparatus according to claim 115, wherein said motion-blur eliminating means comprises:

model acquiring means for acquiring a model corresponding to the unit of processing and a motion vector;

equation generating means for generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components contained in the unit of processing; and

calculation means for calculating the foreground object components contained in the unit of processing based on the generated equation.

123. An image-capturing apparatus according to claim 115, wherein said motion-blur eliminating means simultaneously performs processing for separating the pixel data of the mixed area into the foreground object components and the background object components and processing for eliminating motion blur from the separated foreground object components based on area information indicating a non-mixed area consisting of the foreground area and the background area or the mixed area and based on the equal portion.

124. An image-capturing apparatus according to claim 123, further comprising unit-of-processing determining means for determining a unit of processing consisting of a plurality of the foreground object components and the background object components based on a position of the equal portion, wherein said processing execution means simultaneously performs, for each unit of processing, the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components.

125. An image-capturing apparatus according to claim 124, wherein said unit-of-processing determining means determines the unit of processing corresponding to the pixel data belonging to the mixed area or the foreground area which is located on one straight line and which is other than the pixel data of the equal portion.

126. An image-capturing apparatus according to claim 123, further comprising area specifying means for specifying the foreground area, the background area, or the mixed area.

127. An image-capturing apparatus according to claim 123, wherein said equal-portion detection means detects the equal portion by comparing a difference of the pixel data with a threshold.

128. An image-capturing apparatus according to claim 123, wherein said equal-portion detection means detects the equal portion consisting of adjacent pixel data having the number of pixels greater than or equal to the number of pixels corresponding to an amount of movement of the foreground object.

129. An image-capturing apparatus according to claim 123, wherein said unit-or-processing determining, means simultaneously performs the processing for separating the foreground object components and the background object components and the processing for eliminating motion blur from the separated foreground object components by applying a calculation corresponding to a motion vector.

130. An image-capturing apparatus according to claim 123, wherein said unit-of-processing determining means comprises:

model acquiring means for acquiring a model corresponding to the unit of processing and a motion vector;

equation generating means for generating, based on the acquired model, an equation corresponding to a relationship between the pixel data of the unit of processing and the foreground object components and the background object components contained in the unit of processing; and

calculation means for calculating the foreground object components and the background object components contained in the unit of processing base on the generated equation.