Image signal interpolation

• US 6,563,537 B1
• Filed: 07/30/1998
• Issued: 05/13/2003
• Est. Priority Date: 07/31/1997
• Status: Expired due to Term

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Kawamura, Kazuo, Yashima, Hideaki
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): Rosendale, Matthew L
• Application Number:
  US09/127,128
• Time in Patent Office: 1,748 Days
• Field of Search: 348/272, 348/273, 348/282, 348/253, 348/625, 348/628, 348/629, 348/630, 348/252, 348/280, 382/199, 382/266
• US Class Current: 348/252
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

Image signal processing has the steps of: judging from four pixels adjacent to an object pixel whether the object pixel constitutes a single edge; judging whether the single edge is horizontal or vertical, if it is judged that the object pixel constitutes the single edge; and interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if it is judged that the single edge is horizontal, and interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if it is judged that the single edge is vertical.

Claims

1. An image signal processing apparatus comprising:

first edge judgement means for judging from at least four pixels adjacent to an object pixel whether the object pixel constitutes a single edge;

first edge direction judging means for judging whether the single edge is horizontal or vertical, if said first edge judging means judges that the object pixel constitutes the single edge;

interpolating means for interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if said first edge direction judging means judges that the single edge is horizontal, and for interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if said first edge direction judging means judges that the single edge is vertical;

second edge judgement means for judging from the at least four pixels adjacent to the object pixel whether the object pixel constitutes two edges, if said first edge judgement means judges that the object pixel does not constitute the single edge; and

second edge direction judging means for judging whether the two edges are horizontal or vertical, if said second edge judging means judges that the object pixel constitutes the two edges,

wherein said interpolating means interpolates the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in the horizontal direction, if said second edge direction judging means judges that the two edges are horizontal, and interpolates the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in the vertical direction, if said second edge direction judging means judges that the two edges are vertical.

4. An image signal processing apparatus according to claim 1, wherein said first edge judging means judges whether the object pixel constitutes the single edge in accordance with a difference between values of right and left pixels adjacent to the object pixel in the horizontal direction and a difference between values of upper and lower pixels adjacent to the object pixel in the vertical direction.

5. An image signal processing apparatus according to claim 1, wherein said first edge direction judging means judges an edge direction in accordance with a difference between values of right and left pixels adjacent to the object pixel in the horizontal direction and a difference between values of upper and lower pixels adjacent to the object pixel in the vertical direction.

6. An image processing apparatus according to claim 5, wherein the judging means determines an edge direction based on a relative value of an absolute value of the difference between values of the right and left pixels and an absolute value of the difference between values of the upper and lower pixels.

7. An image signal processing apparatus according to claim 1, wherein the image signal processing apparatus sequentially interpolates each of a plurality of pixels in an image as the object pixel and said second edge direction judging means judges an edge direction in accordance with a value of a pixel shifted by one pixel in the horizontal direction and then in the vertical direction from the object pixel and interpolated by said interpolating means.

8. An image signal processing apparatus according to claim 1, wherein said second edge direction judging means comprises:

horizontal direction difference means for calculating a difference of values between a first column including the object pixel and upper and lower pixels adjacent to the object pixel in the vertical direction and a second column adjacent to the first column and disposed in the vertical direction;

vertical direction difference means for calculating a difference of values between a first row including the object pixel and right and left pixels adjacent to the object pixel in the horizontal direction and a second row adjacent to the first row and disposed in the horizontal direction; and

difference judging means for judging an edge direction in accordance with the difference of values between the first and second columns and the difference of values between the first and second rows.

9. An image signal processing apparatus according to claims 8, wherein said horizontal direction difference means calculates a difference between a sum of values of at least first two pixels among the object pixel and upper and lower pixels adjacent to the object pixel in the first column and a sum of values of at least second two pixels adjacent to the first two pixels in the horizontal direction in the second column, and said vertical direction difference means calculates a difference between a sum of values of at least first two pixels among the object pixel and right and left pixels adjacent to the object pixel in the first row and a sum of values of at least second two pixels adjacent to the first two pixels in the vertical direction in the second row.

10. An image signal processing apparatus according to claim 9, wherein said horizontal direction difference means calculates a difference between a sum of values of first three pixels including the object pixel and upper and lower pixels adjacent to the object pixel in the first column and a sum of values of second three pixels adjacent to the first three pixels in the horizontal direction in the second column, and said vertical direction difference means calculates a difference between a sum of values of first three pixels including the object pixel and right and left pixels adjacent to the object pixel in the first row and a sum of values of second three pixels adjacent to the first three pixels in the vertical direction in the second row.

11. An image signal processing apparatus according to claim 1, wherein said second edge judging means judges whether the object pixel constitutes the two edges, in accordance with a sum of values of right and left pixels adjacent to the object pixel in the horizontal direction and a sum of values of upper and lower pixels adjacent to the object pixel in the vertical direction.

2. An image signal processing apparatus according to claim 2, wherein said interpolating means interpolates the object pixel in accordance with at least four pixels adjacent to the object pixel, if said second edge judging means judges that the object pixel does not constitutes the two edges.

3. An image signal processing apparatus according to claim 2, wherein the image signal processing apparatus sequentially interpolates each of a plurality of pixels in an image as the object pixel and further comprises:

storage means for storing an edge direction of each of the plurality of object pixels, the edge direction being judged by said first and second edge direction judging means; and

re-judging means for re-judging that the object pixel has a different direction if a majority of the edge directions of pixels adjacent to the object pixel and stored in said storage means has an edge direction different from an edge direction of the object pixel, and for instructing said interpolating means to interpolate the object pixel or for interpolating the object pixel.

12. An image signal processing apparatus comprising:

first edge judgement means for judging from at least four pixels adjacent to an object pixel whether the object pixel constitutes a single edge;

first edge direction judging means for judging whether the single edge is horizontal or vertical, if said first edge judging means judges that the object pixel constitutes the single edge;

interpolating means for interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if said first edge direction judging means judges that the single edge is horizontal, and for interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if said first edge direction judging means judges that the single edge is vertical;

wherein the image signal processing apparatus sequentially interpolates each of a plurality of pixels in an image as the object pixel;

storage means for storing an edge direction of each of the plurality of object pixels, the edge direction being judged by said first edge direction judging means; and

re-judging means for re-judging that the object pixel has a different direction if a majority of the edge directions of pixels adjacent to the object pixel and stored in said storage means has an edge direction different from an edge direction of the object pixel, and for instructing said interpolating means to interpolate the object pixel or for interpolating the object pixel.

13. An image signal processing apparatus for sequentially interpolating each of a plurality of pixels in an image as the object pixel, comprising:

edge direction judging means for judging whether an edge constituted by the object pixel is horizontal or vertical;

storage means for storing an edge direction of each of the plurality of object pixels judged by said edge judging means;

edge direction re-judging means for re-judging that the object pixel has a different edge direction if a majority of the edge directions of pixels adjacent to the object pixel and stored in said storage means has an edge direction different from an edge direction of the object pixel; and

interpolating means for interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if said edge direction judging means or said edge direction re-judging means judges that the edge is horizontal, and for interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if said edge direction judging means or said edge direction re-judging means judges that the edge is vertical.

14. An image signal processing apparatus comprising:

chrominance signal acquiring means for acquiring at least green signals from some pixels of a one-dimensional pixel array and red or blue signals from other pixels;

green signal interpolating means for obtaining green signals of all pixels through interpolation between pixels of the pixel array;

first converting means for obtaining as a red color difference signal a difference between red and green signals at a same pixel in the pixel array and obtaining as a blue color difference signal a difference between blue and green signals at the same pixel;

color signal interpolating means for interpolating the red color difference signal and the blue color difference signal by using pixels of the pixel array, in accordance with the red color difference signal and the blue color difference signal converted by said first converting means; and

second converting means for obtaining the red signal and the blue signal by adding the green signal at the same pixel to the red color difference signal and the blue color difference signal interpolated by said color signal interpolating means.

15. An image signal processing apparatus according to claim 14, wherein said chrominance signal acquiring means acquires a chrominance signal of a two-dimensional pixel matrix, and said green signal interpolating means and said color difference signal interpolating means interpolate by using pixels of the two-dimensional pixel matrix.

16. An image signal processing apparatus according to claim 15, wherein said green signal interpolating means comprises:

edge judging means for judging whether an object pixel to be interpolated constitutes an edge;

edge direction judging means for judging whether a direction of the edge is vertical or horizontal, if said edge judging means judges that the object pixel constitutes the edge; and

interpolating means for interpolating the green signal in accordance with the direction of the edge, if said edge judging means judges that the object pixel constitutes the edge, and the image signal processing apparatus further comprises red/blue signal interpolating means for interpolating the red signal and the blue signal in accordance with the red signal and the blue signal obtained by said second converting means and the direction of the edge of the object pixel.

17. An image signal processing apparatus according to claim 16, wherein said red/blue signal interpolating means interpolates by using at least upper and lower two pixels adjacent to the object pixel, if the direction of the edge is vertical, and interpolates by using at least right and left two pixels adjacent to the object pixel, if the direction of the edge is horizontal.

18. A medium storing a program to be executed by a computer, the program comprising the steps of:

(a) judging from four pixels adjacent to an object pixel whether the object pixel constitutes a single edge;

(b) judging whether the single edge is horizontal or vertical, if it is judged that the object pixel constitutes the single edge;

(c) interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if it is judged that the single edge is horizontal, and interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if it is judged that the single edge is vertical;

(d) judging from four pixels adjacent to the object pixel whether the object pixel constitutes two edges, if said step (a) judges that the object pixel does not constitute the single edge; and

(e) judging whether the two edges are horizontal or vertical, if it is judged that the object pixel constitutes the two edges,

wherein said step (c) interpolates the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in the horizontal direction, if said step (e) judges that the two edges are horizontal, and interpolates the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in the vertical direction, if said step (e) judges that the two edges are vertical.

19. A medium storing a program according to claim 18, wherein said step (c) interpolates the object pixel in accordance with at least four pixels adjacent to the object pixel, if said step (d) judges that the object pixel does not constitutes the two edges.

20. A medium storing a program according to claim 18, wherein the program sequentially interpolates each of a plurality of pixels in an image as the object pixel and further comprises the step of:

(f) re-judging that the object pixel has a different direction if a majority of the edge directions of pixels adjacent to the object pixel and judged by said steps (b) and (e) has an edge direction different from an edge direction of the object pixel, and instructing to interpolate the object pixel at said step (c) or interpolating the object pixel.

21. A medium storing a program according to claim 18, wherein said step (a) judges whether the object pixel constitutes the single edge in accordance with a difference between values of right and left pixels adjacent to the object pixel in the horizontal direction and a difference between values of upper and lower pixels adjacent to the object pixel in the vertical direction.

22. A medium storing a program according to claim 18, wherein said step (b) judges an edge direction in accordance with a difference between values of right and left pixels adjacent to the object pixel in the horizontal direction and a difference between values of upper and lower pixels adjacent to the object pixel in the vertical direction.

23. A medium according to claim 22, wherein the judgment of an edge direction comprises determining an edge direction based on a relative value of an absolute value of the difference between values of the right and left pixels and an absolute value of the difference between values of the upper and lower pixels.

24. A medium storing a program according to claim 18, wherein the program sequentially interpolates each of a plurality of pixels in an image as the object pixel and said step (e) judges an edge direction in accordance with a value of a pixel shifted by one pixel in the horizontal direction and then in the vertical direction from the object pixel and interpolated by said step (c).

25. A medium storing a program according to claim 18, wherein said step (e) comprises the steps of:

(e-1) calculating a difference of values between a first column including the object pixel and upper and lower pixels adjacent to the object pixel in the vertical direction and a second column adjacent to the first column and disposed in the vertical direction;

(e-2) calculating a difference of values between a first row including the object pixel and right and left pixels adjacent to the object pixel in the horizontal direction and a second row adjacent to the first row and disposed in the horizontal direction; and

(e-3) judging an edge direction in accordance with the difference of values between the first and second columns and the difference of values between the first and second rows.

26. A medium storing a program according to claim 25, wherein said step (e-1) calculates a difference between a sum of values of at least first two pixels among the object pixel and upper and lower pixels adjacent to the object pixel in the first column and a sum of values of at least second two pixels adjacent to the first two pixels in the horizontal direction in the second column, and said step (e-2) calculates a difference between a sum of values of at least first two pixels among the object pixel and right and left pixels adjacent to the object pixel in the first row and a sum of values of at least second two pixels adjacent to the first two pixels in the vertical direction in the second row.

27. A medium storing a program according to claim 26, wherein said step (e-1) calculates a difference between a sum of values of first three pixels including the object pixel and upper and lower pixels adjacent to the object pixel in the first column and a sum of values of second three pixels adjacent to the first three pixels in the horizontal direction in the second column, and said step (e-2) calculates a difference between a sum of values of first three pixels including the object pixel and right and left pixels adjacent to the object pixel in the first row and a sum of values of second three pixels adjacent to the first three pixels in the vertical direction in the second row.

28. A medium storing a program according to claim 18, wherein aid step (d) judges whether the object pixel constitutes the two edges, in accordance with a sum of values of right and left pixels adjacent to the object pixel in the horizontal direction and a sum of values of upper and lower pixels adjacent to the object pixel in the vertical direction.

29. A medium storing a program to be executed by a computer, wherein the program sequentially interpolates each of a plurality of pixels in an image as the object pixel, the program comprising the steps of:

(a) judging from four pixels adjacent to an object pixel whether the object pixel constitutes a single edge;

(b) judging whether the single edge is horizontal or vertical, if it is judged that the object pixel constitutes the single edge;

(c) interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if it is judged that the single edge is horizontal, and interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if it is judged that the single edge is vertical;

(d) re-judging that the object pixel has a different direction if a majority of the edge directions of pixels adjacent to the object pixel and judged by said step (b) has an edge direction different from an edge direction of the object pixel, and instructing to interpolate the object pixel at said step (c) or interpolating the object pixel.

30. A medium storing a program for sequentially interpolating each of a plurality of pixels in an image as the object pixel, the program comprising the steps of:

(a) judging whether an edge constituted by the object pixel is horizontal or vertical;

(b) re-judging that the object pixel has a different edge direction if a majority of the edge directions of pixels adjacent to the object pixel and judged at said step (a) has an edge direction different from an edge direction of the object pixel; and

(c) interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if said step (a) or (b) judges that the edge is horizontal, and for interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if said step (a) or (b) judges that the edge is vertical.

31. A medium storing a program to be executed by a computer, the program comprising the steps of:

(a) acquiring at least green signals of some pixels of a one-dimensional pixel array and red or blue signals of other pixels;

(b) obtaining green signals of all pixels through interpolation between pixels of the pixel array;

(c) obtaining as a red color difference signal a difference between red and green signals of a same pixel in the pixel array and obtaining as a blue color difference signal a difference between blue and green signals of the same pixel;

(d) interpolating the red color difference signal and the blue color difference signal by using pixels of the pixel array, in accordance with the read color difference signal and the blue color difference signal converted by said step (c); and

(e) obtaining the red signal and the blue signal by adding the green signal of the same pixel to the red color difference signal and the blue color difference signal interpolated by said step (d).

32. A medium storing a program according to claim 31, wherein said step (a) acquires a chrominance signal of a two-dimensional pixel matrix, and said steps (b) and (d) interpolate by using pixels of the two-dimensional pixel matrix.

33. A medium storing a program according to claim 32, wherein said step (b) comprises the steps of:

(b-1) judging whether an object pixel to be interpolated constitutes an edge;

(b-2) judging whether a direction of the edge is vertical or horizontal, if it is judged that the object pixel constitutes the edge; and

(b-3) interpolating the green signal in accordance with the direction of the edge, if it is judged that the object pixel constitutes the edge, and the program further comprises the step of interpolating the red signal and the blue signal in accordance with the red signal and the blue signal obtained by said step (e).

34. A medium storing a program according to claim 33, wherein said step (f) interpolates by using at least upper and lower two pixels adjacent to the object pixel, if the direction of the edge is vertical, and interpolates by using at least right and left two pixels adjacent to the object pixel, if the direction of the edge is horizontal.