Image processing system, program, information storage medium, and image processing method

• US 7,295,699 B2
• Filed: 05/18/2004
• Issued: 11/13/2007
• Est. Priority Date: 05/20/2003
• Status: Active Grant

• Current Assignee: Bandai Namco Entertainment Inc. (BANDAI NAMCO Holdings Inc.)
• Original Assignee: Namco Bandai Games Inc. (BANDAI NAMCO Holdings Inc.)
• Inventors: Ohba, Yasuo, Yamaguchi, Kentaro
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US10/847,447
• Publication Number:
  US 20050036673A1
• Time in Patent Office: 1,274 Days
• Field of Search: 382/154, 382/285, 345419-427, 356 12- 14, 348 42- 60, 359462-477, 352 57- 65, 33/204, 353 7- 9, 378 41- 42, 396324-331
• US Class Current: 382/154
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/334 using spectral multiplexing

Abstract

An original image area IMR is divided into divided image areas DR1 to DR4, texture coordinates specified by division point coordinates are obtained, and images of the divided image areas DR1 to DR4 are mapped onto polygons PL1 to PL4, based on the obtained texture coordinates. The original image area IMR is divided in such a manner that division spacing in a direction DIR1 towards a vanishing point VP1 from the original image becomes narrower as the divided image areas are closer to the vanishing point VP1 and texture coordinates are obtained, to implement correction processing for removing perspective of the original image. Correction processing is performed to remove perspective of images at a base surface of images IL1 and IR1 for the left eye and the right eye, the images IL2 and IR2 for the left eye and the right eye are generated, and a stereoscopic image is generated by synthesizing (by anaglyph processing) IL2 and IR2.

Claims

1. An image processing system for performing correction processing on an original image, the image processing system comprising:

a texture coordinates calculation section which divides an area of the original image into first to kth divided image areas (K is an integer greater than or equal to

     2) in a mesh form, and obtains texture coordinates specified by coordinates of division points of the first to kth divided image areas; and

a texture mapping section which maps images of the first to kth divided image areas onto first to kth polygons based on the obtained texture coordinates, the first to Kth polygons being disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas,

wherein the texture coordinates calculation section obtains texture coordinates by dividing the area of the original image into the first to kth divided image areas in such a manner that division spacing in a direction from the original image toward a vanishing point of perspective of the original image becomes narrower as the divided image areas are closer to the vanishing point.

2. The image processing system as defined in claim 1,

wherein the texture coordinates calculation section:

obtains coordinates of a first vanishing point which is an intersection between a line extending from a fourth edge of a quadrilateral of which vertices are first to fourth reference points that are set within the area of the original image, and a line extending from a second edge of the quadrilateral which is opposite to the fourth edge;

obtains coordinates of the first reference point and coordinates of the fourth reference point in an X-Y coordinate system having the first vanishing point as the origin;

performs linear interpolation between reciprocals of coordinate values of the first reference point and reciprocals of coordinate values of the fourth reference point, and obtains coordinates of division points on the fourth edge which links the first and fourth reference points, based on reciprocals of values obtained by the linear interpolation;

obtains coordinates of the second reference point and coordinates of the third reference point in the X-Y coordinate system having the first vanishing point as the origin;

performs linear interpolation between reciprocals of coordinate values of the second reference point and reciprocals of coordinate values of the third reference point, and obtains coordinates of division points on the second edge which links the second and third reference points, based on reciprocals of values obtained by the linear interpolation; and

obtains coordinates of division points on edges linking the division points on the fourth and second edges, based on the coordinates of the division points on the fourth edge and the coordinates of the division points on the second edge.

3. The image processing system as defined in claim 2, wherein the texture coordinates calculation section further:

obtains coordinates of new first to fourth reference points by extrapolation processing on the first to fourth reference points, and obtains coordinates of division points based on the obtained coordinates of the new first to fourth reference points.

4. The image processing system as defined in claim 1 further comprising:

a fetch section which fetches a first image for the left eye and a first image for the right eye for stereoscopic viewing; and

a stereoscopic image generation section which generates a stereoscopic image,

wherein the texture coordinates calculation section;

obtains texture coordinates by dividing an area of the first image for the left eye into first to kth divided image areas for the left eye in such a manner that division spacing in the direction from the first image for the left eye to a vanishing point of perspective of the first image for the left eye becomes narrower as the divided image areas are closer to the vanishing point; and

obtains texture coordinates by dividing an area of the first image for the right eye into first to kth divided image areas for the right eye in such a manner that division spacing in the direction from the first image for the right eye to a vanishing point of perspective of the first image for the right eye becomes narrower as the divided image areas are closer to the vanishing point,

wherein the texture mapping section;

maps images for the first to kth divided image areas for the left eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the left eye, to generate a second image for the left eye which has been subjected to correction processing to remove perspective; and

maps images for the first to kth divided image areas for the right eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the right eye, to generate a second image for the right eye which has been subjected to correction processing to remove perspective, and

wherein the stereoscopic image generation section generates a stereoscopic image, based on the second image for the left eye and the second image for the right eye.

5. The image processing system as defined in claim 2,

a fetch section which fetches a first image for the left eye and a first image for the right eye for stereoscopic viewing;

and

a stereoscopic image generation section which generates a stereoscopic image,

wherein the texture coordinates calculation section;

obtains texture coordinates by dividing an area of the first image for the left eye into first to kth divided image areas for the left eye in such a manner that division spacing in the direction from the first image for the left eye to a vanishing point of perspective of the first image for the left eye becomes narrower as the divided image areas are closer to the vanishing point; and

obtains texture coordinates by dividing an area of the first image for the right eye into first to kth divided image areas for the right eye in such a manner that division spacing in the direction from the first image for the right eye to a vanishing point of perspective of the first image for the right eye becomes narrower as the divided image areas are closer to the vanishing point,

wherein the texture mapping section;

maps images for the first to kth divided image areas for the left eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the left eye, to generate a second image for the left eye which has been subjected to correction processing to remove perspective; and

maps images for the first to kth divided image areas for the right eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the right eye, to generate a second image for the right eye which has been subjected to correction processing to remove perspective, and

wherein the stereoscopic image generation section generates a stereoscopic image, based on the second image for the left eye and the second image for the right eye.

6. The image processing system as defined in claim 4,

wherein a stereoscopic image is generated by using anaglyph processing to synthesize the second image for the left eye and the second image for the right eye.

7. The image processing system as defined in claim 5,

wherein a stereoscopic image is generated by using anaglyph processing to synthesize the second image for the left eye and the second image for the right eye.

8. A computer-readable information storage medium storing a program for image processing, the program causing a computer to implement processing of:

dividing an area of the original image into first to kth divided image areas (K is an integer greater than or equal to

     2) in a mesh form, and obtaining texture coordinates specified by coordinates of division points of the first to kth divided image areas;

mapping images of the first to kth divided image areas onto first to kth polygons based on the obtained texture coordinates, the first to Kth polygons being disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas; and

obtaining texture coordinates by dividing the area of the original image into the first to kth divided image areas in such a manner that division spacing in a direction from the original image toward a vanishing point of perspective of the original image becomes narrower as the divided image areas are closer to the vanishing point.

9. The computer-readable information storage medium as defined in claim 8, the program further causing a computer to implement processing of:

obtaining coordinates of a first vanishing point which is an intersection between a line extending from a fourth edge of a quadrilateral of which vertices are first to fourth reference points that are set within the area of the original image, and a line extending from a second edge of the quadrilateral which is opposite to the fourth edge;

obtaining coordinates of the first reference point and coordinates of the fourth reference point in an X-Y coordinate system having the first vanishing point as the origin;

performing linear interpolation between reciprocals of coordinate values of the first reference point and reciprocals of coordinate values of the fourth reference point, and obtaining coordinates of division points on the fourth edge which links the first and fourth reference points, based on reciprocals of values obtained by the linear interpolation;

obtaining coordinates of the second reference point and coordinates of the third reference point in the X-Y coordinate system having the first vanishing point as the origin;

performing linear interpolation between reciprocals of coordinate values of the second reference point and reciprocals of coordinate values of the third reference point, and obtaining coordinates of division points on the second edge which links the second and third reference points, based on reciprocals of values obtained by the linear interpolation; and

obtaining coordinates of division points on edges linking the division points on the fourth and second edges, based on the coordinates of the division points on the fourth edge and the coordinates of the division points on the second edge.

10. The computer-readable information storage medium as defined in claim 9, the program further causing a computer to implement processing of:

obtaining coordinates of new first to fourth reference points by extrapolation processing on the first to fourth reference points, and obtains coordinates of division points based on the obtained coordinates of the new first to fourth reference points.

11. The computer-readable information storage medium as defined in claim 8, the program further causing a computer to implement processing of:

fetching a first image for the left eye and a first image for the right eye for stereoscopic viewing;

obtaining texture coordinates by dividing an area of the first image for the left eye into first to kth divided image areas for the left eye in such a manner that division spacing in the direction from the first image for the left eye to a vanishing point of perspective of the first image for the left eye becomes narrower as the divided image areas are closer to the vanishing point;

obtaining texture coordinates by dividing an area of the first image for the right eye into first to kth divided image areas for the right eye in such a manner that division spacing in the direction from the first image for the right eye to a vanishing point of perspective of the first image for the right eye becomes narrower as the divided image areas are closer to the vanishing point;

mapping images for the first to kth divided image areas for the left eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the left eye, to generate a second image for the left eye which has been subjected to correction processing to remove perspective; and

mapping images for the first to kth divided image areas for the right eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the right eye, to generate a second image for the right eye which has been subjected to correction processing to remove perspective; and

generating a stereoscopic image, based on the second image for the left eye and the second image for the right eye.

12. The computer-readable information storage medium as defined in claim 9, the program further causing a computer to implement processing of:

fetching a first image for the left eye and a first image for the right eye for stereoscopic viewing;

obtaining texture coordinates by dividing an area of the first image for the left eye into first to kth divided image areas for the left eye in such a manner that division spacing in the direction from the first image for the left eye to a vanishing point of perspective of the first image for the left eye becomes narrower as the divided image areas are closer to the vanishing point;

obtaining texture coordinates by dividing an area of the first image for the right eye into first to kth divided image areas for the right eye in such a manner that division spacing in the direction from the first image for the right eye to a vanishing point of perspective of the first image for the right eye becomes narrower as the divided image areas are closer to the vanishing point;

mapping images for the first to kth divided image areas for the left eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the left eye, to generate a second image for the left eye which has been subjected to correction processing to remove perspective; and

mapping images for the first to kth divided image areas for the right eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the right eye, to generate a second image for the right eye which has been subjected to correction processing to remove perspective; and

generating a stereoscopic image, based on the second image for the left eye and the second image for the right eye.

13. The computer-readable information storage medium as defined in claim 11, the program further causing a computer to implement processing of:

generating a stereoscopic image by using anaglyph processing to synthesize the second image for the left eye and the second image for the right eye.

14. The computer-readable information storage medium as defined in claim 12, the program further causing a computer to implement processing of:

generating a stereoscopic image by using anaglyph processing to synthesize the second image for the left eye and the second image for the right eye.

15. A method for image processing comprising:

dividing an area of the original image into first to kth divided image areas (K is an integer greater than or equal to

     2) in a mesh form, and obtaining texture coordinates specified by coordinates of division points of the first to kth divided image areas;

mapping images of the first to kth divided image areas onto first to kth polygons based on the obtained texture coordinates, the first to Kth polygons being disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas; and

obtaining texture coordinates by dividing the area of the original image into the first to kth divided image areas in such a manner that division spacing in a direction from the original image toward a vanishing point of perspective of the original image becomes narrower as the divided image areas are closer to the vanishing point.

16. The method as defined in claim 15,

obtaining coordinates of a first vanishing point which is an intersection between a line extending from a fourth edge of a quadrilateral of which vertices are first to fourth reference points that are set within the area of the original image, and a line extending from a second edge of the quadrilateral which is opposite to the fourth edge;

obtaining coordinates of the first reference point and coordinates of the fourth reference point in an X-Y coordinate system having the first vanishing point as the origin;

performing linear interpolation between reciprocals of coordinate values of the first reference point and reciprocals of coordinate values of the fourth reference point, and obtaining coordinates of division points on the fourth edge which links the first and fourth reference points, based on reciprocals of values obtained by the linear interpolation;

obtaining coordinates of the second reference point and coordinates of the third reference point in the X-Y coordinate system having the first vanishing point as the origin;

performing linear interpolation between reciprocals of coordinate values of the second reference point and reciprocals of coordinate values of the third reference point, and obtaining coordinates of division points on the second edge which links the second and third reference points, based on reciprocals of values obtained by the linear interpolation; and

obtaining coordinates of division points on edges linking the division points on the fourth and second edges, based on the coordinates of the division points on the fourth edge and the coordinates of the division points on the second edge.

17. The method as defined in claim 15,

fetching a first image for the left eye and a first image for the right eye for stereoscopic viewing;

obtaining texture coordinates by dividing an area of the first image for the left eye into first to kth divided image areas for the left eye in such a manner that division spacing in the direction from the first image for the left eye to a vanishing point of perspective of the first image for the left eye becomes narrower as the divided image areas are closer to the vanishing point;

obtaining texture coordinates by dividing an area of the first image for the right eye into first to kth divided image areas for the right eye in such a manner that division spacing in the direction from the first image for the right eye to a vanishing point of perspective of the first image for the right eye becomes narrower as the divided image areas are closer to the vanishing point;

mapping images for the first to kth divided image areas for the left eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the left eye, to generate a second image for the left eye which has been subjected to correction processing to remove perspective; and

mapping images for the first to kth divided image areas for the right eye onto first to kth polygons which are disposed in a mesh form in a one-to-one relationship with the first to kth divided image areas for the right eye, to generate a second image for the right eye which has been subjected to correction processing to remove perspective; and

generating a stereoscopic image, based on the second image for the left eye and the second image for the right eye.