Image processing apparatus and method and information providing medium

• US 6,621,925 B1
• Filed: 06/28/1999
• Issued: 09/16/2003
• Est. Priority Date: 06/29/1998
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Habets, Hugo, Ohmori, Mutsuhiro
• Primary Examiner(s): Patel, Jayanti K.
• Assistant Examiner(s): AZARIAN, SEYED H
• Application Number:
  US09/340,420
• Time in Patent Office: 1,541 Days
• Field of Search: 382/167, 382/162, 382/233, 382/276, 382/285, 382/154, 382/292, 382/300, 358/519, 358/518, 358/523, 358/520, 358/506, 358/531, 345/430, 345/509, 345/521, 345/515, 345/582
• US Class Current: 382/167
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 15/40 Hidden part removal

Abstract

Instead of tracing using first rays of the ray tracing method, an object is divided into a plurality of triangles, each triangle is projected onto a screen, and data on a three-dimensional coordinate space are converted to data on a screen coordinate system. Pixel data inside a triangle are generated by interpolation based on data of the three vertices of the triangle on the screen coordinate system. Pixel data are corrected by performing a tracing operation using second rays on the pixel data generated by the interpolation.

Claims

1. An image processing apparatus which converts objects in a three-dimensional coordinate space to image data on a screen based on traditional ray tracing in which a first ray is traced from an eye point through a target pixel on a screen coordinate system to an object and the target pixel is subsequently corrected by calculating the influences of at least a second ray traced between the object and a second object independent of the eye point, said apparatus comprising:

substituting means for eliminating the need for the first ray tracing, the substituting means comprising projecting means for generating image data by projecting data representing surfaces of the object onto a screen coordinate system, and interpolating means for generating pixel data inside the surfaces on the screen coordinate system by interpolation based on the projected image data on the screen coordinate system; and

correcting means for generating a ray for a point on the object that corresponds to the pixel data generated by the interpolating means, and for generating correction information for correcting a value of the pixel data on the screen coordinate system based on a judgment of intersection of the ray with another object.

2. The image processing apparatus according to claim 1, wherein the correcting means generates correction information for correcting color information of the pixel data.

3. The image processing apparatus according to claim 2, wherein the correcting means generates the correction information for correcting the color information of the pixel data based on ambient light in the three-dimensional coordinate space.

4. The image processing apparatus according to claim 1, wherein the interpolating means interpolates at least one of a light vector, a reflection vector, a refraction vector, a normal vector, and position coordinates in the three-dimensional coordinate space.

5. The image processing apparatus according to claim 1, further comprising storing means for storing pixel data that are closest to an eye point based on depth coordinates of the pixel data interpolated by the interpolating means, wherein the correcting means generates the correction information for the pixel data stored in the storing means.

6. The image processing apparatus according to claim 5, wherein the storing means comprises depth storing means for storing depth coordinates of the pixel data corrected by the correcting means, and data storing means for storing the pixel data that are closest to the eye point.

7. The image processing apparatus according to claim 5, wherein the correcting means generates the correction information for the pixel data stored in the storing means after pixel data of one picture have been stored in the storing means.

8. The image processing apparatus according to claim 1, wherein the interpolating means performs an interpolation operation on the data representing the surfaces in order of their closeness to an eye point.

9. The image processing apparatus according to claim 8, wherein every time each pixel data is interpolated by the interpolating means the correcting means corrects the pixel data just interpolated.

10. The image processing apparatus according to claim 9, wherein the interpolating means judges whether the pixel just interpolated is closest to the eye point, and, if the pixel just interpolated is judged closest to the eye point, the correcting operation of the pixel is carried out in the correcting means.

11. The image processing apparatus according to claim 9, wherein the interpolating means judges whether the pixel just interpolated is closest to the eye point, and, if the pixel just interpolated is judged not closest to the eye point, the correcting operation of the pixel is not carried out in the correcting means.

12. The image processing apparatus according to claim 8, further comprising sorting means for sorting the data representing the surfaces of the object based on depth coordinates that represent the surfaces of the object, wherein the interpolating means performs an interpolation operation on the data representing the surfaces in order of their closeness to the eye point as judged by the sorting means.

13. The image processing apparatus according to claim 8, wherein the interpolating means interpolates a reflection vector or a refraction vector at each pixel of the surfaces, and wherein the correcting operation of the pixel is not carried out if a value of the interpolated reflection vector or refraction vector at the pixel is smaller than a prescribed reference value.

14. The image processing apparatus according to claim 1, wherein if transparency of surfaces of object is greater than a prescribed reference value, the correcting means performs operations for higher order rays until a ray strikes the object whose transparency is smaller than the reference value.

15. The image processing apparatus according to claim 1, wherein if there exists a surface of an object that is closer to an eye point than a surface of a target object is, the interpolating means and the correcting means skip operations for the surface of the target object.

16. The image processing apparatus according to claim 1, wherein in a correcting operation for a shadow ray, a reflection ray, or a refraction ray, the correcting means first performs a judgment of a strike on the object that was struck by a ray originating from a pixel immediately preceding a pixel currently processed.

17. The image processing apparatus according to claim 16, wherein for second and following pixels in a span of the surface of the object, the correcting means first performs the judgment of a strike on the object that was struck by the ray originating from the pixel immediately preceding the pixel currently processed.

18. The image processing apparatus according to claim 16, wherein if the judgment of a strike on the object that was struck by the ray originating from the immediately preceding pixel produces a judgment result to the effect that a strike occurs, the correcting means does not perform subsequent intersection judgments in the correcting operation for a shadow ray, a reflection ray, or a refraction ray.

19. The image processing apparatus according to claim 1, wherein the correcting means performs a correcting operation for a shadow ray to a third ray and performs a correcting operation for a reflection ray or a refraction ray to a second ray, and wherein correcting operations using rays of the correcting means are started from a second ray.

20. The image processing apparatus according to claim 1, further comprising rendering means for determining color information of the pixel data based on correction values generated by the correcting means, and for recording the pixel data in a rendering buffer.

21. The image processing apparatus according to claim 20, wherein if transparency of surfaces of object is greater than a prescribed reference value, the rendering means blends a color of new pixels with a color of pixels that have already been rendered while rendering the pixel data in order of distance from an eye point.

22. The image processing apparatus according to claim 21, wherein the correcting means does not perform a correcting operation for a refraction ray.

23. The image processing apparatus according to claim 20, wherein if a plurality of objects correspond to one pixel, the rendering means weights the plurality of objects based on areas of the pixel covered by the respective objects.

24. The image processing apparatus according to claim 1, wherein the interpolating means interpolates pixel data in order of distance from an eye point in a depth direction of the screen coordinate system.

25. The image processing apparatus according to claim 1, wherein the data representing the surfaces of the object in the three-dimensional coordinate space are data representing triangles.

26. An image processing method for converting objects in a three-dimensional coordinate space to image data on a screen, said method based on traditional ray tracing in which a first ray is traced from an eye point through a target pixel on a screen coordinate system to an object and the target pixel is subsequently corrected by calculating the influences of at least a second ray traced between the object and a second object independent of the eye point, said method comprising:

substituting the first ray tracing with projection and interpolation steps wherein the projection step comprises generating image data by projecting data representing surfaces of the object onto a screen coordinate system, and the interpolating step comprises generating pixel data inside the surfaces on the screen coordinate system by interpolation based on the projected image data on the screen coordinate system; and

a correcting step of generating a ray for a point on the object that corresponds to the pixel data generated by the interpolating step, and for generating correction information for correcting a value of the pixel data on the screen coordinate system based on a judgment of intersection of the ray with another object.

27. The image processing method according to claim 26, wherein the correcting step generates correction information for correcting color information of the pixel data.

28. The image processing method according to claim 27, wherein the correcting step generates the correction information for correcting the color information of the pixel data based on ambient light in the three-dimensional coordinate space.

29. The image processing method according to claim 26, wherein the interpolating step interpolates at least one of a light vector, a reflection vector, a refraction vector, a normal vector, and position coordinates in the three-dimensional coordinate space.

30. The image processing method according to claim 26, further comprising a storing step of storing pixel data that are closest to an eye point based on depth coordinates of the pixel data interpolated by the interpolating step, wherein the correcting step generates the correction information for the pixel data stored by the storing step.

31. The image processing method according to claim 30, wherein the storing step comprises a depth storing step of storing depth coordinates of the pixel data corrected by the correcting step, and a data storing step of storing the pixel data that are closest to the eye point.

32. The image processing method according to claim 30, wherein the correcting step generates the correction information for the pixel data stored by the storing step after pixel data of one picture have been stored by the storing step.

33. The image processing method according to claim 26, wherein the interpolating step performs an interpolation operation on the data representing the surfaces in order of their closeness to an eye point.

34. The image processing method according to claim 33, wherein every time each pixel data is interpolated by the interpolating step the correcting step corrects the pixel data just interpolated.

35. The image processing method according to claim 34, wherein the interpolating step judges whether the pixel just interpolated is closest to the eye point, and, if the pixel just interpolated is judged closest to the eye point, the correcting operation of the pixel is carried out in the correcting step.

36. The image processing method according to claim 34, wherein the interpolating step judges whether the pixel just interpolated is closest to the eye point, and, if the pixel just interpolated is judged not closest to the eye point, the correcting operation of the pixel is not carried out in the correcting step.

37. The image processing method according to claim 33, further comprising a sorting step of sorting the data representing the surfaces of the object based on depth coordinates that represent the surfaces of the object, wherein the interpolating step performs an interpolation operation on the data representing the surfaces in order of their closeness to the eye point as judged by the sorting step.

38. The image processing method according to claim 33, wherein the interpolating step interpolates a reflection vector or a refraction vector at each pixel of the surfaces, and wherein the correcting operation is not carried out if a value of the interpolated reflection vector or refraction vector at the pixel is smaller than a prescribed reference value.

39. The image processing method according to claim 26, wherein if transparency of surfaces of object is greater than a prescribed reference value, the correcting step performs operations for higher order rays until a ray strikes the object whose transparency is smaller than the reference value.

40. The image processing method according to claim 26, wherein if there exists a surface of an object that is closer to an eye point than a surface of a target object is, the interpolating step and the correcting step skip operations for the surface of the target object.

41. The image processing method according to claim 26, wherein in a correcting operation for a shadow ray, a reflection ray, or a refraction ray, the correcting step first performs a judgment of a strike on the object that was struck by a ray originating from a pixel immediately preceding a pixel currently processed.

42. The image processing method according to claim 41, wherein for second and following pixels in a span of the surface of the object, the correcting step first performs the judgment of a strike on the object that was struck by the ray originating from the pixel immediately preceding the pixel currently processed.

43. The image processing method according to claim 41, wherein if the judgment of a strike on the object that was struck by the ray originating from the immediately preceding pixel produces a judgment result to the effect that a strike occurs, the correcting step does not perform subsequent intersection judgments in the correcting operation for a shadow ray, a reflection ray, or a refraction ray.

44. The image processing method according to claim 26, wherein the correcting step performs a correcting operation for a shadow ray to a third ray and performs a correcting operation for a reflection ray or a refraction ray to a second ray, and wherein correcting operations using rays of the correcting step are started from a second ray.

45. The image processing method according to claim 26, further comprising a rendering step of determining color information of the pixel data based on correction values generated by the correcting step, and for recording the pixel data in a rendering buffer.

46. The image processing method according to claim 45, wherein if transparency of surfaces of object is greater than a prescribed reference value, the rendering step blends a color of new pixels with a color of pixels that have already been rendered while rendering the pixel data in order of distance from an eye point.

47. The image processing method according to claim 46, wherein the correcting step does not perform a correcting operation for a refraction ray.

48. The image processing method according to claim 45, wherein if a plurality of objects correspond to one pixel, the rendering step weights the plurality of objects based on areas of the pixel covered by the respective objects.

49. The image processing method according to claim 26, wherein the interpolating step interpolates pixel data in order of distance from an eye point in a depth direction of the screen coordinate system.

50. The image processing method according to claim 26, wherein the data representing the surfaces of the object in the three-dimensional coordinate space are data representing triangles.

51. An information providing medium for providing a program that causes an image processing apparatus to execute a process of converting objects in a three-dimensional coordinate space to image data on a screen, said process based on traditional ray tracing in which a first ray is traced from an eye point through a target pixel on a screen coordinate system to an object and the target pixel is subsequently corrected by calculating the influences of at least a second ray traced between the object and a second object independent of the eye point, the process comprising:

substituting the first ray tracing with projection and interpolation steps wherein the projection step comprises generating image data by projecting data representing surfaces of the object onto a screen coordinate system; and

the interpolating step comprises generating pixel data inside the surfaces on the screen coordinate system by interpolation based on the projected image data on the screen coordinate system; and

a correcting step of generating a ray for a point on the object that corresponds to the pixel data generated by the interpolating step, and for generating correction information for correcting a value of the pixel data on the screen coordinate system based on a judgment of intersection of the ray with another object.