Method and apparatus for generating confidence data

• US 20030001837A1
• Filed: 05/20/2002
• Published: 01/02/2003
• Est. Priority Date: 05/18/2001
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Baumberg, Adam Michael
• Granted Patent: US 7,006,089 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 15/20 Perspective computation

Abstract

An image processing apparatus (2) is disclosed in which input images are processed to generate texture map data for texture rendering a generated three-dimensional computer model of object(s) appearing in the images. In order to select the portions of the images utilized, confidence data is generated indicative of the extent portions of the surface of a model are visible in each of the images. The images are then combined utilizing this confidence data, where image data representative of different spatial frequencies are blended in different ways utilizing the confidence data.

Claims

1. A method of generating visibility data indicative of the relative visibility of points for at least part of the surface of an object from a viewpoint, said method comprising the steps of:

receiving model data defining a plurality of polygons indicative of at least part of the surface of an object for which visibility data is to be determined;

generating a representation of said object as viewed from said viewpoint wherein the surface of said object viewed from said viewpoint in said representation is rendered utilizing a determined visibility for the corresponding portion of said object as seen from said viewpoint; and

generating visibility data utilizing said generated representation.

2. A method in accordance with claim 1, wherein said generation of visibility data comprises utilizing said representation to generate visibility data for portions of the surface of an object visible from said viewpoint and associating portions of the surface of an object not visible from said viewpoint with data indicative of low visibility.

3. A method in accordance with claim 1, wherein said generation of a representation comprises generation of a representation in which visible portions of an object viewed from said viewpoint adjacent to portions of said object the surface of which is not visible from said viewpoint are associated with data representative of a low level of visibility.

4. A method in accordance with claim 1, wherein said generation of a representation comprises for each polygon of said model of said object visible from said viewpoint, determining a visibility score representative of the relative visibility of said polygon viewed from said viewpoint, wherein said determined visibility scores are utilized to generate said representation of said object from said viewpoint.

5. A method in accordance with claim 4, wherein said determination of a visibility score comprises determining a visibility score proportional to the ratio of the number of pixels utilized to represent said polygon in an image of said object viewed from said viewpoint relative to the surface area of said polygon represents.

6. A method in accordance with claim 1, wherein said generation of said representation comprises the steps of:

determining for polygons of an object as viewed from said viewpoint whether any edges of said polygons are adjacent to polygons oriented away from said viewpoint and associating said edges of said visible polygons adjacent to polygons orientated away from said viewpoint with data indicative of low visibility.

7. A method in accordance with claim 2, wherein said generation of a representation comprises generation of a representation in which data representative of the visibility of an object varies smoothly across the visible surface of an object as viewed from said viewpoint.

8. A method in accordance with claim 7, wherein said generation of a representation comprises the steps of generating an initial representation in which polygons visible from said viewpoint are uniformly represented by data indicative of the relative visibility of said polygons and generating said representation by performing a smoothing operation on said initial image.

9. A method in accordance with claim 1, wherein said generation of visibility data comprises:

associating polygons not visible from said viewpoint with data indicative of low visibility;

for polygons completely visible from said viewpoint, utilizing said representation to determine visibility data for the surface of said polygons;

and for polygons partially visible from said viewpoint, utilizing said representation to determine visibility data for said visible portions; and

associating the remaining portions of said partially visible polygons with data representative of low visibility.

10. A method of generating data indicative of the reliability of data of an object from a viewpoint to generate corresponding data for another viewpoint comprising the steps of:

generating visibility data for said viewpoint in accordance with claim 1;

and

determining a projection of said visibility data associated with points on the surface of said object as perceived from said another viewpoint.

11. A method of utilising image data of an object from a plurality of viewpoints to generate data indicative of a view of said object of from another viewpoint comprising the steps of:

receiving image data of an object from a plurality of viewpoints;

determining for each of said plurality of viewpoints data indicative of reliability of data from said viewpoint to generate corresponding data for said another viewpoint in accordance with claim 10;

determining for each of said plurality of viewpoints, representations of said object as viewed from said another viewpoint utilising said image data for said viewpoints; and

generating image data for said another viewpoint by processing portions of said projected image data utilising said data indicative of reliability.

12. A method in accordance with claim 11 wherein said processing comprises the step of selecting portions of representations associated with reliability data indicative of high reliability of data to generate said image data for said another viewpoint.

13. A method in accordance with claim 12 wherein said processing comprises the step of determining image data for each portion of an image for said another viewpoint by calculating weighted averages of corresponding portions of said representations weighted by reliability data indicative of reliability for portion of said representations.

15. An apparatus in accordance with claim 14, wherein visibility generation unit is operable to utilize representation generated by said view generation unit to generate visibility data for portions of the surface of an object visible from said viewpoint and to associate portions of the surface of an object not visible from said viewpoint with data indicative of low visibility.

16. An apparatus in accordance with claim 14, wherein said view generation unit comprises a representation unit operable to generate a representation in which visible portions of an object viewed from a viewpoint adjacent to portions of a said object the surface of which is not visible from said viewpoint are associated with data representative of a low level of visibility.

17. An apparatus in accordance with claim 14, wherein said representation unit is operable to generate a representation in which each polygon of said object visible from said viewpoint is represented utilizing data determined on the basis of a visibility score representative of the relative visibility of said polygon viewed from said viewpoint.

18. An apparatus in accordance with claim 17, wherein said representation unit comprises:

a visibility score determinator operable to determine visibility scores proportional to the ratio of the number of pixels utilized to represent a said polygon in an image of said object viewed from said viewpoint relative to the surface area of said polygon represents.

19. An apparatus in accordance with claim 14, wherein said view generation unit comprises:

an edge determinator operable to determine for polygons of an object as viewed from a viewpoint whether any edges of said polygons are adjacent to polygons oriented away from said viewpoint and to associate said edges of said visible polygons adjacent to polygons pointed away from said viewpoint with data indicative of low visibility.

20. An apparatus in accordance with claim 15, wherein said view generation unit is operable to generate representations in which data representative of the visibility of an object varies smoothly across the visible surface of an object as viewed from a said viewpoint.

21. An apparatus in accordance with claim 20, wherein said view generation unit comprises:

a first image generator operable to generate an initial representation in which polygons visible from said viewpoint are uniformly represented by data indicative of the relative visibility of said polygons; and

a second image generator operable to generate said representation by performing a smoothing operation on an initial image generated by said first image generator.

22. An apparatus in accordance with claim 14, wherein said visibility generation unit comprises:

a first association unit operable to associate polygons not visible from said viewpoint with data indicative of low visibility;

a second association unit operable to associate polygons completely visible from said viewpoint, with visibility data utilizing a said representation generated by said view generation unit to determine visibility data for the surface of said polygons; and

a third association unit operable to associate polygons partially visible from said viewpoint with visibility data, utilizing a said representation generated by said view generation unit to determine visibility data for said visible portions; and

associating the remaining portions of said partially visible polygons with data representative of low visibility.

25. An apparatus in accordance with claim 24 wherein said image generator is operable to process image data by selecting portions of representations associated with reliability data indicative of high reliability of data by said generated reliability data to generate said image data for said another viewpoint.

26. An apparatus in accordance with claim 25 wherein said image generator is operable to process image data by determining image data for each portion of an image for said another viewpoint by calculating weighted averages of corresponding portions of said representations weighted by reliability data generated by said apparatus for generating data indicative of reliability.

27. A recording medium for storing computer implementable process steps for generating within a programmable computer apparatus in accordance with any of claims 14 to 26 or for causing a programmable computer to perform a method in accordance with any claims 1 to 13.

28. A recording medium in accordance with claim 27 comprising a computer disc.

29. A computer disc in accordance with claim 28 comprising an optical, magneto-optical or magnetic disc.

30. A recording medium in accordance with claim 27, comprising an electrical signal transferred via the Internet.

14. Apparatus for generating visibility data indicative of the relative visibility of points for at least part of the surface of an object from a viewpoint, said apparatus comprising:

a receiver operable to receive model data defining a plurality of polygons indicative of at least part of the surface of an object for which visibility data is to be determined;

a view generation unit operable to generate utilizing model data received by said receiver, a representation of an object as viewed from a viewpoint wherein the surface of said object viewed from said viewpoint in said representation is rendered utilizing a determined visibility for the corresponding portion of said object as seen from said viewpoint; and

a visibility data generation unit operable to generate visibility data utilizing representations generated by said view generation unit.

23. An apparatus for generating data indicative of the reliability of data of an object from a viewpoint to generate corresponding data for another viewpoint comprising:

apparatus for generating visibility data for said viewpoint in accordance with claim 14; and

a projector operable to determine a projection of said visibility data associated with points on the surface of said object as perceived from said another viewpoint.

24. An apparatus for utilising image data of an object from a plurality of viewpoints to generate data indicative of a view of said object of from another viewpoint comprising:

an image receiver operable to receive image data of an object from a plurality of viewpoints;

apparatus for generating data indicative of reliability of data from a viewpoint to generate corresponding data for said another viewpoint in accordance with claim 23;

a view generator operable to determine for each of said plurality of viewpoints, projected image data for representations of said object as viewed from said another viewpoint utilising image data for said viewpoints received by said image receiver; and

an image generator operable to generate image data for said another viewpoint by processing portions of said projected image data generated by said view generator utilising data indicative of reliability generated by said apparatus for generating data indicative of reliability.