Image processing apparatus

• US 7,508,977 B2
• Filed: 03/08/2004
• Issued: 03/24/2009
• Est. Priority Date: 01/20/2000
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Haslam, Jane, Lyons, Alexander Ralph, Taylor, Richard Ian, Rowe, Simon Michael
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US10/793,850
• Publication Number:
  US 20040247174A1
• Time in Patent Office: 1,842 Days
• Field of Search: 382/154, 382/285, 382/305, 345419-427, 345/700, 356 12- 14, 348 42- 60, 348/850, 359462-477, 352 57- 65, 33/204, 353 7- 9, 378 41- 42, 396324-331, 707/104.1
• US Class Current: 382/154
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

A method of generating model data representative of a model in a three-dimensional space of an object from input signals representative of a set of images of the object taken from a plurality of respective camera positions includes the steps of displaying a model image derived from the model data and having a plurality of primitives for viewing by a user, receiving at least one primitive selection signal responsive to user actuation of an input whereby each primitive selection signal identifies a respective selected primitive of the model, and defining a plurality of virtual cameras in the three dimensional space having positions and look directions relative to the model which correspond substantially to those of the respective actual cameras relative to the object. Additional steps include evaluating which of the virtual cameras is an optimum virtual camera for generating a view of the selected primitives, identifying from the camera images a first camera image of the plurality of camera images taken from a camera position corresponding to that of the optimum virtual camera, and determining from the camera images a second camera image as being suitable for matching features in the first camera image and displaying the second camera image for comparison by the user with the first camera image.

Claims

1. A method of generating model data representative of a model in a three dimensional space of an object from input signals representative of a set of images of the object taken from a plurality of respective camera positions, the method comprising the steps of:

using a computer to perform the following steps;

displaying a model image derived from the model data and comprising a plurality of primitives for viewing by a user;

receiving at least one primitive selection signal responsive to user actuation of an input means whereby each primitive selection signal identifies a respective selected primitive of the model;

defining a plurality of virtual cameras in the three dimensional space having positions and look directions relative to the model which correspond substantially to those of the respective actual cameras relative to the object;

evaluating which of the virtual cameras is an optimum virtual camera for generating a view of the selected primitives;

identifying from the camera images a first camera image of the plurality of camera images taken from a camera position corresponding to that of the optimum virtual camera, and

determining from the camera images a second camera image as being suitable for matching features in the first camera image and displaying the second camera image for comparison by the user with the first camera image.

2. A method as claimed in claim 1, wherein the second camera image is taken from a camera position proximate to the optimum camera position.

3. A method as claimed in claim 1, further comprising the step of receiving feature matching selection signals representative of user matched points in the first and second camera images.

4. A method as claimed in claim 3, further comprising the step of generating updated model data to include additional detail corresponding to the received feature matching selection signals rendering the updated model data to generate an updated model image and displaying the updated model image.

9. A method as claimed in claim 1, wherein the input means is a pointing means co-operable with a display means to provide input signals in the form of image co-ordinates of the displayed image.

10. A method as claimed in claim 1, further comprising the step of generating the displayed model image by rendering the image data.

5. A method of generating model data representative of a model in a three dimensional space of an object from input signals representative of a set of images of the object taken from a plurality of respective camera positions, the method comprising the steps of:

using a computer to perform the following steps;

displaying a model image derived from the model data and comprising a plurality of primitives for viewing by a user;

receiving at least one primitive selection signal responsive to user actuation of an input means whereby each primitive selection signal identifies a respective selected primitive of the model;

defining a plurality of virtual cameras in the three dimensional space having positions and look directions relative to the model which correspond substantially to those of the respective actual cameras relative to the object;

evaluating which of the virtual cameras is an optimum virtual camera for generating a view of the selected primitives; and

identifying from the camera images a first camera image of the plurality of camera images taken from a camera position corresponding to that of the optimum virtual camera,

wherein the evaluating step comprises;

calculating for a selected primitive an aspect measurement representative of the visibility of the primitive when viewed in projection in the look direction of one of the virtual cameras;

repeating the calculating step to obtain a respective aspect measurement for each of the virtual cameras;

comparing the aspect measurements for the selected primitive and determining a candidate virtual camera to be the virtual camera for which the corresponding aspect measurement is a maximum;

repeating the calculating, comparing and determining steps for each of the selected primitive whereby candidate virtual cameras are determined for each selected primitive; and

choosing the optimum virtual camera on the basis of the frequency with which virtual cameras are determined to be candidate virtual cameras.

6. A method as claimed in claim 5, wherein the primitives comprise facets.

7. A method as claimed in claim 6, wherein the calculation of the aspect measurement comprises, for a given facet and a given virtual camera, calculating a scalar product of a unit vector normal to the facet and a unit vector parallel to the look direction of the virtual camera.

8. A method as claimed in claim 6, wherein the calculation of aspect measurement comprises calculating, for a given facet and for a given virtual camera, an area of the facet when viewed in projection in the look direction of the virtual camera.

11. Apparatus for generating model data representative of a model in a three dimensional space of an object from input signals representative of a set of images of the object taken from a plurality of respective camera positions, the apparatus comprising:

display means and control means operable to control the display means to display a model image derived from the model data and comprising a plurality of primitives for viewing by a user;

means for receiving at least one primitive selection signal responsive to user actuation of an input means whereby each primitive selection signal identifies a respective selected primitive of the model;

means for defining a plurality of virtual cameras in the three dimensional space having positions and look directions relative to the model which correspond substantially to those of the respective actual cameras relative to the object;

evaluating means for evaluating which of the virtual cameras is an optimum virtual camera for generating a view of the selected primitives;

identifying means for identifying from the camera images a first camera image of the plurality of camera images taken from a camera position corresponding to that of the optimum virtual camera; and

determining means for determining from the camera images a second camera image as being suitable for matching features in the first camera image, the control means being operable to control the display means to display the second camera image for comparison by the user with the first camera image.

12. Apparatus as claimed in claim 11, wherein the second camera image is taken from a camera position proximate to the optimum camera position.

13. Apparatus as claimed in claim 11, further comprising means for receiving feature matching selection signals representative of user matched points in the first and second camera images.

14. Apparatus as claimed in claim 13, further comprising means for generating updated model data to include additional detail corresponding to the received feature matching selection signals, means for rendering the updated model data to generate an updated model image and means for controlling the display means to display the updated model image.

19. Apparatus as claimed in claim 11, wherein the input means is a pointing means co-operable with the display means to provide input signals in the form of image co-ordinates of the displayed image.

20. Apparatus as claimed in claim 11, comprising means for generating the displayed model image by rendering the image data.

15. Apparatus for generating model data representative of a model in a three dimensional space of an object from input signals representative of a set of images of the object taken from a plurality of respective camera positions, the apparatus comprising:

display means and control means operable to control the display means to display a model image derived from the model data and comprising a plurality of primitives for viewing by a user;

means for receiving at least one primitive selection signal responsive to user actuation of an input means whereby each primitive selection signal identifies a respective selected primitive of the model;

means for defining a plurality of virtual cameras in the three dimensional space having positions and look directions relative to the model which correspond substantially to those of the respective actual cameras relative to the object;

evaluating means for evaluating which of the virtual cameras is an optimum virtual camera for generating a view of the selected primitives; and

identifying means for identifying from the camera images a first camera image of the plurality of camera images taken from a camera position corresponding to that of the optimum virtual camera,

wherein the evaluating means comprises;

means for calculating for a selected primitive an aspect measurement representative of the visibility of the primitive when viewed in projection in the look direction of one of the virtual cameras;

means for repeating the calculating step to obtain a respective aspect measurement for each of the virtual cameras;

means for comparing the aspect measurements for the selected primitive and for determining a candidate virtual camera to be the virtual camera for which the corresponding aspect measurement is a maximum;

means for repeating the calculating, comparing and determining steps for each of the selected primitive whereby candidate virtual cameras are determined for each selected primitive; and

means for choosing the optimum virtual camera on the basis of the frequency with which virtual cameras are determined to be candidate virtual cameras.

16. Apparatus as claimed in claim 15, wherein the primitives comprise facets.

17. Apparatus as claimed in claim 16, wherein the means for calculation of the aspect measurement comprises, for a given facet and a given virtual camera, means for calculating a scalar product of a unit vector normal to the facet and a unit vector parallel to the look direction of the virtual camera.

18. Apparatus as claimed in claim 16, wherein the means for calculation of aspect measurement comprises means for calculating, for a given facet and for a given virtual camera, an area of the facet when viewed in projection in the look direction of the virtual camera.

21. A computer readable medium encoded with a computer program to implement a method of generating model data representative of a model in a three dimensional space of an object from input signals representative of a set of images of the object taken from a plurality of respective camera positions, comprising the steps of:

displaying a model image derived from the model data and comprising a plurality of primitives for viewing by a user;

receiving at least one primitive selection signal responsive to user actuation of an input means whereby each primitive selection signal identifies a respective selected primitive of the model;

defining a plurality of virtual cameras in the three dimensional space having positions and look directions relative to the model which correspond substantially to those of the respective actual cameras relative to the object;

evaluating which of the virtual cameras is an optimum virtual camera for generating a view of the selected primitives;

identifying from the camera images a first camera image of the plurality of camera images taken from a camera position corresponding to that of the optimum virtual camera; and

determining from the camera images a second camera image as being suitable for matching features in the first camera image and displaying the second camera image for comparison by the user with the first camera image.