Determining a vantage point of an image

• US 6,028,955 A
• Filed: 06/29/1998
• Issued: 02/22/2000
• Est. Priority Date: 02/16/1996
• Status: Expired due to Term

• Current Assignee: Uber Technologies, Inc.
• Original Assignee: Microsoft Corporation
• Inventors: Cohen, Michael F., Grzeszczuk, Radek
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Werner, Brian P.
• Application Number:
  US09/106,812
• Time in Patent Office: 603 Days
• Field of Search: 382/151, 382/154, 382/162-164, 382/287, 382/291, 382/293, 382/312, 348/20, 348/50, 348/94, 348/95, 348/137, 348/140, 348/587, 356/142, 356/143, 356/375, 356/248, 250/589.1, 250/559.29, 396/333, 901/47
• US Class Current: 382/154
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 15/50 Lighting effects

Abstract

A computer-based method and system for digital 3-dimensional imaging of an object which allows for viewing images of the object from arbitrary vantage points. The system, referred to as the Lumigraph system, collects a complete appearance of either a synthetic or real object (or a scene), stores a representation of the appearance, and uses the representation to render images of the object from any vantage point. The appearance of an object is a collection of light rays that emanate from the object in all directions. The system stores the representation of the appearance as a set of coefficients of a 4-dimensional function, referred to as the Lumigraph function. From the Lumigraph function with these coefficients, the Lumigraph system can generate 2-dimensional images of the object from any vantage point. The Lumigraph system generates an image by evaluating the Lumigraph function to identify the intensity values of light rays that would emanate from the object to form the image. The Lumigraph system then combines these intensity values to form the image.

Claims

1. A method for determining a vantage point from which a picture of an object was taken, the method comprising:

providing a plurality of visible markers, each marker comprising a plurality of concentric geometric shapes each having a radius, each marker uniquely identifiable be the number of geometric shapes and the ratio of the radii of the innermost geometric shape to the outermost geometric shape;

placing the object in a position with a background that includes the plurality of markers, the background being in a base color and the markers being in a different shade of the base color;

taking a picture of the object from a vantage point having a position and orientation in which multiple markers are visible within the picture;

identifying the markers that are visible within the picture based on the number of geometric shapes and the ratio of the radii of the innermost geometric shape to the outermost geometric shape, and determining the position of the identified markers within the picture; and

calculating both the position and orientation of the vantage point from which the picture was taken based on the identified markers and the determined positions of the identified markers within the picture.

2. The method of claim 1 wherein the base color is cyan.

3. The method of claim 1 wherein each marker comprises concentric rings of the different shade.

4. The method of claim 3 wherein each marker contains between 2 and 5 concentric rings.

5. The method of claim 3 wherein each marker is uniquely identifiable by the number of concentric rings of the marker and the ratio of the radii of the inner concentric rings to the outermost concentric ring of the marker.

6. A method for determining a vantage point from which a picture of an object was taken, the method comprising:

placing the object on a stage with a base and two sides, the stage having a background that includes a plurality of visible markers, each marker comprising a plurality of concentric geometric shapes each having a radius, each marker uniquely identifiable by the number of geometric shapes and the ratio of the radii of the innermost geometric shape to the outermost geometric shape;

taking a picture of the object from a vantage point having a position and orientation in which multiple markers are visible within the picture;

identifying the markers that are visible within the picture based on the number of geometric shapes and the ratio of the radii of the innermost geometric shape to the outermost geometric shape, and determining the position of the identified markers within the picture; and

calculating the position and orientation of the vantage point from which the picture was taken based on the identified markers and the determined positions of the identified markers within the picture.

7. The method of claim 6 wherein 12 markers are placed on the base and 9 markers are placed on each side.

8. The method of claim 6 wherein the base is rotatable so that pictures of the object can be take from various vantage points around the object.

9. The method of claim 6 wherein the pictures are taken with a hand-held camera.

10. The method of claim 6 wherein the background and the markers upon the background are in different colors.

11. The method of claim 6 wherein the background is in a base color and the markers comprise a different shade of the base color.

12. The method of claim 11 wherein the base color is cyan.

13. The method of claim 11 wherein each marker comprises concentric rings of the different shade.

14. The method of claim 13 wherein each marker contains between 2 and 5 concentric rings.

15. The method of claim 13 wherein each marker is uniquely identifiable by the number of concentric rings of the marker and the ratio of the radii of the inner concentric rings to the outermost concentric ring of the marker.

16. A method for determining a vantage point from which a picture of an object was taken, the method comprising:

placing the object in a position with a background that includes a plurality of visible markers, each marker comprising a plurality of concentric geometric patterns each having a radius, each marker uniquely identifiable by the number of geometric patterns and the ratio of the radii of the innermost geometric pattern to the outermost geometric pattern;

taking a picture of the object from an arbitrary vantage point having a position and orientation in which multiple markers are visible within the picture, the picture being taken using a hand-held camera;

identifying the markers that are visible within the picture based on the number of geometric patterns and the ratio Of the radii of the innermost geometric pattern to the outermost geometric pattern, and determining the position of the identified markers within the picture; and

calculating the position and orientation of the vantage point from which the picture was taken based on the identified markers and the determined positions of the identified markers within the picture.

17. The method of claim 16 wherein the background and the marker are in different colors.

18. The method of claim 16 wherein the background is in a base color and the markers comprise a different shade of the base color.

19. The method of claim 18 wherein the base color is cyan.

20. The method of claim 18 wherein each marker comprises concentric rings of the different shade.

21. The method of claim 20 wherein each marker contains between 2 and 5 concentric rings.

22. The method of claim 20 wherein each marker is uniquely identifiable by the number of concentric rings of the marker and the ratio of the radii of the inner concentric rings to the outermost concentric ring of the marker.

23. The method of claim 16 wherein the background includes a base and two sides of a stage upon which the object is placed.

24. The method of claim 23 wherein markers are positioned on the base and on the two sides of the stage.

25. The method of claim 24 wherein 12 markers are placed on the base and 9 markers are placed on each side.

26. The method of claim 23 wherein the base is rotatable so that pictures of the object can be take from various vantage points around the object.

27. A method for determining a vantage point including a position and orientation from which a picture of an object was taken and determining an outline of the object, thc method comprising:

placing the object in a position with a background that includes a plurality of visible markers, the background and the markers being of different color than the object, each marker comprising a plurality of concentric geometric shapes each having a radius, each marker uniquely identifiable by the number of geometric shapes and the ratio of the radii of the innermost geometric shape to the outermost geometric shape;

taking a picture of the object from a vantage point having a position and orientation in which multiple markers are visible within the picture;

calculating the position and orientation of the vantage point from which the picture was taken based on the markers that arc visible within the picture; and

determining the outline of the object based on the difference in color between the object and the background and markers.

28. The method of claim 27 wherein the background is in a base color and the markers comprise a different shade of the base color.

29. The method of claim 28 wherein the base color is cyan.

30. The method of claim 27 wherein each marker comprises concentric rings of a different shade from the base color from the background.

31. The method of claim 30 wherein each marker contains between 2 and 5 concentric rings.

32. The method of claim 30 wherein each marker is uniquely identifiable by the number of concentric rings of the marker and the ratio of the radii of the inner concentric rings to the outermost concentric ring of the marker.

33. The method of claim 27 wherein the background includes a base and two sides of a stage upon which the object is placed.

34. The method of claim 33 wherein markers are positioned on the base and on the two sides of the stage.

35. The method of claim 34 wherein 12 markers are placed on the base and 9 markers are placed on each side.

36. The method of claim 34 wherein the base is rotatable so that pictures of the object can be take from arbitrary vantage points around the object.

37. The method of claim 27 wherein the pictures are taken with a hand-held camera.

38. The method of claim 37 wherein the background and the markers upon the background are in different colors.

39. A computer-readable medium containing instructions for causing a computer system to determine a vantage point from which a picture of an object was taken, by:

receiving a picture of the object taken while the object is positioned on a stage with a base and two sides and with a background that includes a plurality of visible markers, each marker comprising a plurality of concentric geometric shapes each having a radius, each marker uniquely identifiable by the number of geometric shapes and the ratio of the radii of the innermost geometric shape to the outermost geometric shape, the picture being taken from a vantage point having a position and orientation in which multiple markers are visible within the picture;

identifying the markers that arc visible within the picture based on the number of geometric shapes and the ratio of the radii of the innermost geometric shape to the outermost geometric shape, and determining the position of the identified markers within the picture; and

calculating the position and orientation of the vantage point from which the picture was taken based on the identified markers and the determined positions of the identified markers within the picture.

40. The computer-readable medium of claim 39 wherein 12 markers are positioned on the base and 9 markers are positioned on each side.

41. The computer-readable medium of claim 39 wherein the base is rotatable so that pictures of the object can be take from various vantage points around the object.

42. The computer-readable medium of claim 39 wherein the pictures are taken with a hand-held camera.

43. The computer-readable medium of claim 39 wherein the background and the markers upon the background are in different colors.

44. The computer-readable medium of claim 39 wherein the background is in a base color and the markers comprise a different shade of the base color.

45. The computer-readable medium of claim 44 wherein the base color is cyan.

46. The computer-readable medium of claim 44 wherein each marker comprises concentric rings of the different shade.

47. The computer-readable medium of claim 46 wherein each marker contains between 2 and 5 concentric rings.

48. The computer-readable medium of claim 46 wherein each marker is uniquely identifiable by the number of concentric rings of the marker and the ratio of the radii of the inner concentric rings to the outermost concentric ring of the marker.

49. A computer-readable medium containing instructions for causing a computer system to determine a vantage point from which a picture of an object was taken and to determine an outline of the object, by:

receiving a picture of the object taken while the object is positioned with a background that includes a plurality of visible markers, the background and the markers being of different color than the object, each marker comprising a plurality of concentric geometric shapes each having a radius, each marker uniquely identifiable by the number of geometric shapes and the ratio of the radii of the innermost geometric shape to the outermost geometric shape, the picture being taken from a vantage point having a position and orientation in which multiple markers are visible within the picture;

calculating the position and orientation of the vantage point from which the picture was taken based on the markers that are visible within the picture; and

determining the outline of the object based on the difference in color between the object and the background and markers.

50. The computer-readable medium of claim 49 wherein the background is in a base color and the markers comprise a different shade of the base color.

51. The computer-readable medium of claim 50 wherein the base color is cyan.

52. The computer-readable medium of claim 49 wherein each marker comprises concentric rings of a different shade from the base color from the background.

53. The computer-readable medium of claim 52 wherein each marker contains between 2 and 5 concentric rings.

54. The computer-readable medium of claim 52 wherein each marker is uniquely identifiable by the number of concentric rings of the marker and the ratio of the radii of the inner concentric rings to the outermost concentric ring of the marker.

55. The computer-readable medium of claim 49 wherein the background includes a base and two sides of a stage upon which the object is positioned.

56. The computer-readable medium of claim 55 wherein markers are positioned on the base and on the two sides of the stage.

57. The computer-readable medium of claim 56 wherein 12 markers are placed on the base and 9 markers are placed on each side.

58. The computer-readable medium of claim 56 wherein the base is rotatable so that pictures of the object can be take from arbitrary vantage points around the object.

59. The computer-readable medium of claim 49 wherein the pictures are taken with a hand-held camera.

60. The computer-readable medium of claim 49 wherein the background and the markers upon the background are in different colors.

61. A computer-readable medium containing instructions for causing a computer system to determine a vantage point from which a picture of an object was taken, by:

receiving a picture of the object taken while the object is positioned on a stage with a base and two sides and with a background that includes a plurality of visible markers, each marker comprising a plurality of concentric geometric patterns each having a radius, each marker uniquely identifiable by thc number of geometric patterns and the ratio of the radii of the innermost geometric pattern to the outermost geometric pattern, the picture being taken from a vantage point having a position and orientation in which multiple markers are visible within the picture;

identifying the markers that are visible within the picture based on the number of geometric patterns and the ratio of the radii of the innermost geometric pattern to the outermost geometric pattern, and determining the position of the identified markers within the picture; and

calculating the position and orientation of the vantage point from which the picture was taken based on the identified markers and the determined positions of the identified markers within the picture.

62. The computer-readable medium of claim 61 wherein 12 markers are placed on the base and 9 markers are placed on each side.

63. The computer-readable medium of claim 61 wherein the base is rotatable so that pictures of the object can be take from various vantage points around the object.

64. The computer-readable medium of claim 61 wherein the pictures are taken with a hand-held camera.

65. The computer-readable medium of claim 61 wherein the background and the markers upon the background are in different colors.

66. The computer-readable medium of claim 61 wherein the background is in a base color and the markers comprise a different shade of the base color.

67. The computer-readable medium of claim 66 wherein the base color is cyan.

68. The computer-readable medium of claim 66 wherein each marker comprises concentric rings of the different shade.

69. The computer-readable medium of claim 68 wherein each marker contains between 2 and 5 concentric rings.