Method and system for tracking vantage points from which pictures of an object have been taken

• US 6,222,937 B1
• Filed: 06/30/1997
• Issued: 04/24/2001
• Est. Priority Date: 02/16/1996
• Status: Expired due to Term

• Current Assignee: Uber Technologies, Inc.
• Original Assignee: Microsoft Corporation
• Inventors: Cohen, Michael F., Grzeszczuk, Radek
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Werner, Brian P.
• Application Number:
  US08/885,251
• Time in Patent Office: 1,394 Days
• Field of Search: 345/425, 345/440, 345/115, 345/116, 345/133, 345/139, 345/113, 345/114, 345/302, 345/328, 396/9, 396/10, 396/308, 396/309, 396/334, 396/329, 396/333, 382/154, 382/285, 434/3, 434/23, 434/20, 434/40, 434/111, 434/307.R, 352/70, 348/46, 348/47, 348/49, 348/50
• US Class Current: 382/154
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 15/50 Lighting effects

Abstract

A computer-based method and system for digital 3-dimensional imaging of an object which allows for viewing images of the object from arbitrary vantage points. The system, referred to as the Lumigraph system, collects a complete appearance of either a synthetic or real object (or a scene), stores a representation of the appearance, and uses the representation to render images of the object from any vantage point. The appearance of an object is a collection of light rays that emanate from the object in all directions. The system stores the representation of the appearance as a set of coefficients of a 4-dimensional function, referred to as the Lumigraph function. From the Lumigraph function with these coefficients, the Lumigraph system can generate 2-dimensional images of the object from any vantage point. The Lumigraph system generates an image by evaluating the Lumigraph function to identify the intensity values of light rays that would emanate from the object to form the image. The Lumigraph system then combines these intensity values to form the image.

Claims

1. A method in a computer system for tracking vantage points from which pictures of an object have been taken, the computer system having a display device, the method comprising:

displaying an image of the object on the display device;

displaying around the displayed image of the object a three-dimensional view of a translucent hemisphere having a surface that is visually indicated, the displaying such that the image of the object is visible through the surface of the translucent hemisphere;

identifying a plurality of three-dimensional vantage points from which pictures of the object have been taken; and

for each of the plurality of vantage points from which pictures of the object have been taken, and in the order that pictures have been taken,

displaying a two-dimensional, geometric indication of the vantage point at a position on the surface of the displayed translucent hemisphere corresponding to the direction of the vantage point from which a picture of the object was taken.

2. The method of claim 1 wherein each vantage point is derived from the picture of the object.

3. The method of claim 2 wherein the pictures are taken using a hand-held camera from arbitrary vantage points.

4. The method of claim 2 wherein the object is positioned within a background with uniquely identifiable markers and wherein the vantage point of each picture is derived from the position of the uniquely identifiable markers in the picture.

5. The method of claim 1 wherein the indication of each vantage point is a rectangle.

6. A method in a computer system for displaying on a display device an indication of each of a plurality of directions, from which pictures of an object have been taken, in three dimensions, comprising:

displaying a perspective view of a translucent hemisphere surrounding the object on the display device;

identifying a plurality of the directions; and

for each of the plurality of directions and in the order that directions have been identified,

determining a position on the hemisphere that corresponds to the direction; and

displaying a two-dimensional, geometric indication at the determined position on the hemisphere.

7. The method of claim 6 wherein the directions are derived from vantage points from which pictures of a scene outside the hemisphere are taken from a location corresponding to within the hemisphere.

8. The method of claim 6 wherein the displayed indications are rectangular shapes.

9. The method of claim 6 wherein the displayed indications represent vantage points from which picture have been taken.

10. The method of claim 9 wherein as each new picture is taken an new indication of the vantage point of the picture is displayed.

11. The method of claim 9 wherein the pictures are taken with a hand-held camera.

12. A computer-readable medium containing instructions for causing a computer system to display on a display device an indication of each of a plurality of directions, from which pictures of an object have been taken, in three dimensions, by:

displaying a perspective view of a translucent three-dimensional surface surrounding the object on the display device;

identifying a plurality of the directions; and

for each of the plurality of directions and in the order that directions have been identified,

determining a position on the surface that corresponds to the direction; and

displaying a two-dimensional, geometric indication at the determined position on the surface.

13. The computer-readable medium of claim 12 wherein the directions are derived from vantage points from which pictures of a scene outside the surface are taken from a location corresponding to within the surface.

14. The computer-readable medium of claim 12 wherein the displayed indications are rectangular shapes.

15. The computer-readable medium of claim 12 wherein the displayed indications represent vantage points from which picture have been taken.

16. The computer-readable medium of claim 15 wherein as each new picture is taken an new indication of the vantage point of the picture is displayed.

17. The computer-readable medium of claim 15 wherein the pictures are taken with a hand-held camera.

18. A method in a computer system for tracking vantage points from which pictures of an object have been taken, the computer system having a display device, the method comprising:

displaying an image of the object on the display device;

displaying a three-dimensional translucent hemisphere around the displayed image of the object;

identifying a plurality of three-dimensional vantage points from which pictures of the object have been taken; and

for each of the plurality of vantage points from which pictures of the object have been taken, and in the order that pictures have been taken,

displaying on the displayed three-dimensional hemisphere a two-dimensional, geometric indication of the vantage point.

19. The method of claim 18 wherein each vantage point is derived from the picture of the object.

20. The method of claim 19 wherein the pictures are taken using a hand-held camera from arbitrary vantage points.

21. The method of claim 19 wherein the object is positioned within a background with uniquely identifiable markers and wherein the vantage point of each picture is derived from the position of the uniquely identifiable markers in the picture.

22. The method of claim 18 wherein the indication of each vantage point is a rectangle.

23. A system for tracking directions from which pictures of an object have been taken, the system comprising:

a display device with an image of the object and a three-dimensional translucent hemisphere displayed around the displayed image of the object; and

a direction tracking component that identifies a direction from which pictures of the object have been taken, and for each of the directions from which pictures of the object have been taken and in the order that pictures have been taken, displays on the displayed hemisphere a two-dimensional, geometric indication of the direction.

24. The system of claim 23 including a camera for taking the pictures.

25. The system of claim 23 wherein each vantage point is derived from the picture of the object.

26. The system of claim 25 wherein the pictures are taken using a hand-held camera from arbitrary directions.

27. The system of claim 25 wherein the object is positioned within a background with uniquely identifiable markers and wherein the direction of each picture is derived from the position of the uniquely identifiable markers in the picture.

28. The system of claim 23 wherein the indication of each direction is a rectangle.

29. A computer-readable medium containing instructions for causing a computer system to display indication of directions from which pictures of an object have been taken, by:

displaying a three-dimensional translucent surface surrounding the object;

identifying a plurality of the directions; and

for each of the plurality of directions, and in the order the directions have been identified, receiving the direction; and

displaying on the displayed three-dimensional surface a two-dimensional, geometric indication of the direction.

30. The computer-readable medium of claim 29 wherein each direction is derived from a picture of the object.

31. The computer-readable medium of claim 30 wherein the pictures are taken using a hand-held camera from arbitrary vantage points.

32. The computer-readable medium of claim 30 wherein the object is positioned within a background with uniquely identifiable markers and wherein the vantage point of each picture is derived from the position of the uniquely identifiable markers in the picture.

33. The computer-readable medium of claim 29 wherein the indication of each direction is a rectangle.

34. The computer-readable medium of claim 29 wherein the three-dimensional surface is a hemisphere.

35. The computer-readable medium of claim 29 wherein the three-dimensional surface is transparent except for a visual indication ot the shape of the surface.

36. The computer readable medium of claim 29 wherein the directions are derived from pictures of a scene outside the three-dimensional surface.