Pop-up light field

• US 20050219264A1
• Filed: 04/01/2004
• Published: 10/06/2005
• Est. Priority Date: 04/01/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Jian, Shum, Heung-Yeung, Yamazaki, Shuntaro, Li, Yin
• Granted Patent: US 7,633,511 B2
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G06T 15/205 Image-based rendering

Abstract

Techniques are disclosed to produce virtual views of a complex scene. The virtual views are substantially free from aliasing even when using a relatively sparse set of images of the scene. In a described implementation, a scene is split into one or more coherent layers. The boundaries of the coherent layers are propagated across a plurality of frames corresponding to the scene. The splitting may be further refined (e.g., in accordance with user feedback) to present a virtual view of the scene.

Claims

1. A method comprising:

splitting a scene into one or more coherent layers;

propagating boundaries of the coherent layers across a plurality of frames corresponding to the scene; and

refining the splitting to present a virtual view of the scene.

2. A method as recited in claim 1, wherein the virtual view of the scene is substantially free from aliasing.

3. A method as recited in claim 1, wherein each of the coherent layers has a corresponding background layer.

4. A method as recited in claim 1, wherein the plurality of frames correspond to different images of the scene.

5. A method as recited in claim 1, wherein the refining is initiated by a user.

6. A method as recited in claim 1, wherein each layer of the scene has a corresponding plane equation to represent a local geometry of that layer.

7. A method as recited in claim 1, further comprising rendering the coherent layers with a corresponding background layer to present the virtual view of the scene.

8. A method as recited in claim 1, further comprising rendering the coherent layers with a corresponding background layer to present the virtual view of the scene, wherein the background layer is provided by combining a plurality of under-segmented regions.

9. One or more computer-readable media storing computer executable instructions that, when executed, perform the method as recited in claim 1.

10. A method comprising:

segmenting a light field into one or more coherent layers;

propagating boundaries of the coherent layers across a plurality of frames corresponding to the light field;

providing a background layer for the coherent layers; and

rendering the coherent layers with the background layer to provide a pop-up light field.

11. A method as recited in claim 10, further comprising refining the coherent layers.

12. A method as recited in claim 10, further comprising determining uncertain regions between the coherent layers and the background layer.

13. A method as recited in claim 10, further comprising applying alpha matting to the coherent layers.

14. A method as recited in claim 10, wherein the background layer is provided by combining a plurality of under-segmented regions.

15. A method as recited in claim 10, wherein each of the coherent layers have a background layer.

16. A method as recited in claim 10, further comprising applying a coherent feathering function to the coherent layer boundaries.

17. A method as recited in claim 10, wherein a plurality of polygons represent the coherent layer boundaries.

18. A method as recited in claim 10, wherein the rendering utilizes texture-mapped triangles.

19. A method as recited in claim 10, wherein the rendering sequentially combines the coherent layers and the background by alpha blending.

20. A method as recited in claim 10, wherein the plurality of frames correspond to different images of the light field.

21. A method as recited in claim 10, wherein the pop-up light field is substantially free from aliasing.

22. One or more computer-readable media storing computer executable instructions that, when executed, perform the method as recited in claim 10.

23. A method comprising:

determining a plurality of texture-mapped triangles for each layer of a scene;

binding a plurality of textures to each of the plurality of triangles;

assigning a blending ratio to each vertex of the plurality of triangles; and

blending the textures of each of the plurality of triangles to present a virtual view of the scene.

24. A method as recited in claim 23, wherein at least three textures are bound to each of the plurality of triangles.

25. A method as recited in claim 23, wherein the blending ratio is assigned as a primary color on each vertex.

26. A method as recited in claim 23, wherein the blending ratio is assigned as a primary color on each vertex and the primary color is interpolated on each of the plurality of triangles.

27. A method as recited in claim 23, wherein the virtual view of the scene is substantially free from aliasing.

28. A method as recited in claim 23, wherein the blending utilizes a blending equation.

29. A method as recited in claim 23, wherein the blending utilizes a blending equation stored in a pixel shader of a graphics hardware device.

30. A user interface comprising:

a layer pop-up module to allow a user to define one or more coherent layers corresponding to a scene;

a refinement module to refine the coherent layers; and

a rendering module to render the coherent layers to present a virtual view of the scene.

31. A user interface as recited in claim 30, wherein a plurality of polygons represent boundaries of the coherent layers.

32. A user interface as recited in claim 30, wherein the virtual view of the scene is substantially free from aliasing.

33. A user interface as recited in claim 30, further comprising a background construction module to provide a background layer corresponding to the coherent layers.

34. A user interface as recited in claim 30, further comprising a background construction module to provide a background layer corresponding to the coherent layers, wherein the background layer is provided by removing the coherent layers from a key frame corresponding to the scene.

35. A system comprising:

a layer pop-up module to split a scene into one or more coherent layers;

a boundary propagation module to propagate boundaries of the coherent layers across a plurality of frames corresponding to the scene; and

a refinement module to refine the splitting to present a virtual view of the scene.

36. A system as recited in claim 35, wherein the virtual view of the scene is substantially free from aliasing.

37. A system as recited in claim 35, wherein the plurality of frames correspond to different images of the scene.

38. A system as recited in claim 35, wherein the refinement module is activated by a user.

39. A system as recited in claim 35, wherein each layer of the scene has a corresponding plane equation to represent a local geometry of that layer.

40. A system as recited in claim 35, further comprising a rendering module to render the coherent layers with a corresponding background layer to present the virtual view of the scene.

41. A system as recited in claim 35, further comprising a rendering module to render the coherent layers with a corresponding background layer to present the virtual view of the scene, wherein the background layer is provided by combining a plurality of under-segmented regions.

42. A system as recited in claim 35, further comprising a memory module to store instructions.

43. A system as recited in claim 35, further comprising one or more processing units to execute a plurality of stored instructions on one or more memory modules coupled to the processors.

44. One or more computer-readable media having instructions stored thereon that, when executed, direct a machine to perform acts comprising:

splitting a scene into one or more coherent layers;

propagating boundaries of the coherent layers across a plurality of frames corresponding to the scene; and

refining the splitting to present a virtual view of the scene.

45. A computer-readable media as recited in claim 44, wherein the virtual view of the scene is substantially free from aliasing.

46. A computer-readable media as recited in claim 44, wherein the acts further comprise rendering the coherent layers with a corresponding background layer to present the virtual view of the scene.

47. A computer-readable media as recited in claim 44, wherein the acts further comprise rendering the coherent layers with a corresponding background layer to present the virtual view of the scene, wherein the background layer is provided by combining a plurality of under-segmented regions.

48. One or more computer-readable media having instructions stored thereon that, when executed, direct a machine to perform acts comprising:

determining a plurality of texture-mapped triangles for each layer of a scene;

binding a plurality of textures to each of the plurality of triangles;

assigning a blending ratio to each vertex of the plurality of triangles; and

blending the textures of each of the plurality of triangles to present a virtual view of the scene.

49. A computer-readable media as recited in claim 48, wherein at least three textures are bound to each of the plurality of triangles.

50. A computer-readable media as recited in claim 48, wherein the blending ratio is assigned as a primary color on each vertex.

51. A computer-readable media as recited in claim 48, wherein the blending ratio is assigned as a primary color on each vertex and the primary color is interpolated on each of the plurality of triangles.

52. A computer-readable media as recited in claim 48, wherein the virtual view of the scene is substantially free from aliasing.

53. A computer-readable media as recited in claim 48, wherein the blending utilizes a blending equation.

54. A computer-readable media as recited in claim 48, wherein the blending utilizes a blending equation stored in a pixel shader of a graphics hardware device.

55. An apparatus comprising:

means for splitting a scene into one or more coherent layers;

means for propagating boundaries of the coherent layers across a plurality of frames corresponding to the scene; and

means for refining the splitting to present a virtual view of the scene.

56. An apparatus as recited in claim 55, further comprising means for rendering the coherent layers with a corresponding background layer to present the virtual view of the scene.

57. An apparatus as recited in claim 55, further comprising means for rendering the coherent layers with a corresponding background layer to present the virtual view of the scene, wherein the background layer is provided by combining a plurality of under-segmented regions.