Method and system for culling view dependent visual data streams for a virtual environment

• US 20050253872A1
• Filed: 10/09/2003
• Published: 11/17/2005
• Est. Priority Date: 10/09/2003
• Status: Abandoned Application

• Current Assignee: Ergodex Incorporated, Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Malzbender, Thomas, Gelb, Daniel G., Goss, Michael E.
• Application Number:
  US10/684,030
• Publication Number:
  US 20050253872A1
• Time in Patent Office: Days
• US Class Current: 345/660
• CPC Class Codes: G06T 15/30 Clipping
• CPC Class Codes: G06T 2210/08 Bandwidth reduction
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A method for culling visual data streams. Specifically, one embodiment of the present invention discloses a method for culling view dependent visual data streams for a virtual environment. The method begins by determining a view volume of a viewing participant within the virtual environment. The view volume determines a field-of-view of the viewing participant within the virtual environment. The embodiment of the method then determines a proximity of a representation of an observed object in the virtual environment to the view volume. Thereafter, the embodiment of the method processes a view dependent visual data stream of the observed object only when the representation is within a specified proximity to the view volume.

Claims

1. A method for culling view dependent visual data streams for a virtual environment, comprising:

determining a view volume of a viewing participant within said virtual environment, wherein said view volume defines a field-of-view of said viewing participant within said virtual environment;

determining a proximity of a representation of an observed object in said virtual environment to said view volume; and

processing a view dependent visual data stream of said observed object only when said representation is within a specified proximity to said view volume.

2. The method of claim 1, wherein said providing access to a source of said visual data further comprises:

computing a three-dimensional model of said observed object, said three-dimensional model based on a plurality of real-time video streams taken of said observed object from a plurality of sample viewpoints.

3. The method of claim 2, wherein said generating visual data streams further comprises:

generating a view dependent video image stream by applying a new view synthesis technique to said three-dimensional model of said observed object, wherein said video image stream is generated from a viewpoint of said viewing participant.

4. The method of claim 1, further comprising:

sending said visual data stream to said viewing participant.

5. The method of claim 1, wherein said determining a view volume further comprises:

determining a view direction of said viewing participant to define said view volume, wherein said view volume comprises a series of expanding cross-sections of a geometric object along said view direction from said viewing participant within said virtual environment.

6. The method of claim 5, wherein said geometric object comprises a four-sided rectangular plane.

7. The method of claim 1, wherein said determining a proximity of a representation of an observed object in said virtual environment to said view volume, further comprises:

determining that said representation is within said specified proximity;

determining when said representation is occluded in said view volume such that said observed is not visible to said viewing participant; and

not generating said video image stream when said representation is occluded.

8. The method of claim 1, further comprising:

providing for hysteresis and anticipation in delivering said video image stream to said viewing participant by defining an extended bounding volume that surrounds said observed object within said three-dimensional virtual environment, wherein said representation comprises said extended bounding volume when determining said proximity.

9. The method of claim 1, further comprising:

enabling a change in a location of said viewing participant within said three-dimensional virtual environment by determining a new view volume of said viewing participant within said virtual environment;

determining when said representation falls within said new view volume; and

generating a video image stream of said observed object from said three-dimensional model when said representation is within said specified proximity to said new view volume

10. The method of claim 1, further comprising:

enabling a change in location of said observed object within said three-dimensional virtual environment and reflecting said change in location in said representation.

11. The method of claim 1, wherein said observed object comprises a local participant.

12. The method of claim 1, wherein said virtual environment comprises a three dimensional N-way virtual collaborative environment.

13. A system for culling view dependent visual data for a virtual environment, comprising:

a view volume generator for determining a view volume of a viewing participant within said virtual environment, wherein said view volume defines a field-of-view of said viewing participant within said virtual environment;

a comparator for determining a proximity of a representation of an observed object in said virtual environment to said view volume; and

a processor for processing a view dependent visual data stream of said observed object only when said representation is within a specified proximity to said view volume.

14. The system of claim 13, wherein said source comprises:

a model generator computing a three-dimensional model of said observed object that is based on a plurality of real-time video streams taken of said observed object from a plurality of sample viewpoints; and

a new view synthesis module for generating a view dependent video image stream by applying a new view synthesis technique to said three-dimensional model of said observed object, wherein said video image stream is generated from a viewpoint of said viewing participant.

15. The system of claim 13, further comprising:

a transmitter for sending said visual data stream to said viewing participant.

16. The system of claim 13, wherein said view volume generator determines a view direction of said viewing participant to define said view volume, wherein said view volume comprises a series of expanding cross-sections of a geometric object along said view direction from said viewing participant within said virtual environment

17. The system of claim 13, wherein said comparator determines when said representation is occluded in said view volume such that said viewing participant is unable to view said observed object, such that said video image stream is not generated when said representation is occluded.

18. The system of claim 13, wherein said representation comprises an extended bounding volume that surrounds said observed object within said virtual environment, wherein said representation comprises said extended bounding volume when determining said proximity.

19. The system of claim 13, wherein said view volume generator enables a change in a location of said viewing participant to a new location within said virtual environment by changing said view volume of said viewing participant within said virtual environment to reflect said new location.

20. The system of claim 13, wherein said comparator enables a change in location of said observed object to a new location within said three-dimensional virtual environment and reflects said change in location in said representation.

21. A computer system comprising:

a processor; and

a computer readable memory coupled to said processor and containing program instructions that, when executed, implement a method for culling view dependent visual data streams for a virtual environment, comprising;

determining a view volume of a viewing participant within said virtual environment, wherein said view volume defines a field-of-view of said viewing participant within said virtual environment;

determining a proximity of a representation of an observed object in said virtual environment to said view volume; and

processing a view dependent visual data stream of said observed object only when said representation is within a specified proximity to said view volume.

22. The computer system of claim 21, wherein said providing access to a source of said visual data in said method further comprises:

computing a three-dimensional model of said observed object, said three-dimensional model based on a plurality of real-time video streams taken of said observed object from a plurality of sample viewpoints.

23. The computer system of claim 22, wherein said generating visual data streams in said method further comprises:

generating a view dependent video image stream by applying a new view synthesis technique to said three-dimensional model of said observed object, wherein said video image stream is generated from a viewpoint of said viewing participant.

24. The computer system of claim 21, wherein said method further comprises:

sending said visual data stream to said viewing participant.

25. The computer system of claim 21, wherein said determining a view volume in said method further comprises:

determining a view direction of said viewing participant to define said view volume, wherein said view volume comprises a series of expanding cross-sections of a geometric object along said view direction from said viewing participant within said virtual environment.

26. The computer system of claim 25, wherein said geometric object comprises a four-sided rectangular plane.

27. The computer system of claim 21, wherein said determining a proximity of a representation of an observed object in said virtual environment to said view volume in said method, further comprises:

determining that said representation is within said specified proximity;

determining when said representation is occluded in said view volume such that said observed is not visible to said viewing participant; and

not generating said video image stream when said representation is occluded.

28. The computer system of claim 21, wherein said method further comprises:

providing for hysteresis and anticipation in delivering said video image stream to said viewing participant by defining an extended bounding volume that surrounds said observed object within said three-dimensional virtual environment, wherein said representation comprises said extended bounding volume when determining said proximity.

29. The computer system of claim 21, wherein said method further comprises:

enabling a change in a location of said viewing participant within said three-dimensional virtual environment by determining a new view volume of said viewing participant within said virtual environment;

determining when said representation falls within said new view volume; and

generating a video image stream of said observed object from said three-dimensional model when said representation is within said specified proximity to said new view volume

30. The computer system of claim 21, wherein said method further comprises:

enabling a change in location of said observed object within said three-dimensional virtual environment and reflecting said change in location in said representation.

31. The computer system of claim 21, wherein said observed object comprises a local participant.

32. The computer system of claim 21, wherein said virtual environment comprises a three dimensional N-way virtual collaborative environment.

33. A computer readable medium containing executable instructions which, when executed in a processing system, causes the system to perform the steps for a method of culling view dependent visual data streams for a virtual environment, comprising:

determining a view volume of a viewing participant within said virtual environment, wherein said view volume defines a field-of-view of said viewing participant within said virtual environment;

determining a proximity of a representation of an observed object in said virtual environment to said view volume; and

processing a view dependent visual data stream of said observed object only when said representation is within a specified proximity to said view volume.