System and method for view management in three dimensional space

• US 8,681,146 B2
• Filed: 07/02/2009
• Issued: 03/25/2014
• Est. Priority Date: 05/17/2001
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Bell, Blaine A., Feiner, Steven K., Hoellerer, Tobias H
• Primary Examiner(s): BROOME, SAID A
• Application Number:
  US12/496,882
• Publication Number:
  US 20100141648A1
• Time in Patent Office: 1,727 Days
• Field of Search: None
• US Class Current: 345/419
• CPC Class Codes: G06T 15/40 Hidden part removal
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G09G 1/06 using single beam tubes G09G1/26, G09G1/28 take precedence, e.g. three-dimensional or perspective representation, rotation or translation of display pattern, hidden lines, shadows G09G1/28 takes precedence; stereoscopic TV-systems, details thereof H04N13/00; oscilloscopes for three-dimensional representation G01R13/206; vectorscopes G01R13/208
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

A method for managing a display space for a 3D environment is provided. A 3D scene having at least one scene object is displayed and the visible surfaces of the scene objects are represented as visible space in a 2D view plane representation. Controllable objects that are to be placed in the scene are defined by parameters such as size, placement priority, proximity relationships and the like. The available space for placing controllable objects, which can include empty space and low priority background and foreground regions, is determined for each controllable object. The placement for controllable objects in the 3D space is then determined in accordance with at least placement parameter and one of the visible space and available space of the view-plane representation such that view management objectives, such as not occluding important scene objects, are accomplished.

Claims

1. A method for managing a display space for a 3D environment comprising:

determining visible surfaces of a plurality of non-controllable scene objects in the 3D environment and representing the visible surfaces of the plurality of non-controllable scene objects as visible space in a view-plane representation, using a computer processor, wherein at least one of the plurality of non-controllable scene objects overlaps with another of the plurality of non-controllable scene objects in the view-plane representation;

defining at least one controllable object to be placed in the 3D environment;

for each controllable object, determining an available space in the view-plane representation; and

determining a placement of each controllable object in the 3D environment in accordance with at least one placement parameter and the available space for the controllable object in the view-plane representation,

wherein the controllable objects are displayed using a head-tracked display device,

wherein a particular controllable object is associated with one of the plurality of non-controllable objects and wherein, if the particular controllable object fits within the visible space of the associated non-controllable object, then the placement of the particular controllable object is determined to be within the extents of the visible surfaces of the associated non-controllable object, otherwise, if the particular controllable object does not fit within the visible space of the associated non-controllable object, then the placement of the particular controllable object in the 3D environment is determined in accordance with at least one placement parameter and the available space for the controllable object in the view-plane representation.

2. The method for managing a display space of claim 1, wherein the visible space for each object is represented as rectangles which are axis aligned with respect to the view-plane representation.

3. The method for managing a display space of claim 2, wherein the axis aligned rectangles comprise the set of largest visible space rectangles on the view-plane representation.

4. The method for managing a display space of claim 1, wherein the available space for each object is represented as rectangles which are axis aligned with respect to the view-plane representation.

5. The method for managing a display space of claim 4 wherein the axis aligned rectangles comprise the set of largest available space rectangles on the view-plane representation.

6. The method for managing a display space of claim 1, wherein a set of available space rectangles is determined from the view-plane representation and one available space rectangle from said set is selected based on the at least one placement parameter.

7. The method for managing a display space of claim 1, wherein the at least one placement parameter includes a relationship to the position of the projection of the controllable object in a previous frame of the 3D scene.

8. The method for managing a display space of claim 7, wherein the relationship to the position of the projection of the controllable object in the previous frame includes temporal state hysteresis.

9. The method for managing a display space of claim 7, wherein the relationship to the position of the controllable object in the previous frame includes positional stability with respect to an associated object.

10. The method for managing a display space of claim 7, wherein the relationship to the position of the controllable object in the previous frame includes interpolating between a last frame position and a current frame position.

11. The method for managing a display space of claim 1, wherein the controllable objects are displayed in a 3D environment in accordance with the determined placement.

30. The method for managing a display space of claim 1, wherein determining the placement of each controllable object comprises:

determining the placement of a first controllable object in accordance with at least one placement parameter and the available space for the controllable object in the view-plane representation; and

determining the placement of a second controllable object in accordance with at least one placement parameter, the available space for the controllable object in the view-plane representation, and the placement of the first controllable object.

12. A method of annotating scene objects in a frame of a 3D environment comprising:

determining visible surfaces of a plurality of non-controllable scene objects in the 3D environment and representing the visible surfaces of the plurality of non-controllable scene objects as visible space in a view-plane representation, using a computer processor, wherein at least one of the plurality of non-controllable scene objects overlaps with another of the plurality of non-controllable scene objects in the view-plane representation;

defining at least one annotation object associated with one of the plurality of non-controllable scene objects;

for each of the at least one annotation object, if the annotation object fits within the visible space of the associated non-controllable scene object, then placing the annotation object within the extents of the visible surfaces of the associated non-controllable scene object in the 3D environment, otherwise, if the annotation object does not fit within the visible space of the associated non-controllable scene object, determining an available space for the annotation object in the view-plane representation and placing the annotation object in the 3D environment in accordance with at least one placement parameter and the available space for the annotation object in the view-plane representation; and

displaying the annotation objects in the 3D environment in accordance with the placement,

wherein the annotation objects are displayed using a head-tracked display device.

13. The method for annotating scene objects of claim 12, wherein the visible space is represented as rectangles which are axis aligned with respect to the view-plane representation.

14. The method for annotating scene objects of claim 12, wherein the available space is represented as rectangles which are axis aligned with respect to the view-plane representation.

15. The method for annotating scene objects of claim 12, wherein a set of available space rectangles proximate the associated non-controllable scene object is determined from the view-plane representation and one rectangle from said set is selected based on at least one placement rule.

16. The method for annotating scene objects of claim 12, wherein the at least one placement rule is related to the position of the annotation object in a previous frame of the 3D scene.

17. A method for managing a display space for a 3D environment comprising:

determining visible surfaces of a plurality of non-controllable scene objects in the 3D environment and representing the visible surfaces of the plurality of non-controllable scene objects as visible space in a view-plane representation, using a computer processor, wherein at least one of the plurality of non-controllable scene objects overlaps with another of the plurality of non-controllable scene objects in the view-plane representation;

defining at least one controllable object to be placed in the 3D environment;

for each controllable object, determining an available space in the view-plane representation; and

determining a placement of each controllable object in the 3D environment in accordance with at least one placement parameter and the available space for the controllable object in the view-plane representation

wherein a particular controllable object is associated with one of the plurality of non-controllable scene objects and wherein, if the particular controllable object fits within the visible space of the associated non-controllable scene object, then the placement of the particular controllable object is determined to be within the extents of the visible surfaces of the associated non-controllable scene object, otherwise, if the particular controllable object does not fit within the visible space of the associated non-controllable scene object, then the placement of the particular controllable object in the 3D environment is determined in accordance with at least one placement parameter and the available space for the controllable object in the view-plane representation.

18. The method for managing a display space of claim 17, wherein the visible space for each object is represented as rectangles which are axis aligned with respect to the view-plane representation.

19. The method for managing a display space of claim 17, wherein the available space for each object is represented as rectangles which are axis aligned with respect to the view-plane representation.

20. The method for managing a display space of claim 17, wherein a set of available space rectangles is determined from the view-plane representation and one available space rectangle from said set is selected based on the at least one placement parameter.

21. The method for managing a display space of claim 17, wherein the at least one placement parameter includes a relationship to the position of the controllable object in a previous frame of the 3D scene.

22. The method for managing a display space of claim 21, wherein the relationship to the position of the controllable object in the previous frame includes temporal state hysteresis.

23. The method for managing a display space of claim 21, wherein the relationship to the position of the controllable object in the previous frame includes positional stability with respect to an associated object.

24. The method for managing a display space of claim 21, wherein the relationship to the position of the controllable object in the previous frame includes interpolating between a last frame position and a current frame position.

25. The method for managing a display space of claim 17, wherein the controllable objects are displayed in a 3D environment in accordance with the determined placement.

26. The method for managing a display space of claim 17, wherein the controllable objects are displayed using a head-tracked display device.

27. The method for managing a display space of claim 17, wherein the controllable objects are displayed using a stereo display device.

28. The method for managing a display space of claim 17, wherein the visible space for each object is represented as set of largest visible space rectangles which are axis aligned with respect to the view-plane representation.

29. The method for managing a display space of claim 17, wherein the available space for each object is represented as a set of largest available space rectangles which are axis aligned with respect to the view-plane representation.

31. The method for managing a display space of claim 17, wherein determining the placement of each controllable object comprises:

determining the placement of a first controllable object in accordance with at least one placement parameter and the available space for the controllable object in the view-plane representation; and

determining the placement of a second controllable object in accordance with at least one placement parameter, the available space for the controllable object in the view-plane representation, and the placement of the first controllable object.