Dockable billboards for labeling objects in a display having a three-dimensional perspective of a virtual or real environment

• US 10,380,799 B2
• Filed: 04/30/2014
• Issued: 08/13/2019
• Est. Priority Date: 07/31/2013
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Arsan, Roy, Raitz, Alexander, Allan, Clark, Noel, Cary Glen
• Primary Examiner(s): Cobb, Michael J
• Application Number:
  US14/266,511
• Publication Number:
  US 20150035822A1
• Time in Patent Office: 1,931 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A system and method that allows a user to view objects in a three-dimensional environment, where one or more of the objects have a data display (e.g., a data billboard, etc.) that shows data about the object. To enhance user experience and to provide relevant contextual data as the user navigates through the three-dimensional environment, the system calculates a location for the user and a location for each object and determines if a relationship between the user frame of reference and each object location satisfies a first criterion. If the first criterion is satisfied, the system is configured to move the data display to the bottom of a viewing area of the three-dimensional environment (e.g. docking the data display to the bottom of the viewing area, etc.). The system may also arrange the data displays in the same order as the objects are perceived by the user in the three-dimensional environment.

Claims

1. A method, comprising:

rendering portions of a three-dimensional environment as viewed from a perspective of a user while moving along a path from a first position in the three-dimensional environment to a second position in the three-dimensional environment based on user navigational input, wherein each three-dimensional object included in a plurality of three-dimensional objects in the three-dimensional environment has an associated data display area that is displayed as appearing fixed to a portion of the dimensional object, wherein at least one additional three-dimensional object is represented in the three-dimensional environment, each of the at least one additional three-dimensional objects having an associated data display area that is displayed as appearing fixed to a portion of the additional three-dimensional object;

as each portion of the three-dimensional environment is rendered;

based on a determination that each of the plurality of three-dimensional objects is within a specified angle as viewed from the perspective of the user, moving each associated data display area from appearing affixed to the portion of the corresponding three-dimensional object to a foreground position in the rendered portion of the three-dimensional environment, wherein the specified angle is less than 360 degrees;

based on a determination that the at least one additional three-dimensional object is not within the specified angle as viewed from the perspective of the user, maintaining each respective associated data display area as affixed to the at least one additional three-dimensional object;

causing display of the rendered portion of the three-dimensional environment; and

based on detecting at least one lateral change in relative positions of the plurality of three-dimensional objects, dynamically rearranging at least a portion of the associated data display areas located in the foreground position to reflect the relative positions,

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein a first three-dimensional object included in the plurality of three-dimensional objects is represented in the three-dimensional environment;

wherein the foreground position in the rendered portion of the three-dimensional environment comprises a first associated data display area, for the first three-dimensional object; and

further comprising;

in response to receiving the user navigational input, performing;

determining whether the first three-dimensional object, as viewed from a current position along the path, is within the specified angle; and

in response to determining that the first three-dimensional object, as viewed from the current position along the path, is within the specified angle, maintaining display of the first associated data display area in the foreground position.

3. The method of claim 1, wherein a first three-dimensional object included in the plurality of three-dimensional objects is represented in the three-dimensional environment;

wherein the foreground position in the rendered portion of the three-dimensional environment comprises a first associated data display area, for the first three-dimensional object; and

further comprising;

in response to receiving the user navigational input, performing;

determining whether the first three-dimensional object, as viewed from a current position along the path, is within the specified angle; and

in response to determining that the first three-dimensional object, as viewed from the current position along the path, is not within the specified angle, moving the first associated data display area away from the foreground position to be displayed as appearing fixed to a portion of the first three-dimensional object.

4. The method of claim 1, wherein display areas are non-overlapping with one another in the foreground position of the three-dimensional environment.

5. The method of claim 1, wherein content displayed in the data display area in the foreground position is the same as content displayed in the data display area that appeared affixed to the portion of the particular three-dimensional object.

6. The method of claim 1, wherein content displayed in the data display area in the foreground position is different from content displayed in the data display area that appeared affixed to the portion of the particular three-dimensional object.

7. The method of claim 1, wherein the foreground position is a display area portion that does not vary with a perspective as represented in the three-dimensional environment.

8. The method of claim 1, wherein the associated data display area when appearing affixed to the portion of the particular three-dimensional object is aligned with one of:

an adjacent area position, a left area portion, a right area portion, a top area portion, a bottom area portion or a center area portion of the particular three-dimensional object.

9. The method of claim 1, wherein the particular three-dimensional object is separated from the first position by a first finite distance;

wherein the particular three-dimensional object is separated from the second position by a second finite distance; and

wherein the first finite distance is different from the second finite distance.

10. The method of claim 1, wherein an associated data display area comprises content from one or more of:

real-time streaming data or non-real-time streaming data.

11. The method of claim 1, wherein the rendered portion of the three-dimensional environment is rendered on one or more of:

a display of a computing device, a display of a wearable computer, a heads-up display, or a display glass.

12. The method of claim 1, wherein the position along the path and a perspective as represented in the three-dimensional environment are determined based on one or more of:

a user'"'"'s actions relating to one or more user input devices or a user'"'"'s physical position and physical perspective as detected based on one or more sensors.

13. The method of claim 1, wherein the position along the path and a perspective as represented in the three-dimensional environment are determined based on a user'"'"'s physical position and physical perspective as detected based on one or more of:

an accelerometer, a geosynchronous positioning sensor, a geomagnetic field sensor, an orientation sensor, a gyroscope, a gravity sensor, or a rotation vector sensor.

14. The method of claim 1, wherein the three-dimensional environment is dynamically superimposed with a portion of a real-world three-dimensional environment in which a user moves, and wherein the user navigational input is generated through one or more sensors configured to track the user'"'"'s motion.

15. The method of claim 1, wherein the perspective of the user comprises at least one of a perspective of a virtual camera positioned in the three-dimensional environment and an imaginary line representing a perspective of a user included in the three-dimensional environment.

16. A non-transitory computer readable storage medium, storing software instructions, which when executed by one or more processors cause performance of steps of:

rendering portions of a three-dimensional environment as viewed from a perspective of a user while moving along a path from a first position in the three-dimensional environment to a second position in the three-dimensional environment based on user navigational input, wherein each three-dimensional object included in a plurality of three-dimensional objects in the three-dimensional environment has an associated datadisplay area that is displayed as appearing fixed to a portion of the dimensional object, wherein at least one additional three-dimensional object is represented in the three-dimensional environment, each of the at least one additional three-dimensional objects having an associated data display area that is displayed as appearing fixed to a portion of the additional three-dimensional object;

as each portion of the three-dimensional environment is rendered;

based on a determination that each of the plurality of three-dimensional objects is within a specified angle as viewed from the perspective of the user, moving each associated data display area from appearing affixed to the portion of the corresponding three-dimensional object to a foreground position in the rendered portion of the three-dimensional environment, wherein the specified angle is less than 360 degrees;

based on a determination that the at least one additional three-dimensional object is not within the specified angle as viewed from the perspective of the user, maintaining each respective associated data display area as affixed to the at least one additional three-dimensional object;

causing display of the rendered portion of the three-dimensional environment; and

based on detecting at least one lateral change in relative positions of the plurality of three-dimensional objects, dynamically rearranging at least a portion of the associated data display areas located in the foreground position to reflect the relative positions.

17. The non-transitory computer readable storage medium of claim 16, wherein a first three-dimensional object included in the plurality of three-dimensional objects is represented in the three-dimensional environment;

wherein the foreground position in the rendered portion of the three-dimensional environment comprises a first associated data display area, for the first three-dimensional object; and

further comprising;

in response to receiving the user navigational input, performing;

determining whether the first three-dimensional object, as viewed from a current position along the path, is within the specified angle; and

in response to determining that the first three-dimensional object, as viewed from the current position along the path, is within the specified angle, maintaining display of the first associated data display area in the foreground position.

18. The non-transitory computer readable storage medium of claim 16, wherein a first three-dimensional object included in the plurality of three-dimensional objects is represented in the three-dimensional environment;

wherein the foreground position in the rendered portion of the three-dimensional environment comprises a first associated data display area, for the first three-dimensional object; and

further comprising;

in response to receiving the user navigational input, performing;

determining whether the first three-dimensional object, as viewed from a current position along the path, is within the specified angle; and

in response to determining that the first three-dimensional object, as viewed from the current position along the path, is not within the specified angle, moving the first associated data display area away from the foreground position to be displayed as appearing fixed to a portion of the first three-dimensional object.

19. An apparatus comprising:

a display processor that renders portions of a three-dimensional environment as viewed from a perspective of a user while moving along a path from a first position in the three-dimensional environment to a second position in the three-dimensional environment based on user navigational input, wherein each three-dimensional object included in a plurality of three-dimensional objects in the three-dimensional environment has an associated data display area that is displayed as appearing fixed to a portion of the dimensional object, wherein at least one additional three-dimensional object is represented in the three-dimensional environment, each of the at least one additional three-dimensional objects having an associated data display area that is displayed as appearing fixed to a portion of the additional three-dimensional object;

as each portion of the three-dimensional environment is rendered;

the display processor, based on a determination that each of the plurality of three-dimensional objects is within a specified angle as viewed from the perspective of the user, moves each associated data display area from appearing affixed to the portion of the corresponding three-dimensional object to a foreground position in the rendered portion of the three-dimensional environment, wherein the specified angle is less than 360 degrees;

the display processor, based on a determination that the at least one additional three-dimensional object is not within the specified angle as viewed from the perspective of the user, maintains each respective associated data display area as affixed to the at least one additional three-dimensional object;

the display processor causes display of the rendered portion of the three-dimensional environment; and

based on detecting at least one lateral change in relative positions of the plurality of three-dimensional objects, the display processor causes at least a portion of the associated data display areas located in the foreground position to be dynamically rearranged to reflect the relative positions.

20. The apparatus of claim 19, wherein a first three-dimensional object included in the plurality of three-dimensional objects is represented in the three-dimensional environment;

wherein the foreground position in the rendered portion of the three-dimensional environment comprises a first associated data display area, for the first three-dimensional object; and

further comprising;

wherein, in response to receiving the user navigational input, the display processor determines whether the first three-dimensional object, as viewed from a current position along the path, is within the specified angle; and

wherein, the display processor, in response to determining that the first three-dimensional object, as viewed from the current position along the path, is within the specified angle, maintains display of the first associated data display area in the foreground position.

21. The apparatus of claim 19, wherein a first three-dimensional object included in the plurality of three-dimensional objects is represented in the three-dimensional environment;

wherein the foreground position in the rendered portion of the three-dimensional environment comprises a first associated data display area, for the first three-dimensional object;

wherein, in response to receiving the user navigational input, the display processor determines whether the first three-dimensional object, as viewed from a current position along the path, is within the specified angle; and

wherein, the display processor, in response to determining that the first three-dimensional object, as viewed from the current position along the path, is not within the specified angle, moves the first associated data display area away from the foreground position to be displayed as appearing fixed to a portion of the first three-dimensional object.