SYSTEM AND METHOD FOR DYNAMICALLY DISPLAYING MULTIPLE VIRTUAL AND AUGMENTED REALITY SCENES ON A SINGLE DISPLAY

• US 20140092135A1
• Filed: 10/02/2013
• Published: 04/03/2014
• Est. Priority Date: 10/02/2012
• Status: Active Grant

• Current Assignee: Aria Glassworks, Inc. (Dropbox, Inc.)
• Original Assignee: Aria Glassworks, Inc. (Dropbox, Inc.)
• Inventors: McArdle, Terrence Edward, Newhouse, Benjamin Zeis, Akbarzadeh, Amir
• Granted Patent: US 9,626,799 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

One variation of a method for dynamically displaying multiple virtual and augmented reality scenes on a single display includes determining a set of global transform parameters from a combination of user-defined inputs, user-measured inputs, and device orientation and position derived from sensor inputs; calculating a projection from a configurable function of the global transform parameters, context provided by the user and context specific to a virtual and augmented reality scene; rendering a virtual and augmented reality scene with the calculated projection on a subframe of the display; and repeating the previous two steps to render additional virtual and augmented reality scenes.

Claims

1. A method for dynamically displaying multiple virtual and augmented reality scenes on a single display comprising:

determining a set of global transform parameters from a combination of user-defined inputs, user-measured inputs, and device orientation and position derived from sensor inputs,

calculating a projection from a configurable function of the global transform parameters, context provided by the user and context specific to a virtual and augmented reality scene; and

rendering a virtual and augmented reality scene with the calculated projection on a subframe of the display; and

repeating the previous two steps to render additional virtual and augmented reality scenes

2. The method of claim 1, wherein detecting orientation from orientation sensor inputs comprises detecting a rotation matrix from a gyroscope, magnetometer and/or accelerometer.

3. The method of claim 2, wherein the rotation around the gravity vector is derived from a user dragging a finger along a vector perpendicular to the gravity vector.

4. The method of claim 3, wherein the user'"'"'s dragging accumulated momentum continues to adjust the rotation around the gravity vector after the user has stopped touching the device.

5. The method of claim 1, wherein user-defined orientation comprises a rotation composed partially of a rotation defined as the rotation that aligns the gravity vector of the virtual and augmented reality scene with a vector parallel to the surface of the device.

6. The method of claim 5, wherein the vector parallel to the surface of the device is derived from a gesture in which the user drags across the device.

7. The method of claim 1, wherein the additional context is the location of the virtual and augmented reality scene frame within the frame of the device.

8. The method of claim 7, wherein when the virtual and augmented reality scene frame is within a scrollable view and the second step of claim 1 is re-calculated dynamically as the container scrollable view is scrolled.

9. The method of claim 1, wherein the user-measured input consists of the location of the primary user'"'"'s head relative to the display.

10. The method of claim 1, wherein the additional context provides a fixed projection that causes the projection calculation to ignore portions of the global transform.

11. The method of claim 1, wherein multiple virtual and augmented reality scenes share one global resource context but maintain individual render targets.

12. The method of claim 11, wherein, when requested, the global resource context can reduce its memory consumption by purging all resources not currently being drawn on-screen.

13. The method of claim 11, wherein multiple virtual and augmented reality scenes can be rendered in one call to the graphics processor by simulating multiple render targets with a single render target and an alpha mask.

14. The method of claim 11, wherein individual render targets are represented as independent objects subclassing platform specific user interface components, which are drawn to sequentially individually or in batches by a globally shared draw loop.

15. The method of claim 1, wherein multiple virtual and augmented reality scene share one global resource context and share a single render target that is moved throughout a user interface as drawing is needed.

16. The method of claim 1, wherein virtual and augmented reality scenes which have projections which are not actively being updated are replaced with a static approximation of each respective projection.

17. The method of claim 11, wherein the rendering of virtual and augmented reality scenes occurs on separate threads separate from whichever thread that handles user input.

18. The method of claim 17, wherein the rendering of virtual and augmented reality scenes maintains separate concurrent queues for loading resources of different resolutions.

19. The method of claim 17, wherein the rendering of virtual and augmented reality scenes to the target render buffer is contingent on the scene frame being within the bounds of the device frame.

20. The method of claim 1, wherein the transform is partially composed from a zooming gesture used to transform the field of view.