Authoring and presenting 3D presentations in augmented reality

• US 10,438,414 B2
• Filed: 01/26/2018
• Issued: 10/08/2019
• Est. Priority Date: 01/26/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bennett, Darren Alexander, Seymour, David J. W., Pereira, Charla M., Guld, Enrico William, Chu, Kin Hang, Taylor-Hell, Julia Faye, Cobb, Jonathon Burnham, Lam, Helen Joan Hem, Yang, You-Da, Wadsworth, Dean Alan, Klein, Andrew Jackson
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US15/881,538
• Publication Number:
  US 20190236842A1
• Time in Patent Office: 620 Days
• Field of Search: 345419
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts

Abstract

Various methods and systems are provided for authoring and presenting 3D presentations. Generally, an augmented or virtual reality device for each author, presenter and audience member includes 3D presentation software. During authoring mode, one or more authors can use 3D and/or 2D interfaces to generate a 3D presentation that choreographs behaviors of 3D assets into scenes and beats. During presentation mode, the 3D presentation is loaded in each user device, and 3D images of the 3D assets and corresponding asset behaviors are rendered among the user devices in a coordinated manner. As such, one or more presenters can navigate the scenes and beats of the 3D presentation to deliver the 3D presentation to one or more audience members wearing augmented reality headsets.

Claims

1. A method for authoring three dimensional (“

3D”

) presentations, the method comprising;

generating a virtual 3D environment comprising a virtual representation of a physical presentation room and 3D assets;

generating a 3D presentation that choreographs behaviors of the 3D assets into a plurality of scenes of the 3D presentation, wherein the 3D presentation comprises behavior parameters for the behaviors recorded using a 3D interface and a 2D interface operationally integrated for spatial authoring, wherein spatial authoring supports a first co-author using the 3D interface to simultaneously co-author with a second co-author using the 2D interface; and

providing the 3D presentation for coordinated rendering, among a plurality of author devices in the presentation room, of 3D images of the 3D assets and corresponding behaviors.

2. The method of claim 1, wherein the 3D presentation is configured to facilitate a selectively rendered 3D virtual image of a gaze heat map indicating a concentration of audience gaze vectors.

3. The method of claim 1, the behaviors parameters for the behaviors being set in association with temporary ownership of corresponding 3D assets.

4. The method of claim 1, additionally comprising:

including in the 3D presentation a toggled scrubbable animation of a first of the 3D images to enable one of a plurality of audience devices to update a current state of an associated one of the 3D assets to scrub the animation; and

providing for distributing the current state substantially in real-time to facilitate coordinated rendering of the scrubbable animation.

5. The method of claim 1, additionally comprising:

toggling a spatial storyboarding mode to enable one of the author devices to record a 3D drawing using a 3D interface; and

distributing the 3D drawing to facilitate coordinated rendering of the 3D drawing, wherein the 3D drawing comprises one or more segments capable of being replaced with a corresponding one of the 3D assets.

6. The method of claim 1, additionally comprising including in the 3D presentation a 3D volumizer animation, wherein the 3D volumizer animation is a 3D visualization comprising 3D image replicates of a selected 3D model, wherein the 3D volumizer animation is configured to visually illustrate data based on first behavior parameters for a first collection of the 3D assets, wherein the first behavior parameters comprise the selected 3D model and a selected quantity of the 3D image replicates of the 3D model to be generated for the first collection.

7. The method of claim 1:

wherein a first of the behaviors is configured to permit an audience device to influence a coordinated rendering of a data visualization based on a location of the audience device; and

wherein a first one of the 3D images comprises the data visualization.

8. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:

generating a virtual three-dimensional (“

3D”

) environment comprising a virtual representation of a physical presentation room and 3D assets;

receiving a 3D presentation that choreographs behaviors of the 3D assets into a plurality of scenes of the 3D presentation, wherein the 3D presentation comprises behavior parameters for the behaviors stored in the 3D presentation in association with the corresponding scenes and pre-recorded using a 3D interface and a 2D interface operationally integrated for spatial authoring, wherein each of the scenes defines a particular arrangement of the 3D assets and the behaviors; and

providing for coordinated rendering, among a presenter device and a plurality of audience devices in the presentation room, of 3D images of the 3D assets and corresponding behaviors in accordance with navigational commands from the presenter device navigating the scenes of the 3D presentation.

9. The media of claim 8, wherein the operations additionally comprise providing for selective rendering, by the presenter device, of a 3D virtual image of a gaze heat map indicating a concentration of audience gaze vectors associated with the audience devices.

10. The media of claim 8, the behaviors parameters for the behaviors being set in association with temporary ownership of corresponding 3D assets.

11. The media of claim 8, wherein the operations additionally comprise:

toggling a scrubbed animation of a first of the 3D images to enable one of the audience devices to update a current state of an associated one of the 3D assets to scrub the animation; and

distributing the current state substantially in real-time to facilitate coordinated rendering of the scrubbed animation.

12. The media of claim 8, wherein the operations additionally comprise:

toggling a drawing with light mode to enable one of the presenter or audience devices to record a time-dependent 3D drawing using a 3D interface; and

distributing the 3D drawing substantially in real-time to facilitate coordinated rendering of the 3D drawing.

13. The media of claim 8, wherein the operations additionally comprise triggering a 3D volumizer animation, wherein the 3D volumizer animation is a 3D visualization comprising 3D image replicates of a selected 3D model, wherein the 3D volumizer animation is configured to visually illustrate data based on first behavior parameters for a first collection of the 3D assets, wherein the first behavior parameters comprise the selected 3D model, a selected quantity of the 3D image replicates of the 3D model to be generated for the first collection, and physics to be applied to the replicates.

14. The media of claim 8:

wherein a first of the behaviors is configured to permit at least one of the audience devices to influence a coordinated rendering of a data visualization based on a location of the at least one audience device; and

additionally comprising triggering a first one of the 3D images comprising the data visualization.

15. A computer system comprising:

one or more hardware processors and memory configured to provide computer program instructions to the one or more hardware processors;

a three-dimensional (“

3D”

) environment generator configured to utilize the one or more hardware processors to generate a virtual 3D environment comprising a virtual representation of a physical presentation room and 3D assets;

an authoring component configured to utilize the one or more hardware processors to generate a 3D presentation that choreographs behaviors of the 3D assets into a plurality of scenes of the 3D presentation, wherein at least one of the scenes comprises a plurality of beats, wherein the 3D presentation comprises behavior parameters for the behaviors pre-recorded using a 3D interface and a 2D interface operationally integrated for spatial authoring; and

a presentation component configured to utilize the one or more hardware processors to provide the 3D presentation for coordinated rendering, among a presenter device and a plurality of audience devices in the presentation room, of 3D images of the 3D assets and corresponding behaviors.

16. The computer system of claim 15, wherein the presentation component is additionally configured to selectively render a 3D virtual image of a gaze heat map indicating a concentration of audience gaze vectors.

17. The computer system of claim 15, wherein the presentation component is additionally configured to:

toggle a scrubbed animation of a first of the 3D images to enable one of the audience devices to update a current state of an associated one of the 3D assets to scrub the animation; and

receive the current state substantially in real-time to facilitate coordinated rendering of the scrubbed animation.

18. The computer system of claim 15, wherein the presentation component is additionally configured to:

toggle a drawing with light mode to enable one of the author devices to record a time-dependent 3D drawing using a 3D interface and to enable capturing an associated audio recording; and

receive the 3D drawing and the associated audio recording to facilitate coordinated playback of the 3D drawing and the associated audio recording, wherein the playback of the 3D drawing comprises visualization trails that do not disappear.

19. The computer system of claim 15, wherein the presentation component is additionally configured to trigger a 3D volumizer animation, wherein the 3D volumizer animation is a 3D visualization comprising 3D image replicates of a selected 3D model, wherein the 3D volumizer animation is configured to visually illustrate data based on first behavior parameters for a first collection of the 3D assets, wherein the first behavior parameters comprise the selected 3D model, a selected quantity of the 3D image replicates of the 3D model to be generated for the first collection, and an idle animation to be applied to the 3D image replicates.

20. The computer system of claim 15:

wherein a first of the behaviors is configured to permit one of the audience devices to influence a coordinated rendering of a data visualization based on a location of the audience device;

wherein the presentation component is additionally configured to trigger a first one of the 3D images comprising the data visualization.