System and method for embedding and viewing media files within a virtual and augmented reality scene

• US 9,118,970 B2
• Filed: 03/02/2012
• Issued: 08/25/2015
• Est. Priority Date: 03/02/2011
• Status: Active Grant

• Current Assignee: Aria Glassworks, Inc. (Dropbox, Inc.)
• Original Assignee: Aria Glassworks, Inc. (Dropbox, Inc.)
• Inventors: Newhouse, Benjamin Zeis, McArdle, Terrence Edward
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US13/411,347
• Publication Number:
  US 20120236029A1
• Time in Patent Office: 1,271 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94

Abstract

A preferred method for viewing embedded media in a virtual and augmented reality (VAR) scene can include at a viewer device, defining a real orientation of the viewer device relative to a projection matrix; and orienting a VAR scene on the viewer device in response to the real orientation in block, in which the VAR scene includes one or both of visual data and orientation data. The preferred method can further include selecting a media file in the VAR scene, wherein the media file is selected at a media location correlated at least to the real orientation of the viewer device; and activating the media file in the VAR scene at the media location. The preferred method and variations thereof functions to allow a viewer to interact with media that is embedded, tagged, linked, and/or associated with a VAR scene viewable on the viewer device.

Claims

1. A method comprising:

determining a first orientation of a device within a three-dimensional space;

determining a second orientation of the device relative to a nodal point, wherein the nodal point is at a first position within the three-dimensional space external to the device and corresponds to a user using the device;

displaying a first portion of a virtual and augmented reality (VAR) scene on the device based on the first orientation and the second orientation, wherein the first portion of the VAR scene corresponds to an overlap between a viewing frustum projected from the nodal point and the VAR scene, and wherein the VAR scene includes visual data and orientation data;

detecting a change in the first orientation of the device and detecting a change in a position of the nodal point from the first position to a second position within the three-dimensional space, wherein the change in the position of the nodal point causes the viewing frustum to overlap with a second portion of the VAR scene;

displaying the second portion of the VAR scene on the device based on the detected change in the first orientation of the device and the detected change in the position of the nodal point;

receiving a selection of a media location in the second portion of the VAR scene, wherein the selected media location is correlated to the first orientation of the device and the second orientation of the device when the selection is received; and

embedding a media file in the VAR scene at the media location, wherein the media file is subsequently activatable when the device is oriented such that the second portion of the VAR scene is displayed.

2. The method of claim 1, wherein the selected media location is further correlated to a depth value within the VAR scene.

3. The method of claim 2, wherein the depth value is determined in response to a user input.

4. The method of claim 1, wherein the depth value is determined in response to the first orientation and the second orientation of the device while the selection of the media location occurs.

5. The method of claim 1, wherein the media file includes one of:

an image file, a video file, an audio file, a second VAR scene, a second rendering of the VAR scene, or a hyperlink.

6. The method of claim 5, wherein activation of the hyperlink directs the device to the second VAR scene.

7. The method of claim 5, wherein selecting a media location of the second VAR scene includes determining the location of the second VAR scene and locating the second VAR scene at its location within the VAR scene.

8. The method of claim 1, further comprising:

associating a media activation parameter with the media file.

9. The method of claim 8, wherein the media activation parameter includes an activation area such that within the VAR scene overlap between the activation area and the media file causes activation of the media file.

10. A method comprising:

determining a first orientation of a device within a three-dimensional space;

determining a second orientation of the device relative to a nodal point, wherein the nodal point is at a first position within the three-dimensional space external to the device and corresponds to a user using the device;

displaying a first portion of a virtual and augmented reality (VAR) scene on the device based on the first orientation and the second orientation, wherein the first portion of the VAR scene corresponds to an overlap between a viewing frustum projected from the nodal point and the VAR scene, and wherein the VAR scene comprises visual data and orientation data;

detecting a change in the first orientation of the device and detecting a change in a position of the nodal point from the first position to a second position within the three-dimensional space, wherein the change in the position of the nodal point causes the viewing frustum to overlap with a second portion of the VAR scene;

displaying the second portion of the VAR scene on the device based on the detected change in the first orientation of the device and the detected change in the position of the nodal point;

receiving a selection of a media file embedded at a media location in the second portion of the VAR scene, wherein the media location is correlated to the first orientation of the device and the second orientation of the device when the media file was embedded, and wherein the media file is activatable when the device is oriented such that the second portion of the VAR scene is displayed; and

activating the media file in the VAR scene at the media location.

11. The method of claim 10, wherein the selected media location is further correlated to a depth value within the VAR scene.

12. The method of claim 10, wherein the media file includes one of:

an image file, a video file, an audio file, a second VAR scene, a second rendering of the VAR scene, or a hyperlink.

13. The method of claim 12, wherein activation of the hyperlink directs the device to the second VAR scene.

14. The method of claim 12, wherein a media location of the second VAR scene includes a representation of the second VAR scene at its real location within the VAR scene.

15. The method of claim 10, wherein activating the media file includes activating the media file in response to an orientation of the device such that an activation area in the VAR scene overlaps the media file.