SYSTEM AND METHOD FOR SHARING VIRTUAL AND AUGMENTED REALITY SCENES BETWEEN USERS AND VIEWERS

• US 20120242798A1
• Filed: 01/10/2012
• Published: 09/27/2012
• Est. Priority Date: 01/10/2011
• Status: Active Grant

• Current Assignee: Aria Glassworks, Inc. (Dropbox, Inc.)
• Original Assignee: Aria Glassworks, Inc. (Dropbox, Inc.)
• Inventors: MCARDLE, TERRENCE EDWARD, NEWHOUSE, BENJAMIN ZEIS, AKBARZADEH, AMIR
• Granted Patent: US 8,953,022 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera

Abstract

A preferred method for sharing user-generated virtual and augmented reality scenes can include receiving at a server a virtual and/or augmented reality (VAR) scene generated by a user mobile device. Preferably, the VAR scene includes visual data and orientation data, which includes a real orientation of the user mobile device relative to a projection matrix. The preferred method can also include compositing the visual data and the orientation data into a viewable VAR scene; locally storing the viewable VAR scene at the server; and in response to a request received at the server, distributing the processed VAR scene to a viewer mobile device.

Claims

1. A method for distributing user-generated virtual and augmented reality scenes comprising:

capturing a VAR scene with a user mobile device at a location of interest, wherein the VAR scene comprises visual data and orientation data comprising a real orientation of the user mobile device relative to a projection matrix;

correlating the visual data and the orientation data at the user mobile device;

compressing at least the visual data at the user mobile device to generate a processed VAR scene; and

transmitting the processed VAR scene to a server.

2. The method of claim 1, further comprising creating a projection matrix representing an orientation of the device in a three-dimensional external frame of reference.

3. The method of claim 1, wherein the orientation data further comprises a user orientation of the user mobile device relative to a nodal point.

4. The method of claim 1, wherein the visual data comprises one or more still images.

5. The method of claim 4, wherein compressing at least the visual data comprises down-sampling the one or more still images and fitting the down-sampled one or more still images onto a single texture.

6. The method of claim 4, further comprising blending the one or more still images.

7. The method of claim 6, wherein blending the one or more still images comprises premultiplying a still image with a one of a Gaussian kernel or a smoothstep kernel.

8. The method of claim 1, wherein the visual data comprises a video comprising a series of frames.

9. The method of claim 1, wherein the VAR scene further comprises audio data.

10. The method of claim 1, further comprising locally storing the processed VAR scene on the user mobile device.

11. method of claim 1, wherein transmitting the processed VAR scene comprises transmitting metadata about the VAR scene to the server.

12. The method of claim 1, wherein transmitting the processed VAR scene comprises asynchronously transmitting the visual data and the orientation data to the server.

13. The method of claim 12, further comprising associating a universally unique identifier with at least the visual data.

14. A method for distributing user-generated virtual and augmented reality scenes comprising:

receiving at a server a VAR scene generated by a user mobile device, wherein the VAR scene comprises visual data and orientation data comprising a real orientation of the user mobile device relative to a projection matrix;

compositing the visual data and the orientation data into a viewable VAR scene;

locally storing the viewable VAR scene at the server; and

in response to a request received at the server, distributing the processed VAR scene to a viewer mobile device.

15. The method of claim 14, wherein compositing the visual data and the orientation data comprises aligning one or more still images into a substantially uniform spherical image.

16. The method of claim 15, wherein compositing the visual data and the orientation data comprises associating one or more projection matrix parameters with the substantially uniform spherical image.

17. The method of claim 14, further comprising associating a first viewer device with the viewer mobile device.

18. The method of claim 17, wherein distributing the viewable VAR scene comprises pushing the VAR scene to the viewer mobile device in response to a request received from the first viewer device.

19. A method for distributing user-generated virtual and augmented reality scenes comprising:

from a viewer device, requesting a VAR scene from a server, wherein the VAR scene comprises visual data and orientation data comprising a real orientation of a mobile device relative to a projection matrix;

receiving the VAR scene at a viewer mobile device;

determining a real orientation of the viewer mobile device relative to a projection matrix;

determining a user orientation of the viewer mobile device relative to a nodal point;

orienting the VAR scene displayable on the viewer mobile device to the viewer in response to the real orientation and the user orientation; and

displaying the VAR scene on the viewer mobile device.

20. The method of claim 19, wherein the VAR scene is processed by the server prior to receipt by the viewer mobile device.

21. The method of claim 19, further comprising creating a projection matrix at the viewer mobile device representing an orientation of the viewer mobile device in a three-dimensional external frame of reference.

22. The method of claim 19, further comprising adapting the scene displayable on the viewer mobile device to the viewer in response to a change in one of the real orientation or the user orientation.

23. The method of claim 19, wherein the viewer device and the viewer mobile device are the same device.