Mixed reality viewer system and method

• US 10,572,000 B2
• Filed: 11/20/2017
• Issued: 02/25/2020
• Est. Priority Date: 03/06/2017
• Status: Active Grant

• Current Assignee: Universal City Studios LLC (Comcast Corporation)
• Original Assignee: Universal City Studios LLC (Comcast Corporation)
• Inventors: Cossairt, Travis, Yeh, Wei
• Primary Examiner(s): Duffy, David
• Assistant Examiner(s): Doshi, Ankit B
• Application Number:
  US15/818,463
• Publication Number:
  US 20180250589A1
• Time in Patent Office: 827 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/843 involving concurrently two or more players on the same game device, e.g. requiring the use of a plurality of controllers or of a specific view of game data for each player
• CPC Class Codes: A63F 2300/6653 for altering the visibility of an object, e.g. preventing the occlusion of an object, partially hiding an object
• CPC Class Codes: A63G 27/00 Russian swings; Great wheels, e.g. Ferris wheels
• CPC Class Codes: A63G 31/00 Amusement arrangements hoop exercising apparatus driven by the user A63B19/00
• CPC Class Codes: A63G 31/16 creating illusions of travel
• CPC Class Codes: A63G 7/00 Up-and-down hill tracks; Switchbacks
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0145 creating an intermediate image
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0101 : characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/017 : Head mounted
• CPC Class Codes: G02B 27/0172 : characterised by optical features
• CPC Class Codes: G06F 1/163 : Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2203/04806 : Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 : Head tracking input arrangements
• CPC Class Codes: G06F 3/04815 : Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 : for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 : Multi-user, collaborative environment
• CPC Class Codes: H04N 13/344 : with head-mounted left-right displays

Abstract

A mixed reality viewing system includes a viewer configured to be secured to a steady platform and operable by a user to view a theme through the viewer. The viewer includes a display device, a user interface comprising a zoom control, and at least one sensor comprising at least one camera. The mixed reality viewing system also includes a computer graphics generation system communicatively coupled to the viewer. The computer graphics generation system is configured to generate streaming media of a real world environment based on image data captured via the at least one camera of the viewer, generate augmented reality graphics, virtual reality graphics, or both, superimposed on the streaming media of the real world environment, and transmit the streaming media of the real world environment along with the superimposed augmented reality graphics, virtual reality graphics, or both, to be displayed on the display device of the viewer.

Claims

1. A mixed reality viewing system, comprising:

a viewer configured to be secured to a steady platform and operable by a user to view a theme through the viewer, wherein the viewer comprises;

a display device;

a user interface comprising a zoom control; and

at least one sensor, the at least one sensor comprising at least one camera; and

a computer graphics generation system communicatively coupled to the viewer and configured to;

determine a magnification of the zoom control;

generate streaming media including augmented reality graphics superimposed on image data of a real world environment captured via the at least one camera of the viewer in response to determining that the zoom control is zoomed to a first magnification;

generate additional streaming media of a virtual reality environment in response to determining that the zoom control is zoomed to a second magnification different than the first magnification; and

transmit the streaming media or the additional streaming media to be displayed on the display device of the viewer.

2. The mixed reality viewing system of claim 1, wherein the computer graphics generation system is configured to:

transmit the streaming media including the augmented reality graphics superimposed on the image data of the real world environment to be displayed on the display device in response to determining that the zoom control is zoomed to the first magnification; and

transmit the additional streaming media of the virtual reality environment to be displayed on the display device in response to determining that the zoom control is zoomed to the second magnification, wherein the second magnification is higher than the first magnification.

3. The mixed reality viewing system of claim 1, wherein the viewer comprises:

a viewer portion comprising the display device and the user interface, wherein the viewer portion is rotatable by the user to adjust an angle of view; and

a fixture portion rotatably coupled to the viewer portion and fixedly coupling the viewer to the steady platform.

4. The mixed reality viewing system of claim 1, wherein the display device of the viewer comprises a first display and a second display, and wherein the first display is configured to display the streaming media or the additional streaming media to a first eye of the user and the second display is configured to display the streaming media or the additional streaming media to a second eye of the user.

5. The mixed reality viewing system of claim 4, wherein the first display and the second display include respective characteristic dimensions that enable guests other than the user to view the streaming media or the additional streaming media on the first display, the second display, or both.

6. The mixed reality viewing system of claim 1, wherein the viewer comprises a speaker.

7. The mixed reality viewing system of claim 1, wherein each camera of the at least one camera of the viewer comprises a digital high speed camera having a frame-rate of at least sixty frames per second with a horizontal resolution of at least 4000 pixels.

8. The mixed reality viewing system of claim 1, wherein the at least one sensor comprises one or more orientation and position sensors, one or more multi degree-of-freedom inertial measurement units, one or more light sensors, one or more presence sensors, one or more antennas, or a combination thereof.

9. The mixed reality viewing system of claim 1, comprising a remote display device, wherein the computer graphics generation system is configured to transmit the streaming media or the additional streaming media to be displayed on the remote display device.

10. The mixed reality viewing system of claim 1, wherein the computer graphics generation system has a model of the real world environment surrounding the viewer, wherein the model comprises spatial information of the viewer relative to the real world environment.

11. A fixed-position viewer, comprising:

a display configured to display a streaming media to a user;

a camera configured to capture image data of a real world environment surrounding the fixed-position viewer;

at least one sensor configured to collect information for generation of the streaming media;

a user interface comprising a zoom control configured to enable the user to zoom-in and zoom-out on the streaming media displayed on the display; and

a computer graphics generation system configured to;

receive the image data of the real world environment and the information collected by the at least one sensor;

render the streaming media based at least on the image data and the information; and

transition the streaming media from an augmented reality environment to a virtual reality environment in response to determining that the zoom control is zoomed beyond a pre-determined threshold magnification.

12. The fixed-position viewer of claim 11, wherein the camera comprises a digital high speed camera with a frame rate of at least sixty frames per second and a horizontal resolution of at least 4000 pixels.

13. The fixed-position viewer of claim 11, comprising one or more speakers configured to provide audio effects corresponding to the streaming media displayed on the display.

14. The fixed-position viewer of claim 11, wherein the at least one sensor comprises one or more orientation and position sensors, one or more multi degree-of-freedom inertial measurement units, one or more light sensors, one or more presence sensors, one or more antennas, or a combination thereof.

15. The fixed-position viewer of claim 11, wherein the display includes a characteristic dimension that enables one or more guests other than the user of the fixed-position viewer to view the streaming media on the display.

16. The fixed-position viewer of claim 11, wherein the information collected by the at least one sensor comprises user information, and wherein at least some content of the streaming media displayed on the display is customized to the user based on the user information.

17. A method, comprising:

receiving real-time data from at least one sensor and a user interface of a fixed-position viewer;

analyzing the real-time data via a computer graphics generation system;

generating gaming effects, via the computer graphics generation system, based on the real-time data, wherein generating the gaming effects via the computer graphics generation system comprises;

generating streaming media including augmented reality graphics overlaid onto a visualization of a real world environment in response to determining that a zoom control of the user interface is zoomed to a first magnification; and

generating additional streaming media including virtual reality graphics that replace the visualization of the real world environment in response to determining that the zoom control of the user interface is zoomed to a second magnification that is different than the first magnification;

transmitting, via the computer graphics generation system, the gaming effects including the streaming media or the additional streaming media to the fixed-position viewer; and

displaying, via the computer graphics generation system, the streaming media or the additional streaming media on a display of the fixed-position viewer.

18. The method of claim 17, wherein the second magnification is greater than the first magnification.

19. The method of claim 17, wherein receiving the real-time data comprises receiving, via the computer graphics generation system, data collected via at least one or more orientation and position sensors, one or more multi degree-of-freedom inertial measurement units, one or more light sensors, one or more presence sensors, or one or more antennas of the fixed-position viewer.

20. The method of claim 17, comprising:

transmitting, via the computer graphics generation system, auxiliary streaming media or auxiliary additional streaming media to an additional fixed-position viewer, wherein the auxiliary streaming media and the auxiliary additional streaming media comprise visualization content corresponding to the streaming media and the additional streaming media, respectively, displayed on the fixed-position viewer from a perspective of the additional fixed-position viewer; and

displaying, via the computer graphics generation system, the auxiliary streaming media or the auxiliary additional streaming media on a respective display of the additional fixed-position viewer.