Virtual camera control using motion control systems for augmented reality

• US 8,885,022 B2
• Filed: 01/04/2010
• Issued: 11/11/2014
• Est. Priority Date: 01/04/2010
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Gay, Michael, Thiel, Aaron
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): LUO, KATE H
• Application Number:
  US12/655,683
• Publication Number:
  US 20110164030A1
• Time in Patent Office: 1,772 Days
• Field of Search: None
• US Class Current: 348/47
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

There is provided a system and method for integrating a virtual rendering system and a motion control system to provide an augmented reality. There is provided a method for integrating a virtual rendering system and a motion control system for outputting a composite render to a display, the method comprising obtaining, from the motion control system, a robotic camera configuration of a robotic camera in a real environment, programming the virtual rendering system using the robotic camera configuration to correspondingly control a virtual camera in a virtual environment, obtaining a virtually rendered feed using the virtual camera, capturing a video capture feed using the robotic camera, rendering the composite render by processing the feeds, and outputting the composite render to the display.

Claims

1. A method for integrating a virtual rendering system and a motion control system to output a composite render to a display, the method comprising:

obtaining, from the motion control system, a first camera configuration, wherein the first camera configuration includes camera position data, camera field of view orientation data, camera movement data and camera lens characteristics data;

programming the virtual rendering system using the first camera configuration to correspondingly control a first virtual camera in a virtual environment;

obtaining, from the virtual rendering system, a first virtually rendered feed of the virtual environment using the first virtual camera;

capturing, from a video capture system, a first video capture feed of a real environment using the first camera configuration;

rendering the composite render, based on a target broadcast region, by processing the first virtually rendered feed and the first video capture feed, wherein the composite render includes a real tracked object and at least one virtual player in the virtual environment; and

outputting the composite render to the display.

2. The method of claim 1, wherein the first motion path includes a three-dimensional position with respect to time.

3. The method of claim 1, wherein the first motion path includes a camera orientation or field of view with respect to time.

4. The method of claim 1, wherein the camera lens characteristics data includes camera lens parameters.

5. The method of claim 1, wherein the first motion path is based on a predetermined path.

6. The method of claim 1, wherein the first motion path is based on a previously recorded path.

7. The method of claim 1, wherein the first motion path is based on a video analysis using the video capture system.

8. The method of claim 1 further comprising prior to the programming of the virtual rendering system, modifying the first motion path according to a user input.

9. The method of claim 1 further comprising prior to the rendering of the composite render:

obtaining, from the motion control system, a second camera configuration including a second motion path in the real environment;

programming the virtual rendering system using the second camera configuration to correspondingly control a second virtual camera in the virtual environment;

obtaining, from the virtual rendering system, a second virtually rendered feed of the virtual environment using the second virtual camera;

capturing, from a video capture system, a second video capture feed of the real environment using the second camera configuration;

wherein the rendering of the composite render further processes the second virtually rendered feed and the second video capture feed.

10. A rendering controller for outputting a composite render to a display, the rendering device comprising:

a processor configured to;

obtain, from a motion control system, a first camera configuration, wherein the first camera configuration includes camera position data, camera field of view orientation data, camera movement data and camera lens characteristics data;

program a virtual rendering system using the first camera configuration to correspondingly control a first virtual camera in a virtual environment;

obtain, from the virtual rendering system, a first virtually rendered feed of the virtual environment using the first virtual camera;

capture, from a video capture system, a first video capture feed of a real environment using the first camera configuration;

render the composite render, based on a target broadcast region, by processing the first virtually rendered feed and the first video capture feed, wherein the composite render includes a real tracked object and at leastone virtual player in the virtual environment; and

output the composite render to the display.

11. The rendering controller of claim 10, wherein the first motion path includes a three-dimensional position with respect to time.

12. The rendering controller of claim 10, wherein the first motion path includes a camera orientation or field of view with respect to time.

13. The rendering controller of claim 10, wherein the camera lens characteristics data includes camera lens parameters.

14. The rendering controller of claim 10, wherein the first motion path is based on a predetermined path.

15. The rendering controller of claim 10, wherein the first motion path is based on a previously recorded path.

16. The rendering controller of claim 10, wherein the first motion path is based on a video analysis using the video capture system.

17. The rendering controller of claim 10, wherein prior to the programming of the virtual rendering system, the processor is configured to modify the first motion path according to a user input.

18. The rendering controller of claim 10, wherein prior to the rendering of the composite render the processor is configured to:

obtain, from the motion control system, a second camera configuration including a second motion path in the real environment;

program the virtual rendering system using the second camera configuration to correspondingly control a second virtual camera in the virtual environment;

obtain, from the virtual rendering system, a second virtually rendered feed of the virtual environment using the second virtual camera; and

capture, from a video capture system, a second video capture feed of the real environment using the second camera configuration;

wherein the processor is configured to render the composite render by further processing the second virtually rendered feed and the second video capture feed.