System and method for virtual camera control using motion control systems for augmented three dimensional reality

• US 8,885,023 B2
• Filed: 09/01/2010
• Issued: 11/11/2014
• Est. Priority Date: 09/01/2010
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Thiel, Aaron, Gay, Michael, Bailey, Anthony
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): LUO, KATE H
• Application Number:
  US12/807,330
• Publication Number:
  US 20120050256A1
• Time in Patent Office: 1,532 Days
• Field of Search: None
• US Class Current: 348/47
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems

Abstract

There is provided a system and method for integrating a virtual rendering system and a motion control system to provide an augmented three-dimensional reality. There is provided a method for integrating a virtual rendering system and a motion control system for outputting a composite three-dimensional render to a three-dimensional display, the method comprising obtaining, from the motion control system, a robotic three-dimensional camera configuration of a robotic three-dimensional camera in a real environment, programming the virtual rendering system using the robotic three-dimensional camera configuration to correspondingly control a virtual three-dimensional camera in a virtual environment, obtaining a virtually rendered three-dimensional feed using the virtual three-dimensional camera, capturing a video capture three-dimensional feed using the robotic three-dimensional camera, rendering the composite three-dimensional render by processing the feeds, and outputting the composite three-dimensional render to the three-dimensional display.

Claims

1. A method for integrating a virtual rendering system and a motion control system to output a composite three-dimensional render to a three-dimensional display, the method comprising:

obtaining, from the motion control system, a first three-dimensional camera configuration, wherein the first three-dimensional camera configuration includes camera position data, camera field of view orientation data, camera movement data and camera lens characteristics data;

programming the virtual rendering system using the first three-dimensional camera configuration to correspondingly control a first virtual three-dimensional camera in a virtual environment;

obtaining, from the virtual rendering system, a first virtually rendered three-dimensional feed of the virtual environment using the first virtual three-dimensional camera;

capturing, from a video capture system, a first video capture three-dimensional feed of a real environment using the first three-dimensional camera configuration;

rendering the composite three-dimensional render, based on a target broadcast region, by processing the first virtually rendered three-dimensional feed and the first video capture three-dimensional feed, wherein the composite three-dimensional render includes simulating at least one virtual player with a real tracked object; and

outputting the composite three-dimensional render to the three-dimensional display.

2. The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path includes a three-dimensional position with respect to time.

3. The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path includes a camera orientation or field of view with respect to time.

4. The method of claim 1, wherein the camera lens characteristics data includes three-dimensional camera lens parameters.

5. The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a predetermined path.

6. The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a previously recorded path.

7. The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a video analysis using the video capture system.

8. The method of claim 1, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and wherein the method further comprising prior to the programming of the virtual rendering system, modifying the first motion path according to a user input.

9. The method of claim 1 further comprising prior to the rendering of the composite three-dimensional render:

obtaining, from the motion control system, a second three-dimensional camera configuration including a second motion path in the real environment;

programming the virtual rendering system using the second three-dimensional camera configuration to correspondingly control a second virtual three-dimensional camera in the virtual environment;

obtaining, from the virtual rendering system, a second virtually rendered three-dimensional feed of the virtual environment using the second virtual three-dimensional camera;

capturing, from a video capture system, a second video capture three-dimensional feed of the real environment using the second three-dimensional camera configuration;

wherein the rendering of the three-dimensional composite render further processes the second virtually rendered three-dimensional feed and the second video capture three-dimensional feed.

10. A rendering controller for outputting a composite three-dimensional render to a three-dimensional display, the rendering controller comprising:

a processor configured to;

obtain, from a motion control system, a first three-dimensional camera configuration, wherein the first three-dimensional camera configuration includes camera position data, camera field of view orientation data, camera movement data and camera lens characteristics data;

program a virtual rendering system using the first three-dimensional camera configuration to correspondingly control a first virtual three-dimensional camera in a virtual environment;

obtain, from the virtual rendering system, a first virtually rendered three-dimensional feed of the virtual environment using the first virtual three-dimensional camera;

capture, from a video capture system, a first video capture three-dimensional feed of a real environment using the first three-dimensional camera configuration;

render the composite three-dimensional render, based on a target broadcast region, by processing the first virtually rendered three-dimensional feed and the first video capture three-dimensional feed, wherein the composite three-dimensional render includes simulating at least one virtual player with a real tracked object; and

output the composite three-dimensional render to the three-dimensional display.

11. The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path includes a three-dimensional position with respect to time.

12. The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path includes a camera orientation or field of view with respect to time.

13. The rendering controller of claim 10, wherein the camera lens characteristics data includes three-dimensional camera lens parameters.

14. The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a predetermined path.

15. The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a previously recorded path.

16. The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and the first motion path is based on a video analysis using the video capture system.

17. The rendering controller of claim 10, wherein the first three-dimensional camera configuration includes a first motion path in the real environment, and wherein prior to the programming of the virtual rendering system, the processor is configured to modify the first motion path according to a user input.

18. The rendering controller of claim 10, wherein prior to the rendering of the composite three-dimensional render the processor is configured to:

obtain, from the motion control system, a second three-dimensional camera configuration including a second motion path in the real environment;

program the virtual rendering system using the second three-dimensional camera configuration to correspondingly control a second virtual three-dimensional camera in the virtual environment;

obtain, from the virtual rendering system, a second virtually rendered three-dimensional feed of the virtual environment using the second virtual three-dimensional camera; and

capture, from a video capture system, a second video capture three-dimensional feed of the real environment using the second three-dimensional camera configuration;

wherein the processor is configured to render the composite three-dimensional render by further processing the second virtually rendered three-dimensional feed and the second video capture three-dimensional feed.