METHOD AND DEVICE FOR SPHERICAL RESAMPLING FOR VIDEO GENERATION

• US 20140232818A1
• Filed: 09/30/2013
• Published: 08/21/2014
• Est. Priority Date: 02/19/2013
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: CARR, G. Peter K., MISTRY, Michael Peter K., MATTHEWS, Iain, SHEIKH, Yaser
• Granted Patent: US 10,165,157 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2224 related to virtual studio applications

Abstract

A system and method generates a broadcast image. The method includes receiving a first image captured by a first camera having a first configuration incorporating a first center of projection. The method includes determining a first mapping from a first image plane of the first camera onto a sphere. The method includes determining a second mapping from the sphere to a second image plane of a second virtual camera having a second configuration incorporating the first center of projection of the first configuration. The method includes generating a second image based upon the first image and a concatenation of the first and second mappings.

Claims

1. A method, comprising:

receiving a first image captured by a first camera having a first configuration incorporating a first center of projection;

determining a first mapping from a first image plane of the first camera onto a sphere;

determining a second mapping from the sphere to a second image plane of a second virtual camera having a second configuration incorporating the first center of projection of the first configuration; and

generating a second image based upon the first image and a concatenation of the first and second mappings.

2. The method of claim 1, wherein the first camera remains stationary, and the first configuration is fixed.

3. The method of claim 1, wherein the first camera is a Pan-Tilt-Zoom (PTZ) camera, and the first configuration is determined by an internal state of the PTZ camera.

4. The method of claim 3, wherein the PTZ camera is steered by a control loop based on a discrepancy between a desired first configuration and an actual first configuration.

5. The method of claim 4, wherein at least one of the desired first and second configurations is determined by an analysis of sensor data.

6. The method of claim 4, wherein the at least one of the desired first and second configurations is determined by a human operator.

7. The method of claim 1, wherein the determination of the second configuration is delayed relative to the determination of the first configuration.

8. The method of claim 1, wherein the second configuration is restricted from deviating from the first configuration beyond a predetermined value.

9. The method of claim 4, wherein the control loop is based on a discrepancy between a desired first configuration, an actual first configuration, a desired second configuration, and an actual second configuration.

10. The method of claim 9, wherein the desired second configuration and actual second configuration are delayed relative to the desired first configuration and actual first configuration.

11. A device, comprising:

a processor coupled to a memory, wherein the processor is programmed to generate a broadcast image by;

receiving a first image captured by a first camera having a first configuration incorporating a first center of projection;

determining a first mapping from a first image plane of the first camera onto a sphere;

determining a second mapping from the sphere to a second image plane of a second virtual camera having a second configuration incorporating the first center of projection of the first configuration; and

generating a second image based upon the first image and a concatenation of the first and second mappings.

12. The device of claim 11, wherein the first camera remains stationary, and the first configuration is fixed.

13. The device of claim 11, wherein the first camera is a Pan-Tilt-Zoom (PTZ) camera, and the first configuration is determined by an internal state of the PTZ camera.

14. The device of claim 13, wherein the PTZ camera is steered by a control loop based on a discrepancy between a desired first configuration and an actual first configuration.

15. The device of claim 14, wherein at least one of the desired first and second configurations is determined by an analysis of sensor data.

16. The device of claim 14, wherein the at least one of the desired first and second configurations is determined by a human operator.

17. The device of claim 11, wherein the determination of the second configuration is delayed relative to the determination of the first configuration.

18. The device of claim 11, wherein the second configuration is restricted from deviating from the first configuration beyond a predetermined value.

19. The device of claim 14, wherein the control loop is based on a discrepancy between a desired first configuration, an actual first configuration, a desired second configuration, and an actual second configuration.

20. A non-transitory computer readable storage medium including a set of instructions executable by a processor, the set of instructions operable to:

receive a first image captured by a first camera having a first configuration incorporating a first center of projection;

determine a first mapping from a first image plane of the first camera onto a sphere;

determine a second mapping from the sphere to a second image plane of a second virtual camera having a second configuration incorporating the first center of projection of the first configuration; and

generate a second image based upon the first image and a concatenation of the first and second mappings.