Virtual Video Camera

• US 20140320592A1
• Filed: 06/11/2013
• Published: 10/30/2014
• Est. Priority Date: 04/30/2013
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Amadio, Louis, Lang, Eric Glenn, Gutmann, Michael M.
• Application Number:
  US13/915,610
• Publication Number:
  US 20140320592A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

The subject disclosure is directed towards a technology in which a virtual camera composes a plurality of views obtained from one or more physical and/or synthetic cameras into a single video stream, such as for sending to a remote telepresence client. The virtual camera may appear to applications as a single, real camera, yet provide video composed from multiple views and/or sources. Transforms may be applied at the virtual camera using hardware acceleration to generate the views, which are then composed into a rendered view and output to a video pipeline as if provided by a single video source.

Claims

1. A system comprising, a virtual camera configured to compose frames of video corresponding to a plurality of views into frames of video from a single source for rendering, the virtual camera including a compositor component having a rendering loop that processes frame data corresponding to the plurality of views into composed frame data to provide the composed frame data to a video pipeline at a desired frame rate.

2. The system of claim 1 wherein the virtual camera publishes information that represents the virtual camera as a conventional camera to an application program.

3. The system of claim 1 wherein the compositor component processes the frame data based upon a plurality of view objects that each creates at least one:

of rendering geometry, shaders or animation properties associated with the frame data corresponding to a view.

4. The system of claim 1 wherein the compositor component is further configured to perform at least one transform on at least one set of frame data.

5. The system of claim 4 wherein the compositor component performs the at least one transform using zero or more hardware accelerated transforms.

6. The system of claim 4 wherein the at least one transform comprises a de-warping transform.

7. The system of claim 4 wherein the zero or more transforms comprise a transform that processes high-resolution frame data into a higher-resolution subpart and downsampled lower-resolution frame data into a lower-resolution subpart, the higher-resolution subpart comprising one of the plurality of views and the downsampled lower-resolution frame data comprising another of the plurality of views that are composed into a frame of video from a single source for rendering.

8. The system of claim 1 wherein at least one of the plurality of views is generated by zero or more synthetic frames.

9. The system of claim 8 wherein the synthetic frame source comprises a source of at least one of:

animation, superimposed data, graphics, text, or prerecorded video frame data.

10. The system of claim 1 wherein the virtual camera is coupled to a telepresence application.

11. The system of claim 1 wherein the video pipeline is coupled to a remote renderer via a network connection.

12. The system of claim 11 further comprising a control channel associated with the remote renderer to receive instructions for controlling the virtual camera.

13. The system of claim 12 wherein the virtual camera is configured to modify a transform or a transform parameter, or both, based upon an instruction received via the control channel

14. The system of claim 1 wherein the virtual camera obtains the frame data in CPU memory, has the frame data copied from the CPU memory to GPU memory for the composition component to compose into rendered frame data, and copies the rendered frame data from the GPU memory into CPU memory.

15. A method comprising:

at a server-side computing environment, receiving sets of frame data corresponding to a plurality of views from one or more video sources;

composing a single video frame from the sets of frame data including storing frame data corresponding to the frames in GPU memory, and processing the frames in GPU memory to obtain a rendered frame in CPU memory; and

outputting the rendered frame to a remote client-side application as part of a video stream.

16. The method of claim 15 further comprising transforming the frame data received from a single camera into the plurality of views.

17. The method of claim 15 further comprising establishing a connection with each of one or more frame sources and obtaining frames from each source in CPU memory, and wherein storing the frame data in GPU memory comprises copying from the CPU memory.

18. The method of claim 17 further comprising, providing the frames to an application for processing before copying from the CPU memory.

19. One or more computer-readable storage media or logic having computer-executable instructions, which when executed perform steps, comprising, obtaining video frames from at least one physical camera or a synthetic camera, or both, processing the video frames to synthesize or compose the video frames into a resultant frame, and sending the resultant frame to a remote recipient as part of a video stream from one video source.

20. The one or more computer-readable storage media or logic of claim 19 having further computer-executable instructions comprising applying at least one transform to frame data corresponding to at least one of the video frames.