Compositing Video Streams

• US 20110025819A1
• Filed: 06/30/2009
• Published: 02/03/2011
• Est. Priority Date: 06/30/2008
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Allen, Bradley L., Darocher, Michael D., Gorzynski, Mark E.
• Granted Patent: US 8,432,431 B2
• Time in Patent Office: Days
• US Class Current: 348/14.07
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Methods and apparatus for compositing multiple video streams onto a background image having at least one object while keeping at least one of a proper perspective, order, and substantial alignment to the object of the multiple video streams based on the rules of a common layout. The background is defined as a meeting space that fills in gaps between the multiple video streams with appropriate structure of the object. The background creates the context of the meeting space.

Claims

1. A computer for a virtual meeting having computer readable memory configured to allow the computer to perform steps, comprising:

receiving information from a host to setup rules for a common layout of multiple video streams received at the computer;

creating a composited image by compositing the multiple video streams onto a background having an object while keeping at least one of a proper perspective, order, and substantial alignment to the object of the multiple video streams, based on the rules of the common layout, wherein the background is a meeting space that fills in gaps between the multiple video streams with appropriate structure of the object and which the background creates the context of the meeting space; and

displaying the composited image on a display.

2. The computer of claim 1, wherein the computer readable memory is further configured to cause the computer to perform the step of requesting the rules for the common layout from the host for a given meeting type and a given number of participants.

3. The computer of claim 2, wherein the computer readable memory is further configured to cause the computer to perform at least one of the step of matting to replace desired elements and matting to add desired elements onto video streams that do not meet the rules for the common layout.

4. The computer of claim 1, wherein the computer readable memory is further configured to cause the computer to perform the step of generating at least one video stream based on the rules for the common layout using at least one of camera positioning, zoom, angle, perspective, field of capture, and/or aspect ratio.

5. The computer of claim 1, wherein the computer readable memory is further configured to cause the computer to perform at least one of the steps of post-capture editing, matting to replace desired elements, and matting to add desired elements to at least one of the received multiple video streams and at least one generated video stream to conform to rules for the common layout.

6. The computer of claim 1, wherein the background contains multiple objects for the multiple video streams to align to.

7. The computer of claim 6, wherein the objects have at least one of edges that are softened and elements that fade from darker to lighter.

8. A method for providing a virtual meeting common layout service, comprising:

receiving a request from a client to organize a unified meeting space event based on at least one of an event type, a list of event endpoints, and endpoint capabilities;

determining background rules for a background containing an alignment object for each of the one or more of the event endpoints to substantially align video streams to; and

sending the determined background rules to each event endpoint as rules for a common layout of the unified meeting space event.

9. The method of claim 8, further comprising determining rules for video stream management between the event endpoints and/or a host based on the organized unified meeting space and sending the determined rules for video stream alignment as part of the rules for the common layout.

10. The method of claim 8, further comprising receiving video streams from one or more event endpoints and creating at least one composited stream based on the rules for the common layout.

11. The method of claim 10, further comprising sending the composited stream to another event endpoint for display.

12. The method of claim 8, further comprising receiving a video stream from at least one event endpoint and matting it to at least one of replace desired elements and add desired elements consistent with the rules for the common layout.

13. The method of claim 12, further comprising sending the matted video stream to another event endpoint for display.

14. The method of claim 8, wherein the background contains multiple objects to align the video streams to.

15. The method of claim 9, wherein the alignment object has at least one of edges that are softened and elements that fade from darker to lighter.

16. A hosting server for video conferencing, comprising:

a server computer for communication with a network configured to receive requests from clients and to provide responses to the clients, the requests causing the server to perform one or more of the following steps, comprising;

defining a common layout for two or more endpoints in communication with the network and participating in a video conference, wherein the common layout corresponds to a selected meeting type that defines a meeting topology with consistent geometry and relative positioning of the two or more endpoints;

defining one or more background images conforming to the common layout, wherein a background image for each participating endpoint depicts a view of a meeting space of the selected meeting type from a point of view of that participating endpoint;

defining geometric rules for video streams to be received from the two or more endpoints, wherein the geometric rules conform to the common layout; and

defining for each participating endpoint relative locations within a respective background image for placement of video streams from one or more of the other participating endpoints, wherein the relative locations within a background image for placement of video streams comprises defining locations such that edges of at least one element of each of the video streams is placed to align with an edge of at least one corresponding element of that background image; and

wherein the server computer returns the defined results of the request to the respective client.

17. The hosting server of claim 16, further comprising:

wherein the server is configured to provide for editing video streams for participating endpoints that do not conform to the geometric rules.

18. The hosting server of claim 16, wherein the requests further cause the server to perform the step:

compositing video streams from one or more of the endpoints onto a background image conforming to the common layout.

19. The hosting server of claim 16, wherein defining one or more background images comprises defining one or more background images having softened edges of one or more elements of the background images.

20. The hosting server of claim 16, wherein defining one or more background images comprises defining one or more background images having one or more elements of the background images that fade from darker to lighter.