Dynamic adaption of a continuous presence videoconferencing layout based on video content

• US 9,467,657 B2
• Filed: 02/22/2016
• Issued: 10/11/2016
• Est. Priority Date: 05/21/2007
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Polycom Incorporated (Plantronics, Inc.)
• Inventors: Decker, Eran, Wagner, Niv, Barazany, Avihay, Vistuch, Roni
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US15/050,245
• Publication Number:
  US 20160173824A1
• Time in Patent Office: 232 Days
• Field of Search: 348/14.01, 348/14.07, 348/14.12, 348/14.16
• US Class Current: 1/1
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

Dynamically adapting a continuous presence (CP) layout in a videoconference enhances a videoconferencing experience by providing optimum visibility to regions of interest within the CP layout and ignoring regions of no interest. Based on the CP layout, a CP video image can be built, in which a conferee at a receiving endpoint can observe, simultaneously, several other participants'"'"' sites in the conference. For example, more screen space within the CP layout is devoted to presenting the participants in the conference and little or no screen space is used to present an empty seat, an empty room, or an unused portion of a room. Aspect ratios of segments of the CP layout (e.g., landscape vs. portrait) can be adjusted to optimally present the regions of interest. The CP layout can be adjusted as regions of interest change depending on the dynamics of the video conference.

Claims

1. A video conferencing system, for adapting a continuous presence video image, comprising:

a region of interest detector configured to detect a region of interest in video images generated by an endpoint of a plurality of endpoints;

an editor module, configured to manipulate video images; and

a control unit comprising a control module that performs actions comprising;

designing a continuous presence layout based on parameters of the region of interest; and

instructing the editor module how to manipulate at least one video image, and how to place the at least one manipulated video image in the continuous presence video image according to the continuous presence layout.

2. The video conferencing system of claim 1, wherein the control unit is a multipoint control unit.

3. The video conferencing system of claim 1, wherein instructing the editor module how to manipulate the at least one video image comprises instructing the editor module to crop or scale the video image.

4. The video conferencing system of claim 1, further comprising a video module that encodes the continuous presence video image and sends the encoded continuous presence video image toward a display unit of at least one of the plurality of endpoints.

5. The video conferencing system claim 1, wherein the region of interest detector comprises a face detector.

6. The video conferencing system claim 1, wherein the region of interest detector is associated with one of the plurality of endpoints.

7. The video conferencing system claim 1, wherein the continuous presence layout is designed based in part on a size of the region of interest.

8. The video conferencing system of claim 1, wherein the region of interest detector periodically captures a video frame generated by one of the plurality of endpoints to detect changes in the region of interest in the video image.

9. The video conferencing system of claim 8, wherein the continuous presence layout is periodically adjusted based on changes in the region of interest of at least one video image.

10. A method for adapting a video conferencing continuous presence layout, comprising:

determining a region of interest in a video image generated by a transmitting endpoint of a plurality of transmitting endpoints;

designing a continuous presence video image to be displayed at a receiving endpoint based at least in part on the region of interest;

building the continuous presence video image from video images received from the plurality of transmitting endpoints; and

delivering the continuous presence video image toward a display unit of the receiving endpoint.

11. The method of claim 10, wherein designing the continuous presence video image is based at least in part on management instructions, which are based at least in part on audio energy received from the plurality of transmitting endpoints.

12. The method of claim 10, wherein determining one or more regions of interest is executed based at least in part on data generated by a face detector.

13. The method of claim 10, wherein the continuous presence video image includes video images that have one or more regions of interest.

14. The method of claim 10, wherein designing a continuous presence video image is based at least in part on a size of the region of interest.

15. The method of claim 10, wherein determining a region of interest comprises capturing a video frame generated by the transmitting endpoint of the plurality of transmitting endpoints and determining the region of interest in the captured video frame.

16. The method of claim 10, wherein delivering the continuous presence video image comprises encoding the continuous presence video image and transmitting the encoded continuous presence video image toward the receiving endpoint.

17. The method of claim 10, wherein determining one or more regions of interest comprises periodically capturing a video frame transmitted by one of the transmitting endpoints and determining the region of interest in the captured video frame.

18. The method of claim 17, wherein designing a continuous presence video image comprises periodically adjusting the continuous presence video image based on the region of interest in the captured video frame.

19. The method of claim 18, wherein adjusting comprises changing a size of a received video image within the continuous presence video image.

20. The method of claim 18, wherein adjusting comprises eliminating a received video image from the continuous presence video image.