Panoramic image placement to minimize full image interference

• US 10,182,208 B2
• Filed: 12/12/2017
• Issued: 01/15/2019
• Est. Priority Date: 10/05/2015
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Polycom Incorporated (Plantronics, Inc.)
• Inventors: Nimri, Alain Elon, Gao, Shu, Schaefer, Stephen, Murphy, Robert
• Primary Examiner(s): Woo, Stella L.
• Application Number:
  US15/839,476
• Publication Number:
  US 20180167584A1
• Time in Patent Office: 399 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 21/028 using properties of sound source
• CPC Class Codes: G10L 25/72 for transmitting results of analysis
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

An automatic process for producing professional, directed, production crew quality, video for videoconferencing is described. Rule based logic is integrated into an automatic process for producing director quality video for videoconferencing. An automatic process can include a method for composing a display for use in a video system having an active talker video stream and a panoramic view video stream having more than one person in video. The method can include determining a region of interest in a panoramic view video using motion detection and presence sensors, and preparing the panoramic view video by centering the region of interest and by zooming towards the region of interest, based upon the location of persons in the panoramic view video. The method includes determining placement of panoramic view video on a composite display to prevent the panoramic view video overlaying display of an active talker on the active talker video stream.

Claims

1. A method for composing a display for use in a video system, the method comprising:

receiving, at a processor, motion data from at least one motion detection device;

accepting, at the processor, presence data from at least one sensor;

receiving, at the processor, video data from cameras;

determining, under control of the processor, where to place depictions of individuals within a panoramic strip based, at least in part, on the received motion data, the accepted presence data, and the received video data, such that the panoramic strip contains no depictions of empty seats;

identifying an active talker using the received motion data, accepted presence data, and received video data;

determining, at the processor, where to place the panoramic strip within a video layout, so as to prevent the panoramic strip overlaying depiction of the active talker in an active talker video stream while maintaining proportionality of the depiction within the active talker video stream.

2. The method of claim 1, further comprising placing the panoramic strip in one of three locations, a top of the video layout, a bottom of the video layout, or hidden.

3. The method of claim 1, further comprising placing depictions of individuals in a central region of the panoramic strip.

4. The method of claim 3, wherein placing the depictions of individuals in the center of the panoramic strip comprises left-right balancing of individuals therein.

5. The method of claim 1, further comprising two or more panoramic strips are positioned in a stacked configuration, and depictions of individuals are placed in the center of the strips.

6. The method of claim 1, further comprising placing depictions of individuals in the panoramic strip such that no views which do not contain individuals are included in the panoramic strip.

7. The method of claim 1, further comprising placing the depictions of individuals in the panoramic strip such that views which do not contain individuals are not placed between views containing individuals.

8. The method claim 1, wherein maintaining proportionality of the depiction within the active talker video stream comprises maintaining scale of the depiction within the active talker video stream.

9. The method of claim 1, further comprising detecting two active talkers and placing zoomed views of the two active talkers in the panoramic strip such that the zoomed views containing the two active talkers are adjacent, and faces of the two active talkers point towards one another.

10. A video production system for a room, the video production system comprising:

a memory for storing program code for execution; and

a processor which executes stored program code and is configured to;

receive motion detection data;

accept presence sensor data;

receive information regarding video recorded by cameras in a room;

center, using the received motion detection data, accepted presence sensor data and received information, individuals in a room view;

zoom onto the centered individuals of the room view in one or more zoomed views;

obtain sound source localization data and active talker information corresponding to at least one active talker;

determine, using the obtained sound source localization data, the obtained active talker information and the received motion detection data, where, within a panoramic strip in a video layout, to place the zoomed views of individuals in the room; and

place the panoramic strip within the video layout, so as to prevent the panoramic strip overlaying display of the active talker, while maintaining proportionality of the display of the active talker within the video layout.

11. The video production system of claim 10, wherein the processor is further configured to place the panoramic strip in one of three locations, a top of the video layout, a bottom of the video layout, or hidden.

12. The video production system of claim 10, wherein the processor is further configured to place the zoomed views in the center of the panoramic strip.

13. The video production system of claim 12, wherein placing the zoomed views in the center of the panoramic strip comprises left-right balancing of individuals in the room view.

14. The video production system of claim 10, wherein the processor is further configured to position two or more panoramic strips in a stacked configuration in which the zoomed views are placed in the center of the strips.

15. The video production system of claim 10, wherein the processor is further configured to place the zoomed views in the panoramic strip such that no views which do not contain individuals are included in the panoramic strip.

16. The video production system of claim 10, wherein the processor is further configured to place the zoomed views in the panoramic strip such that views which do not contain individuals are not placed between views containing individuals.

17. The video production system of claim 10, wherein the processor is further configured to transmit the strip in hidden video lines.

18. A non-transitory storage medium storing instructions executable by a processor, the instructions comprising instructions to:

receive motion detection data;

accept presence sensor data;

receive video information from one or more cameras;

center, using the received motion detection data, accepted presence sensor data and received video information, individuals in a room view using at least one display device;

zoom onto the centered individuals of the room view in one or more zoomed views;

obtain sound source localization data and active talker information corresponding to at least one active talker;

determine, based, at least in part, on the obtained sound source localization data, the obtained active talker information and the received motion detection data, where, within a panoramic strip in a video layout of the display device, to place the zoomed views of the individuals; and

place the panoramic strip within the video layout, so as to prevent the panoramic strip overlaying display of the active talker, while maintaining proportionality of the display of the active talker within the video layout.

19. The non-transitory storage medium of claim 18, wherein the instructions further comprise instructions to place the panoramic strip in one of three locations, a top of the video layout, a bottom of the video layout, or hidden.

20. The non-transitory storage medium of claim 19, wherein the instructions further comprise instructions to place the zoomed views in the center of the panoramic strip.