Multi-display video conferencing

• US 9,426,422 B2
• Filed: 11/25/2014
• Issued: 08/23/2016
• Est. Priority Date: 11/25/2014
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: Gandhi, Saumil Ashvin
• Primary Examiner(s): Woo, Stella L
• Application Number:
  US14/553,784
• Publication Number:
  US 20160150184A1
• Time in Patent Office: 637 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/16 Constructional details or arrangements
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1656 Details related to functional adaptations of the enclosure, e.g. to provide protection against EMI, shock, water, or to host detachable peripherals like a mouse or removable expansions units like PCMCIA cards, or to provide access to internal components for maintenance or to removable storage supports like CDs or DVDs, or to mechanically mount accessories mounting of accessories to a computer display G06F1/1607; display hoods G06F1/1603; cooling arrangements for portable computers G06F1/203
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Disclosed are a system comprising a computer-readable storage medium storing at least one program, and a computer-implemented method for facilitating multi-display video conferences. An audio-video interface module receives first and second audio input data and first and second video input data of a video conference session. The first audio input data and the first video input data are linked to a first display device of a first user device. The second audio input data and the second video data are linked to a second display device of the first user device. A speaker detector module processes, based on a determination that the first and second audio input data are linked to the first user device, the first and second audio input data to identify an active audio signal to be displayed in an active speaker display frame on a second user device.

Claims

1. A system comprising:

an audio-video interface module configured to receive first and second audio input data and first and second video input data of a video conference session, the first audio input data and the first video input data being linked to a first display device of a first user device, the second audio input data and the second video input data being linked to a second display device of the first user device;

a speaker detector module, including one or more processors, configured to process, based on a determination that the first and second audio input data are linked to the first user device, the first and second audio input data to identify an active audio signal, the speaker detector module being further configured to process the first and second audio input data by being configured to;

compare the first audio input data and the second audio input data;

based on the comparison, determine whether the first audio input data leads or lags the second audio input data; and

either

in response to a determination that the first audio input data leads the second audio input data, identify the first audio input data as the active audio signal;

or

in response to a determination that the first audio input data lags the second audio input data, identify the second audio input data as the active audio signal; and

a video conference module configured to either;

based on an identification of the first audio input data as the active audio signal, provide to a second user device a first message to display the first video input data in an active speaker display frame;

or

based on an identification of the second audio input data as the active audio signal, provide to the second user device a second message to display the second video input data in the active speaker display frame.

2. The system of claim 1, wherein the video conference module is further configured to provide first display data for the first display device and second display data for the second display device.

3. The system of claim 1, wherein the first user device corresponds to a personal computer device comprising the video conference module, the audio-video interface module, the speaker detector module, and the first and second display devices.

4. The system of claim 3, wherein the first display device corresponds to a front-facing display of the personal computer device, and the second display device corresponds to a rear-facing display of the personal computer device.

5. The system of claim 1, wherein the first user device corresponds to a personal computer device comprising the video conference module, the audio-video interface module, the speaker detector module, and the first display device, wherein the second display device corresponds to a display integrated in a third device that is separate from the personal computer device, the personal computer device being communicatively coupleable to the third device.

6. The system of claim 1, wherein the first audio input data corresponds to audio data generated by a first microphone linked to the first display device, wherein the second audio input data corresponds to audio data generated by a second microphone linked to the second display device.

7. The system of claim 6, wherein the first display device is colocated with the first microphone and the second display device is colocated with the second microphone.

8. The system of claim 6, wherein the first microphone is located closer to the first display device than the second display device, wherein the second microphone is located closer to the second display device than the first display device.

9. The system of claim 1, further comprising a multi-display management module configured to provide an interface definition indicative of a visibility characteristic of a display element, wherein the video conference module is configured to generate second display data for the second display device such that the second display device renders the display element in accordance with the interface definition.

10. The system of claim 9, wherein the multi-display management module is further configured to receive user input data linked to the first display device, the user input data being indicative of a value of the visibility characteristic of the display element, the multi-display management module being configured to set the visibility characteristic in accordance with the user input data.

11. The system of claim 1, further comprising an input device management module configured to interface with user input devices, wherein the video conference module is configured to receive user input indicative of a request linking a selected device of the user input devices to a selected one of the first and second display devices.

12. The system of claim 11, wherein the input device management module is further configured to detect a connected user input device, wherein the video conference module is configured to link the connected user input device to the first display device in response to a detection of the connected first user device, wherein the user input indicative of the request corresponds to a request linked to the first display device to link the connected user input device to the second display device.

13. A computer-implemented method comprising:

receiving first and second audio input data and first and second video input data of a video conference session, the first audio input data and the first video input data being linked to a first display device of a first user device, the second audio input data and the second video input data being linked to a second display device of the first user device;

based on a determination that the first and second audio input data are linked to the first user device, processing, by one or more processors, the first and second audio input data to identify an active audio signal in a process comprising;

comparing the first audio input data and the second audio input data;

based on the comparison, determining whether the first audio input data leads or lags the second audio input data; and

either

in response to the determining that the first audio input data leads the second audio input data, identifying the first audio input data as the active audio signal;

or

in response to the determining that the first audio input data lags the second audio input data, identifying the second audio input data as the active audio signal; and

either

based on an identification of the first audio input data as the active audio signal, providing to a second user device a first message to display the first video input data in an active speaker display frame;

or

based on an identification of the second audio input data as the active audio signal, providing to the second user device a second message to display the second video input data in the active speaker display frame.

14. The computer-implemented method of claim 13, wherein the first and second display devices are included by a person computer device.

15. The computer-implemented method of claim 14, wherein the first display device corresponds to a front-facing display of the personal computer device, and the second display device corresponds to a rear-facing display of the personal computer device.

16. The computer-implemented method of claim 13, wherein the first display device is included by a personal computer device, wherein the second display device corresponds to a display integrated in a third device that is separate from the personal computer device, the personal computer device being communicatively coupleable to the third device.

17. A non-transitory machine-readable storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving first and second audio input data and first and second video input data of a video conference session, the first audio input data and the first video input data being linked to a first display device of a first user device, the second audio input data and the second video input data being linked to a second display device of the first user device;

based on a determination that the first and second audio input data are linked to the first user device, processing, by one or more processors, the first and second audio input data to identify an active audio signal in a process comprising;

comparing the first audio input data and the second audio input data;

based on the comparison, determining whether the first audio input data leads or lags the second audio input data; and

either

in response to a determination that the first audio input data leads the second audio input data, identifying the first audio input data as the active audio signal;

or

in response to a determination that the first audio input data lags the second audio input data, identifying the second audio input data as the active audio signal; and

either

based on an identification of the first audio input data as the active audio signal, providing to a second user device a first message to display the first video input data in an active speaker display frame;

or

based on an identification of the second audio input data as the active audio signal, providing to the second user device a second message to display the second video input data in the active speaker display frame.

18. The non-transitory machine-readable storage medium of claim 17, wherein the first and second display devices are included by a person computer device.

19. The non-transitory machine-readable storage medium of claim 18, wherein the first display device corresponds to a front-facing display of the personal computer device, and the second display device corresponds to a rear-facing display of the personal computer device.