Apparatus for providing feedback on nonverbal cues of video conference participants

• US 8,976,218 B2
• Filed: 06/27/2011
• Issued: 03/10/2015
• Est. Priority Date: 06/27/2011
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Thorson, Dean E., Alberth, William P.
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US13/169,512
• Publication Number:
  US 20120327180A1
• Time in Patent Office: 1,352 Days
• Field of Search: 379/93.21, 379/158, 379/202.01, 348/14.01, 348/14.02, 348/14.03, 348/14.08
• US Class Current: 348/14.03
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

An electronic device is communicatively coupled to at least one video camera during a video conference, and also includes a touchscreen display having video conference participant representations displayed on the touchscreen display. A haptics feedback system is integrated into the touchscreen display and is responsive to a user touching the video conference participant representations displayed on the touchscreen display. A speech-to-text controller converts video conference participant'"'"'s speech into text and creates an identification tag for each video conference participant. In addition, an ongoing commentary controller provides feedback on non-verbal cues about the video conference participants.

Claims

1. An electronic device comprising:

a touchscreen display configured to display one or more video conference participant representations in a first region of the touchscreen display, wherein the first region includes an area for displaying a speaking video conference participant, and wherein the area is split into two display windows when two video conference participants are speaking to one another;

a haptics feedback system integrated into the touchscreen display and providing information regarding a video conference participant responsive to receiving a touch input at a location of the touchscreen display corresponding to the video conference participant'"'"'s representation displayed on the touchscreen display;

a speech-to-text controller for converting video conference participants'"'"' speech into text and creating an identification tag for each video conference participant; and

an ongoing commentary controller that provides feedback on non-verbal cues about the video conference participants,

wherein the haptics feedback system is configured to, responsive to receiving a touch input at a location of the touchscreen display corresponding to a second region of the touchscreen display, provide the feedback on non-verbal cues about the video conference participants in the second region of the touchscreen display, wherein the feedback is provided by audio or Braille tags.

2. The electronic device according to claim 1 further comprising a means for prioritizing the non-verbal cues feedback.

3. The electronic device according to claim 1, wherein the touchscreen display is configured to display local video conference participants in a third region of the touchscreen display, and wherein the touchscreen display is configured to display non-local video conference participants in a fourth region of the touchscreen display.

4. The electronic device according to claim 3, wherein the second region displays commentary on non-verbal cues corresponding to speaking video conference participants and non-speaking video conference participants.

11. The electronic device according to claim 1, wherein the information regarding a video conference participant comprises one or more of the identification tag, a title, gender, stature, or pro/con/neutral position relative to presentation materials.

5. A method comprising:

monitoring and recording video conference participants as they join a video conference;

identifying and categorizing the video conference participants to generate identification and category information for each of the video conference participants;

displaying the identified and categorized video conference participants within a first region of a touchscreen display;

displaying the identification and category information regarding a first video conference participant responsive to receiving a touch input at a location of the touchscreen display corresponding to the first video conference participant'"'"'s representation displayed on the touchscreen display;

displaying the video conference participants'"'"' conversation within a second region of the touchscreen display, wherein the second region is divided when at least two of the video conference participants are speaking to one another; and

responsive to receiving a touch input at a location of the touchscreen display corresponding to a third region of the touchscreen display, displaying, via audio or Braille tags, feedback and commentary on non-verbal cues of the first video conference participant.

6. The method according to claim 5, further comprising prioritizing the feedback and commentary on the non-verbal cues of the first video conference participant.

7. The method according to claim 5, further comprising displaying commentary on non-verbal cues corresponding to speaking video conference participants and non-speaking video conference participants.

12. The method according to claim 5, wherein the identification and category information for a respective video conference participant comprises one or more of an identification tag, a title, gender, stature, or pro/con/neutral position relative to presentation materials.

8. A non-transitory machine readable storage device, having stored thereon a computer program that includes a plurality of code sections comprising:

code for monitoring and recording video conference participants as they join a video conference;

code for identifying and categorizing the video conference participants to generate identification and category information for each of the video conference participants;

code for displaying the identified and categorized video conference participants within a first region of a touchscreen display;

code for displaying the identification and category information regarding a first video conference participant responsive to receiving a touch input at a location of the touchscreen display corresponding to the first video conference participant'"'"'s representation displayed on the touchscreen display;

code for displaying the video conference participants'"'"' conversation within a second region of the touchscreen display, wherein the second region is divided when at least two of the video conference participants are speaking to one another; and

code for, responsive to receiving a touch input at a location of the touchscreen display corresponding to a third region of the touchscreen display, displaying, via audio or Braille tags, feedback and commentary on non-verbal cues of the first video conference participant.

9. The non-transitory machine readable storage device claimed in claim 8, further comprising:

code for prioritizing the feedback and commentary on the non-verbal cues of the first video conference participant.

10. The non-transitory machine readable storage device claimed in claim 8, further comprising:

code for displaying commentary on non-verbal cues corresponding to speaking video conference participants and non-speaking video conference participants.

13. The non-transitory machine readable storage device claimed in claim 8, wherein the identification and category information for a respective video conference participant comprises one or more of an identification tag, a title, gender, stature, or pro/con/neutral position relative to presentation materials.