REAL TIME VIDEO COMMUNICATIONS SYSTEM

• US 20100066804A1
• Filed: 09/16/2009
• Published: 03/18/2010
• Est. Priority Date: 09/16/2008
• Status: Active Grant

• Current Assignee: Biscotti Incorporated
• Original Assignee: Wham! Incorporated
• Inventors: Ahmed, Nadeem, Shoemake, Matthew B.
• Granted Patent: US 8,144,182 B2
• Time in Patent Office: Days
• US Class Current: 348/14.20
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 65/1026 at the edge
• CPC Class Codes: H04L 65/1036 at the edge
• CPC Class Codes: H04L 65/1059 End-user terminal functionalities specially adapted for real-time communication
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

Novel tools and techniques for providing video calling solutions. In some such solutions, a video calling device resides functionally inline between a set-top box and a television set. Such solutions can provide, in some cases, high performance video calling, high video quality, simplified installation, configuration and/or use, and/or the ability to enjoy video calling in an inclusive, comfortable environment, such as a family room, den, or media room.

Claims

1. A system for real-time, high-definition video communication, the system comprising:

a first video communication device; and

a second video communication device in communication with the first video communication device over the Internet;

wherein the first video communication device comprises;

a high-definition multimedia (“

HDMI”

) input interface configured to receive high-definition audiovisual input from a set-top box;

an HDMI output interface configured to provide high-definition audiovisual output for display on a high-definition television;

a speaker to provide audible output;

a wireless network interface in communication with a wireless local area network to provide communication with the second video communication device over the Internet;

one or more microphones to capture audio input from a user of the first video communication device;

a high-definition video camera to capture a video stream of the user;

at least one processor; and

a storage medium in communication with the at least one processor, the storage medium having encoded thereon a set of instructions executable by the at least one processor to control operation of the first video communication device, the set of instructions comprising;

instructions for receiving, via the wireless network interface, a connection request from a communication server on the Internet, the connection request originating from the second video communication device;

instructions for notifying the user of the communication request;

instructions for receiving user input accepting the connection request;

instructions for receiving a video call over a peer-to-peer Internet communication session with the second video communication device, via the wireless network interface;

instructions for transmitting an audio watermark signal through the HDMI output interface;

instructions for determining whether a speaker of the high-definition television is powered on and enabled to play audio output from the first video communication device, based on reception of the audio watermark signal through at least one of the one or more microphones;

instructions for receiving a high-definition set-top box audiovisual stream from the set-top box, wherein set-top box audiovisual stream comprises a set-top box video stream and a set-top box audio stream, the set-top box video stream having a display area comprising a plurality of pixels, the plurality of pixels defining a display resolution of the set-top box video stream;

instructions for receiving a remote audiovisual stream from the second video communication device over the wireless network interface, wherein the remote audiovisual stream comprises a remote video stream and a remote audio stream;

instructions for detecting the display resolution of set-top box video stream;

instructions for setting a display resolution of the remote video stream, based at least in part on one or more factors selected from the group consisting of the detected display resolution of the set-top box video stream, a bandwidth of a connection between the first video communication device and the second video communication device, and a user-specified setting;

instructions for modifying the set-top box video stream, in response to receiving the video call;

instructions for allocating at least a portion of the display area of the set-top box video stream, comprising at least some of the plurality of pixels, for displaying at least a portion of the remote video stream, to produce a consolidated high-definition output video stream comprising at least a portion of the remote video stream;

instructions for transmitting the consolidated high-definition output video stream on the HDMI output interface, for display on the high-definition television;

instructions for producing a consolidated audio output stream comprising at least the remote audio stream;

instructions for transmitting the consolidated audio output stream on the HDMI output interface, based on a determination that the speaker of the high-definition television is powered on and enabled to play audio output from the first video communication device;

instructions for controlling the camera to capture a captured video stream;

instructions for controlling the one or more microphones to capture a captured audio stream;

instructions for canceling echo in the captured audio stream;

instructions for encoding the captured video stream and the captured audio stream as a series of packets for transmission over the Internet; and

instructions for transmitting the series of packets over the peer-to-peer communication session.

2. The system of claim 1, wherein the set of instructions further comprises:

instructions for modifying a field of view of the camera, based on commands received from the second video communication device.

3. The system of claim 2, wherein the instructions for modifying an effective field of view of the camera comprise instructions for performing one or more operations selected from the group consisting of vertically panning the effective field of view, horizontally panning the effective field of view, and zooming the effective field of view.

4. The system of claim 1, wherein the connection request comprises a video clip, and wherein the instructions for notifying the user of the connection request comprise instructions for providing a notification comprising the video clip.

5. The system of claim 1, wherein the connection request comprises an audio clip, and wherein the instructions for notifying the user of the connection request comprise instructions for providing a notification comprising the audio clip.

6. A video communication system, comprising:

a first video communication device, comprising;

a video input interface to receive video input from a set-top box;

an audio input interface to receive audio input from the set-top box;

a video output interface to provide video output to a video display device;

an audio output interface to provide audio output to an audio receiver;

a video capture device to capture video;

an audio capture device to capture audio;

a network interface;

at least one processor; and

a storage medium in communication with the at least one processor, the storage medium having encoded thereon a set of instructions executable by the at least one processor to control operation of the first video communication device, the set of instructions comprising;

instructions for controlling the video capture device to capture a captured video stream;

instructions for controlling the audio capture device to capture a captured audio stream;

instructions for encoding the captured video stream and the captured audio stream to produce a series of data packets; and

instructions for transmitting the series of data packets on the network interface for reception by a second video communication device.

7. The video communication system of claim 6, further comprising a remote control, wherein the first video communication device further comprises a remote control receiver.

8. The video communication system of claim 6, wherein the at least one processor is a plurality of processors comprising a first processor to process the captured video stream and a second processor to process a remote video stream received from the second video communication device.

9. The video communication system of claim 6, wherein the set-top box comprises the first video communication device.

10. The video communication system of claim 6, wherein the video display device comprises the video communication device.

11. The video communication system of claim 6, wherein the video communication device is incorporated within the set-top box.

12. The video communication system of claim 6, wherein the network interface comprises a wireless local area network (“

WLAN”

) interface.

13. The video communication system of claim 6, wherein the video input interface comprises a high-definition multimedia interface (“

HDMI”

) to receive high-definition video input from the set-top box.

14. The video communication system of claim 6, wherein the video input interface comprises the audio input interface.

15. The video communication system of claim 6, wherein the video output interface comprises a high-definition multimedia interface (“

HDMI”

) to provide a high-definition output video stream to a high-definition television.

16. The video communication system of claim 6, wherein the video output interface comprises the audio output interface.

17. The video communication system of claim 6, wherein the video display device comprises the audio receiver.

18. The video communication system of claim 6, wherein the video display device is a high-definition television.

19. The video communication system of claim 6, wherein the video capture device comprises a high-definition camera.

20. The video communication system of claim 6, wherein the audio capture device comprises a microphone array comprising at least two microphones.

21. The video communication system of claim 6, wherein the second video communication device is a personal computer programmed with a video chat application.

22. The video communication system of claim 6, wherein instructions for transmitting the series of data packets comprise instructions for transmitting the series of data packets over the Internet.

23. The video communication system of claim 6, wherein instructions for transmitting the series of data packets comprise instructions for transmitting the series of data packets over a private content delivery network.

24. The video communication system of claim 6, wherein the video input interface receives a set-top box video stream, and wherein the network interface receives a remote audiovisual stream from the second video communication device, the remote audiovisual stream comprising a remote video stream and a remote audio stream.

25. The video communication system of claim 24, wherein the set of instructions further comprises:

instructions for creating a consolidated output video stream comprising at least a portion of the remote video stream.

26. The video communication system of claim 25, wherein the consolidated output video stream further comprises at least a portion of the captured video stream.

27. The video communication system of claim 25, wherein the consolidated output video stream further comprises a user interface of the first video communication device.

28. The video communication system of claim 25, wherein the consolidated output video stream further comprises at least a portion of the set-top box video stream.

29. The video communication system of claim 28, wherein the set-top box video stream has a display area comprising a plurality of pixels, the plurality of pixels defining a display resolution of the set-top box video stream, and wherein the instructions for creating a consolidated output video stream comprise:

instructions for allocating at least a portion of the display area, comprising at least some of the plurality of pixels, for displaying at least a portion of the remote video stream.

30. The video communication system of claim 29, wherein the instructions for allocating at least a portion of the display area comprise:

instructions for overlaying at least a portion of the remote video stream over at least a portion of the set-top box video stream, using a specified video α

value to control an apparent transparency of the at least a portion of the remote video stream.

31. The video communication system of claim 29, wherein the instructions for allocating at least a portion of the display area comprise:

instructions for allocating a first portion of the display area for displaying at least a portion of the set-top box video stream; and

instructions for allocating a second portion of the display area for displaying at least a portion of the remote video stream.

32. The video communication system of claim 31, wherein the set of instructions further comprises:

instructions for receiving user input specifying an allocation scheme, wherein the first portion of the display area and the second portion of the display area are defined by the allocation scheme.

33. The video communication system of claim 29, wherein the instructions for allocating at least a portion of the display area comprise:

instructions for detecting the display resolution of the set-top box video stream.

34. The video communication system of claim 33, wherein the set of instructions further comprises:

instructions for setting a resolution of the consolidated output video stream as a function of the input video stream.

35. The video communication system of claim 25, wherein the set of instructions further comprises:

instructions for setting a resolution of the consolidated output video stream based at least in part on a bandwidth of a connection between the first video communication device and the second video communication device.

36. The video communication system of claim 25, wherein the set of instructions further comprises:

instructions for setting a resolution of the consolidated output video stream based at least in part on a user-specified setting.

37. The video communication system of claim 25, wherein the set of instructions further comprises:

instructions for resampling a resolution of the remote video stream to accommodate the resolution of the set-top box video stream.

38. The video communication system of claim 6, wherein the network interface receives a remote audio stream from the second video communication device.

39. The video communication system of claim 38, wherein the set of instructions further comprises:

instructions for mixing the remote audio stream with a set-top box audio stream received via the audio input interface, using a specified audio α

value to create a consolidated output audio stream.

40. The video communication system of claim 39, wherein the specified audio α

value falls between 0 and 1, in which a specified audio α

value of 0 specifies that the consolidated output audio stream consists entirely of the remote audio stream and a specified audio α

value of 1 specifies that the consolidated output audio stream consists entirely of the set-top box audio stream.

41. The video communication system of claim 6, wherein the first video communication device further comprises a speaker to play the audio output stream.

42. The video communication system of claim 41, wherein the audio receiver is external to first video communication device, and wherein the set of instructions further comprises:

instructions for detecting whether the audio receiver is configured to play the audio output stream.

43. The video communication system of claim 42, wherein the set of instructions further comprises:

instructions for controlling operation of the speaker based at least in part on whether the audio receiver is configured to play the audio output stream.

44. The video communication system of claim 42, wherein the instructions for detecting whether the audio receiver is capable of playing the audio output stream comprise:

instructions for inserting an audio watermark in the audio output stream; and

instructions for analyzing the captured audio stream to identify the audio watermark.

45. The video communication system of claim 44, wherein the audio watermark has a known waveform, and wherein the instructions for analyzing the captured audio stream comprise:

instructions for sampling a signal of the captured audio stream; and

instructions for comparing the sampled signal with a static waveform corresponding to the known waveform of the audio watermark.

46. The video communication system of claim 41, the set of instructions further comprising:

instructions for detecting whether the video display device is configured to display the output video stream; and

instructions for transmitting the output video stream over the output video interface based on a determination that the video display device is configured to display the output video stream.

47. The video communication system of claim 6, wherein the network interface receives control input from the second video communication device, the set of instructions further comprising:

instructions for modifying an effective field of view of the camera, based on the control input.

48. The video communication system of claim 47, wherein the instructions for modifying an effective field of view of the camera comprise instructions for performing one or more operations selected from the group consisting of vertically panning the effective field of view, horizontally panning the effective field of view, and zooming the effective field of view.

49. The video communication system of claim 47, wherein the camera has a native resolution comprising a first set of pixels and the effective field of view comprises a second set of pixels, the second set of pixels being a subset of the first set of pixels, and wherein the instructions for modifying an effective field of view of the camera comprise:

instructions for selecting the second set of pixels from within the first set of pixels, in response to the control input;

instructions for generating the captured video stream from the selected second set of pixels.

50. The video communication system of claim 6, wherein the instructions for encoding the captured video stream comprise instructions for processing the captured video stream prior to encoding the captured video stream.

51. The video communication system of claim 6, wherein the instructions for encoding the captured audio stream comprise instructions for processing the captured audio stream prior to encoding the captured video stream.

52. The video communication system of claim 6, wherein video communication device receives a set-top box audiovisual stream from the set-top box, the audiovisual stream comprising an audio stream received via the audio input interface and a video stream received via the video input interface.

53. The video communication system of claim 52, wherein the set-top box audiovisual stream comprises a broadcast television program.

54. The video communication system of claim 52, wherein the set of instructions further comprises:

instructions for detecting a watermark in the set-top box audiovisual stream; and

instructions for identifying content in the set-top box audiovisual stream, based at least in part on the detected watermark.

55. The video communication system of claim 54, wherein the watermark comprises a video watermark in the set-top box video stream.

56. The video communication system of claim 54, wherein the watermark comprises an audio watermark in the set-top box audio stream.

57. The video communication system of claim 54, wherein the set of instructions further comprises:

instructions for transmitting information about the identified content via the network interface.

58. The video communication system of claim 57, wherein the instructions for transmitting information about the identified content comprise instructions for transmitting the information for reception by the second video communication device.

59. The video communication system of claim 54, wherein the set of instructions further comprises:

instructions for obtaining relevant advertising, based on the identified content in the set-top box audiovisual stream; and

instructions for inserting the relevant advertising into an output video stream provided via the video output interface.

60. The video communication system of claim 54, wherein the set of instructions further comprises:

instructions for compiling statistics regarding viewing preferences of a user of the first video communication device, based at least in part on the identified content in the set-top box audiovisual stream.

61. The video communication system of claim 6, the set of instructions further comprising:

instructions for caching the captured video stream and the captured audio stream on a storage device if a connection with the second video communication device is unavailable.

62. The video communication system of claim 61, wherein the storage device is local to the first video communication device.

63. The video communication system of claim 62, the set of instructions further comprising:

instructions for transmitting the captured audio stream and the captured video stream via the network interface after a communication session has been established with the second video communication device.

64. The video communication system of claim 61, wherein the storage device is local to a communication server remote from the first video communication device.

65. The video communication system of claim 6, wherein the first video communication device further comprises a data interface, and wherein the instructions further comprise:

instructions for detecting a media storage device in communication with the data interface;

instructions for obtaining one or more media files from the media storage device;

instructions for caching the one or more media files at first video communication device; and

instructions for transmitting the one or more media files via the network interface for storage at an other device.

66. The video communication system of claim 65, wherein the other device is a communication server remote from the first video communication device.

67. The video communication system of claim 65, wherein the other device is the second video communication device.

68. The video communication system of claim 65, wherein the media storage device is selected from the group consisting of a flash memory device, a still camera, a video camera, a digital music player, a wireless phone, and a handheld computer.

69. A method of providing video calling using a first video communication device comprising an audio capture device, a video capture device, a network interface, an audiovisual input interface, and an audiovisual output interface, the method comprising:

receiving, on the audiovisual input interface, a set-top box audiovisual stream from a set-top box, the set-top box audiovisual stream comprising a set-top box video stream and a set-top box audio stream;

receiving, on the network interface, a remote audiovisual stream via a network connection with a second video communication device, the remote audiovisual stream comprising a remote audio stream and a remote video stream;

transmitting, on the audiovisual output interface, a consolidated output video stream comprising at least a portion of the remote video stream and a consolidated output audio stream comprising at least the remote audio stream;

capturing a captured video stream with the video capture device;

capturing a captured audio stream with the audio capture device;

encoding the captured video stream and the captured audio stream to produce a series of data packets; and

transmitting the series of data packets on the network interface for reception by the second video communication device.

70. The method of claim 69, wherein the audiovisual input interface comprises an audio input interface and a video input interface, wherein receiving the set-top box audiovisual stream comprises receiving the set-top box audio stream on the audio input interface, and wherein receiving the set-top box video stream comprises receiving the set-top box video stream on the video input interface.

71. The method of claim 69, wherein the audiovisual output interface comprises an audio output interface and a video output interface, wherein transmitting the consolidated output audio stream comprises transmitting the consolidated output audio output stream on the audio output interface, and wherein transmitting the consolidated output video stream comprises transmitting the consolidated output video stream on the video output interface.

72. The method of claim 69, wherein the audiovisual input interface is a high-definition multimedia interface (“

HDMI”

).

73. The method of claim 69, wherein the audiovisual output interface is a high-definition multimedia interface (“

HDMI”

).

74. The method of claim 69, wherein the network interface is a wireless local area network (“

WLAN”

) interface.

75. A method of providing real-time video communication services, the method comprising:

generating, at a communication server computer system, a sparse set of identifiers, the sparse set of identifiers comprising a first identifier corresponding to a first video communication device associated with a first user and a second identifier corresponding to a second video communication device associated with a second user;

storing, at the communication server computer system, the sparse set of identifiers;

receiving, at the communication server system, a connection request from the first video communication device, the connection request referencing a second video communication device, the connection request comprising the first identifier and sufficient information to identify the second video communication device;

identifying the first video communication device at the communication server computer system, based at least in part on the first identifier;

identifying the second video communication device at the communication server computer system, based at least in part on the information in the request;

transmitting, for reception by the second video communication device, information sufficient to allow the second user to identify the first user;

providing, from the communication server computer system to at least one of the video communication devices, connection information sufficient to allow the first video communication device and the second video communication device to establish a peer-to-peer communication session.

76. The method of claim 75, further comprising:

receiving, at the communication server computer system, an acceptance of the connection request;

wherein the connection information is provided based on receiving the acceptance of the connection request.

77. The method of claim 75, wherein the information in the connection request comprises a second identifier corresponding to the second video communication device.

78. The method of claim 75, wherein the connection request comprises a video clip of the first user, and wherein the information sufficient to allow the second user to identify the first user comprises the video clip.

79. The method of claim 75, wherein the connection request comprises an audio clip of the first user, and wherein the information sufficient to allow the second user to identify the first user comprises the audio clip.

80. The method of claim 75, wherein the connection information is sufficient to allow the first video communication device and the second video communication device to establish a peer-to-peer communication session over the Internet.

81. The method of claim 75, wherein the connection information is sufficient to allow the first video communication device and the second video communication device to establish a peer-to-peer communication session over a private content delivery network.