Residential video communication system

• US 8,253,770 B2
• Filed: 05/31/2007
• Issued: 08/28/2012
• Est. Priority Date: 05/31/2007
• Status: Expired due to Fees

• Current Assignee: Eastman Kodak Company
• Original Assignee: Eastman Kodak Company
• Inventors: Kurtz, Andrew F., Border, John N., Costello, Kathleen M., Jacoby, Keith A., Parada, Robert J. Jr.
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US11/756,532
• Publication Number:
  US 20080298571A1
• Time in Patent Office: 1,916 Days
• Field of Search: 348 1401- 141, 348/208.14, 370260-261, 709/204
• US Class Current: 348/14.01
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A video communication system and method for operating a video communication system are provided. The video communication system has a video communication device, having an image display device and at least one image capture device, wherein the at least one image capture device acquires video images of a local environment and an individual therein, according to defined video capture settings, an audio system having an audio emission device and an audio capture device; and a computer operable to interact with a contextual interface, a privacy interface, an image processor, and a communication controller to enable a communication event including at least one video scene in which outgoing video images are sent to a remote site. Wherein the contextual interface includes scene analysis algorithms for identifying potential scene transitions and capture management algorithms for providing changes in video capture settings appropriate to any identified scene transitions; and wherein the privacy interface provides privacy settings to control the capture, transmission, display, or recording of video image content from the local environment.

Claims

1. A video communication system comprising:

a video communication device, having at least one image capture device;

an image display device into one image capture device, wherein the at least one image capture device acquires video images of a local environment and an individual therein, according to defined video capture settings;

an audio system processor having an audio emission device operable to generate sound in the local environment and an audio capture device operable to capture sound in the local environment; and

a control logic processor for automatically analyzing the video scene captured by the capture device to determine when an intra-scene transition or in inter-scene transition occurs in the local environment and automatically adjusts the video capture settings of the least one image capture device in accordance with the determined intra-scene transition or inter-scene transition wherein the control logic processor or the audio system processor determines that an intra-scene transition or inter-scene transition has occurred by applying scene analysis rules to determine contextual cues in the video or audio in the local environment.

2. The system according to claim 1, wherein the control logic processor uses scene adjustment rules to define one or more rates for applying adjustments in the video capture settings that occur during the ongoing capture of video images.

3. The system according to claim 1 wherein the video images are modified by changes in the video capture settings for the image capture device or by image processing by the image processor.

4. The system according to claim 1, wherein the control logic processor uses scene analysis algorithms to examine video images based upon contextual cues, scene analysis rules, and statistical analysis to identify the intra-scene or inter-scene transitions.

5. The system according to claim 4, wherein the scene analysis algorithms assess both current and prior video images using either a reactive or anticipatory statistical methodology.

6. The system according to claim 4, wherein the scene analysis algorithms calculate confidence values as probabilistic metrics to indicate the communicative impact of an identification of an intra-scene transition or inter-scene transition.

7. The system according to claim 4, wherein the scene analysis algorithms assess activities occurring in a captured wide field of view provided by an image capture device to identify potential scene transitions, while images from a narrower field of view provided by an image capture device are communicated to a remote viewer using a second video communication device.

8. The system according to claim 4, wherein the scene analysis algorithms use scene analysis rules, classifications of user activities, and knowledge of current video capture settings to identify intra-scene or inter-scene transitions and to determine changes in the video capture settings in accordance with the identified transitions.

9. The system according to claim 1, wherein the control logic processor is operable to interact with the at least one image capture device to provide a range of video capture modes, which correspond to different combinations of defined video capture settings, the video capture modes including manual mode, an automatic mode, a semi-automatic mode, a preview mode, a default modes, or an electronic picture frame mode.

10. The system according to claim 5, wherein the provided automatic video capture mode can be selected from one of a group including a lock and follow mode, a hierarchical mode, a portable mode, a media space, or one of various modes based upon given event classifications.

11. The system according to claim 1, wherein the at least one image capture device comprises a first image capture device that acquires images from a narrow field of view within the local environment and a second image capture device that simultaneously acquires images from a larger field of view that includes the narrow field of view.

12. The system according to claim 11, wherein the contextual interface monitors the large field of view of the second image capture device to identify potential scene transitions or content that is relevant to available privacy settings provided by the user(s).

13. The system according to claim 1, wherein the outgoing video images from one of the video communication devices is produced by the control logic processor using at least a portion of the video images acquired by the at least two different image capture devices to produce composite video images.

14. The system according to claim 1, wherein the computer causes the image display device to simultaneously display video images from one or more portions of the local environment and incoming video images from one or more remote sites each having a second video communication device.

15. The system according to claim 1, wherein the control logic processor uses scene analysis algorithms including face detection, voice detection, motion detection, or motion tracking capabilities to assess the captured activities of one or more individuals in a local environment.

16. The system according to claim 1, which can recognize individuals within the local environment, by means including face recognition or voice recognition.

17. The system according to claim 16, wherein voice recognition is used to interpret the speech of an individual for the purpose of generating sub-titles.

18. The system according to claim 1, wherein during an identified intra-scene transition, then the determined changes in the video capture settings are made in a manner consistent with the currently captured video scene as defined by a known current video capture mode or video context.

19. The system according to claim 18, wherein during an identified inter-scene transition, the changes in the video capture settings are determined to be inconsistent with the settings for capturing the current or prior video scene, then the video capture settings are changed to those corresponding to a determined new video capture mode or video context or to interim settings if the determination results are inconclusive.

20. The system according to claim 1, wherein said at least one image capture device is provided with automatic optical zoom, pan, or tilt adjustment capabilities and wherein the control logic processor can interact with such image capture device to cause automatic adjustments to be made to direct the video image capture when a scene transition is detected or according to privacy settings and according to video image content.

21. The system according to claim 1, wherein the video capture settings are determined from contextual cues identified by the control logic processor or the audio system processor that include event classifications, user classifications, user identity, object identity, environmental conditions, user gestures, user behaviors, user privacy settings, or user audio cues.

22. The system according to claim 1, wherein the audio system further comprises audio enhancement means, such as noise cancellation or noise reduction functions, sound localization analysis functions, or combinations thereof.

23. The system according to claim 1, wherein the video capture settings that are controlled by the control logic processor or the audio system processor comprise a set including parameters related to the video image quality, the audio quality, the field of view of the capture images, privacy settings, eye gaze correction, e-mail or messaging, or combinations thereof.

24. The system according to claim 1, wherein one or more reference images of the local environment or an individual therein are acquired and used as standards analysis of later acquired images.

25. The system according to claim 24, wherein a reference image of the local environment is used as a standard to guide modification of images of the local environment acquired during a video scene.

26. The system according to claim 24, wherein the appearance of an individual within the local environment is used as a standard to guide modification of images of images of the individual acquired during a later video scene.

27. A method for providing video communication between a first individual at a local site and a second individual at a remote site, comprising:

capturing video images at the local site according to defined video image capture settings during a communication event;

capturing audio signals at the local site;

automatically identifying potential inter-scene and intra-scene transitions in the captured video images by applying scene analysis rules to determine contextual cues in the video or audio in the scene at the local site on an ongoing basis during the communication event;

determining changes in video image capture settings appropriate to any identified scene transitions using capture management algorithms, and adapting the capture of the video images accordingly;

transmitting at least a portion of the captured video images and audio signals from the local site to the second individual at the remote site; and

transmitting at least a portion of the remote video images and audio signals from the remote site to the local site;

adapting the capture, transmission, display, or recording of the content of the video images at the respective sites for each of the respective sites when an intra-scene or inter-scene transition is identified.

28. The method according to claim 27, wherein said identifying of potential inter-scene and intra-scene transitions comprises examining the video images based upon contextual cues, scene analysis rules, and statistical analysis to identify intra-scene or inter-scene transitions.

29. The method according to claim 27, wherein the privacy settings are determined by individuals at the respective sites using custom privacy settings for the communication event, default privacy settings, or combinations thereof.

30. A method for providing operating a video communication system using one or more cameras and image processing to provide video communications between an individual in a local environment and a remote viewer using a video communication system, comprising:

capturing video images with the at least one camera of the local environment during a video communication event of one or more video scenes using first video capture settings corresponding to an initial video context or an initial video capture made;

transmitting captured video images to a remote viewer;

automatically analyzing the captured video images with scene analysis algorithms to identify contextual cues from which it can be determined whether intra-scene transitions or inter-scene transitions have occurred in the local environment;

automatically determining new video capture settings appropriate to any identified intra-scene or inter-scene transition using capture management algorithms, wherein for defined intra-scene transitions, the defined changes to second video settings are adjustments within the initial video capture mode or initial video context, and for defined inter-scene transitions, the defined changes to second video settings correspond to those for a new video capture mode;

automatically modifying ongoing video image capture in accordance with the new video settings; and

providing the video images captured according to the new video settings to the remote viewer using the newly determined video settings.

31. The method as in claim 30 wherein the video settings relate to video communication features including those related image capture, image display, audio, maintenance of subject privacy, eye gaze, or e-mail or messaging.

32. The method as in claim 30 wherein the video capture settings involve at least one of a field of view, the use of pan, tilt, or zoom operation, or an image quality of a images captured during video communications.

33. The method as in claim 30 wherein the video capture modes, for which video settings are defined, include manual modes, semi-automatic modes, automated capture modes, or portable modes.

34. The method as in claim 33 wherein the automated video capture modes include modes based upon user classifications, subject identity, lock and follow operation, hierarchical operation, location and duration of activity, or event classifications.

35. The method as in claim 33 wherein at least one of the defined video capture modes operates as a media space or operates for multi-Way communications.

36. The method as in claim 30 further in which the validity of any identified intra-scene or inter-scene transitions are assessed using a transition test.

37. The method as in claim 36 wherein the transition test uses statistical tests of scene analysis results of current and prior video images to determine whether a scene transition is occurring, the scene transition being either an intra-scene transition or an inter-scene transition.

38. The method according to claim 36, wherein the scene analysis algorithms examine current and prior video images using contextual cues, scene analysis rules, and statistical analysis to identify intra-scene or inter-scene transitions.

39. The method according to claim 30 wherein the video content is determined from identified contextual cues that include event classifications, user classifications, user identity, user gestures, user behaviors, or user privacy settings.

40. The method according to claim 30 wherein interim video settings are applied during scene transitions from initial to subsequent video settings, whether during an intra-scene or inter-scene transition.