Multi-camera residential communication system

• US 8,154,578 B2
• Filed: 05/31/2007
• Issued: 04/10/2012
• Est. Priority Date: 05/31/2007
• Status: Expired due to Fees

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Kurtz, Andrew F., Border, John N.
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US11/756,562
• Publication Number:
  US 20080297587A1
• Time in Patent Office: 1,776 Days
• Field of Search: 348 1401- 1416, 386/242, 386/280, 709204-205
• US Class Current: 348/14.01
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Video communication devices and methods are provided. The device has an image display device; a first image capture device that acquires video images depicting a wide field of view within the local environment; a second image capture device that acquires video images depicting a narrow field of view within the wide field of view; a communication system that is adapted to use a communication network to transmit outgoing video images; and a computer causing the capture of video images during a communication event, and further being adapted to identify a video context for the communication event; wherein said computer uses a scene analysis algorithm to examine concurrent video images from the image capture device based upon the identified video context to determine the extent to which video images from the first image capture device and the second image capture device are to be incorporated into the outgoing video images.

Claims

1. A video communication system comprising:

an image display device;

a first image capture device responsive to first video capture settings that acquires video images from a wide field of view of a local environment including images of one or more individuals therein;

a second image capture device responsive to second video capture settings that acquires video images from a narrow field of view that is at least partially included within the wide field of view;

a computer in operative communication with the image display device and the image capture devices with said computer causing the first image capture device and the second image capture device to capture video images in real time during the communication event, and with said computer further being adapted to identify a video context for the communication event based upon the activities of at least the individuals that is occurring in the captured video; and

a communication system that is adapted to use a communication network to transmit outgoing video images;

wherein said computer uses a scene analysis algorithm in real time to examine the concurrent video images from both the first image capture device and the second image capture device based upon the identified video context to determine the extent to which video images from the first image capture device and the second image capture device are provided in a field of view of the outgoing video images transmitted by the communication device; and

wherein the scene analysis algorithm is further used to assess the activity occurring in the fields of view captured by the image capture devices, as compared to that in the transmitted video, so as to identify scene transitions in the video context that responsively cause changes in either the first video capture settings or the second video capture settings such that at least one of the transmitted field of view and an image quality attribute of the outgoing video images are changed.

2. The system according to claim 1, wherein the scene analysis algorithm monitors the wide field of view video images based upon image content data, contextual cues, and scene analysis rules, relative to the identified video context, to determine an image field of view provided in the outgoing video images.

3. The system according to claim 2, wherein the scene analysis rules determine whether video images from outside the image field of view provided in the outgoing video images correlates with the current defined video context or with a new video context.

4. The system according to claim 1 wherein the video context is determined from identified contextual cues that include event classifications, user classifications, user identity, user gestures, user behaviors, or user privacy settings.

5. The system according to claim 1, wherein the video context is determined with respect to defined automatic video capture modes that each specify a range of defined video capture settings, the automatic video capture modes including a fully automatic mode, a semi-automatic mode, a preview mode, a lock and follow mode, a hierarchical mode, a portable mode, a media space, or a mode selected based upon given event classifications.

6. The system according to claim 1, which further includes a privacy interface which enables users to apply privacy settings that limit video capture for either the first image capture device or second image capture device.

7. The system according to claim 1, wherein the computer employs at least one of the scene analysis algorithms that examine the wide field of view images captured by the first camera to identify individuals or activities occurring outside the narrow field of view captured by the second camera.

8. The system according to claim 1, wherein the computer employs at least one of the scene analysis algorithms that examine both current and prior video images from the first or second cameras view to identify individuals or activities occurring outside the narrow field of view captured by the second camera.

9. The system according to claim 1, which further includes an audio system and wherein the computer employs at least one of the scene analysis algorithms to examine the audio data from the audio field of view to identify individuals or activities occurring outside the field of view captured by either the first or the second image capture device.

10. The system according to claim 1, wherein the computer employs at least one of the scene analysis algorithms to determine when to provide an outgoing video signal that includes individuals or activities occurring outside the narrow field of view captured by the second image capture device.

11. The system according to claim 1, wherein the computer causes the image display device to provide a split screen image, wherein one portion of the image display device shows video images of the local environment which were acquired by either the first or second image capture device.

12. The system according to claim 1, wherein the wide field of view captured by the first image capture device spans a full angular width of at least a 60 degrees, or of at least 170 degrees, or spanning 360 degrees.

13. The system according to claim 1, wherein the device further comprises at least a second narrow field of view image capture device one of which is imaging through the image display device, and one of which is not imaging through the display device and has automatic pan, tilt, or optical zoom adjustment capabilities.

14. The system according to claim 1, wherein a remote viewer who is receiving the transmitted images directs the image capture provided by image capture device to determine the transmitted field of view of the outgoing video images.

15. The system according to claim 1, wherein the second image capture device images through the image display device.

16. The system according to claim 1, wherein the second image capture device has pan, tilt, or optical zoom capabilities.

17. The system according to claim 1 wherein the computer changes the video capture setting that determine the outgoing video images by at least switching between the first and second image capture devices.

18. The system according to claim 1 wherein the image quality attributes that are changed, include, but are not limited to, at least one of the image resolution, the image brightness, the image contrast, or the image color quality.

19. The system according to claim 1 in which the computer uses scene adjustment rules to define the rate and extent of adjustments in the video capture settings that are applied in response to a determined change in the video context.

20. The system according to claim 1 wherein the identified scene transitions are classified as either an intra-scene transition or an inter-scene transition.

21. The system according to claim 20 wherein during an identified intra-scene transition, the determined changes in the video capture settings are made in a manner consistent with the currently captured video scene, as defined by a known current video capture mode or video context.

22. The system according to claim 20 wherein during an identified inter-scene transition, in which the changes in the video capture settings are determined to be inconsistent with the settings for capturing the current or prior video scene, then the video capture settings are changed to those corresponding to a determined new video capture mode or video context, or to interim settings if the determination results are inconclusive.

23. A method for operating a video communication system comprising the steps of:

acquiring images with a first image capture device according to associated first video capture settings from a wide field of view of a first field of view within the local environment including images of one or more individuals therein;

acquiring images from a narrow field of view of a second field of view within the first field of view of the local environment with a second image capture device according to associated second video capture settings;

acquiring audio content from the local environment with an audio system;

transmitting outgoing video images from at least a portion of a current field of view acquired by either the first or second image capture device to a remote viewer;

using at least one scene analysis algorithm to examine concurrent video images in real time from both the first image capture device and the second image capture device to automatically determine a video context for the communication event based on classification of the activities of the individuals; and

further using the scene analysis algorithm(s) to determine, based upon the identified video context the extent to which video images from the first image capture device and the second image capture device are to be incorporated into the transmitted outgoing video images;

wherein the scene analysis algorithm is further used to assess the activity occurring in the fields of view captured by the image capture devices, as compared to that in the transmitted video, so as to identify scene transitions in the video context that responsively cause changes in either the first video capture settings or second video capture settings such that at least one of the transmitted field of view and an image quality attribute of the outgoing video images are changed.

24. The method of claim 23, wherein the scene analysis algorithm monitors the wide field of view video images based upon image content data, contextual cues, and scene analysis rules, relative to the identified video context, to determine an image field of view provided in the outgoing video images.

25. The method of claim 23, wherein the scene analysis algorithm further analyzes audio content acquired by the audio system concurrently with the concurrent video image in determining the extent to which video images from the first image capture device and the second image capture device are to be incorporated into the outgoing video images transmitted.