Managing scene transitions for video communication

• US 8,063,929 B2
• Filed: 05/31/2007
• Issued: 11/22/2011
• Est. Priority Date: 05/31/2007
• Status: Expired due to Fees

• Current Assignee: Eastman Kodak Company
• Original Assignee: Eastman Kodak Company
• Inventors: Costello, Kathleen M., Kurtz, Andrew F., Fedorovskaya, Elena A., Border, John N.
• Primary Examiner(s): Ramakrishnaiah; Melur
• Application Number:
  US11/756,563
• Publication Number:
  US 20080297588A1
• Time in Patent Office: 1,636 Days
• Field of Search: 348 1401- 1416, 370260-261, 709204-205, 715/753, 715/751, 715/755
• US Class Current: 348/14.08
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

Video communication systems and methods for operating the same are provided. The system has a computer cooperating with at least one image capture device which acquires video images of a local environment having an individual therein, an audio system, an image processor and a communication controller to acquire video images based upon the acquired video images and to cause output video images to a remote individual during the communication event; said computer further providing a contextual interface including an intra-scene analysis algorithm for identifying potential intra-scene transitions during the communication event and a scene capture management algorithm for determining intra-scene adjustments in video capture settings when an intra-scene transition is detected; and the contextual interface further including a transition test with an inter-scene analysis algorithm, for identifying potential inter-scene transitions in the activities of the individuals, and a transition process structure for determining inter-scene adjustments in video capture settings.

Claims

1. A video communication system, comprising:

an image display device adapted to display video images;

at least one image capture device which acquires video images of a local environment having an individual therein, according to defined video capture settings;

an audio system having an audio emission device operable to generate sound in the local environment and an audio capture device operable to sense sound in the local environment, said audio system operating in accordance with audio settings; and

a computer cooperating with the at least one image capture device, the audio system, an image processor and a communication controller to generate video signals based upon the acquired video images and to cause output video images to a remote individual during the communication event;

said computer further providing a contextual interface including an intra-scene analysis algorithm for identifying potential intra-scene transitions during the communication event and a scene capture management algorithm for determining intra-scene adjustments in video capture settings when an intra-scene transition is detected; and

the contextual interface further including a transition test with an inter-scene analysis algorithm, for identifying potential inter-scene transitions in the activities of the individuals, and a transition process structure for determining inter-scene adjustments in video capture settings,

wherein the intra-scene transitions include changes in the video signal or audio signal that reduce a communicative impact of the video communication event and wherein the inter-scene transitions include changes in the video signal or audio signal that suggest that a nature of the video communication event has changed.

2. The system according to claim 1, wherein the scene analysis algorithms use scene analysis rules, classifications of user activities, and knowledge of current video capture settings to identify intra-scene transitions or to identify inter-scene transitions and to determine changes in the video capture settings in accordance with the identified transitions.

3. The system according to claim 2, wherein the scene analysis algorithms assess both current and prior video images using either a reactive or anticipatory statistical methodology.

4. The system according to claim 2, wherein the scene analysis algorithms calculate confidence values as probabilistic metrics to indicate the communicative impact of an identification of an intra-scene transition or inter-scene transition.

5. The system according to claim 1, wherein the scene capture management algorithm and the transition process structure use scene adjustment rules to define the rate and extent of adjustments in the video capture settings.

6. The system according to claim 1, wherein the video capture settings are determined from identified contextual cues that include event classifications, user classifications, user identity, object identity, environmental conditions, user gestures, user behaviors, user privacy settings, or user audio cues.

7. The system according to claim 1, wherein the at least one image capture device includes at least one camera that captures a wide image field of view and one camera that captures a narrow image field of view.

8. The system according to claim 7, wherein the scene analysis algorithms assess activities occurring in a captured wide field of view to identify potential scene transitions, while image from the narrow field of view are communicated to the remote viewer.

9. The system according to claim 1, wherein the image capture devices include at least one camera automatic with pan, tilt, or optical zoom adjustment capabilities and wherein said determined intra-scene adjustments or inter-scene adjustments include camera adjustments.

10. The system according to claim 1, wherein the individual in the local environment provides initial video capture settings or an initial video context that defines initial video capture settings.

11. The system according to claim 1, which further includes a means for identification of an individual.

12. The method according to claim 11, wherein the means for identification of an individual includes face recognition or voice recognition.

13. The system according to claim 1, which further includes a means for motion detection and tracking of an individual.

14. The system according to claim 1, wherein the contextual interface further includes a privacy interface, which is used by an individual, to provide privacy settings to control the capture, transmission, display, or recording of video image content from the local environment.

15. A method for providing video for communication between an individual engaged in activities in a local environment and a remote viewer during a communication event using a video communication system, comprising:

capturing video images with a video communication system using image capture devices and image processing, whose operation is defined by video capture settings, during a video communication event of one or more video scenes which involve the individual and their local environment;

statistically analyzing the video images of video scenes with scene analysis algorithms provided within a contextual interface, to identify activities corresponding to intra-scene transitions or inter-scene transitions;

testing the validity of any identified intra-scene or inter-scene transitions;

determining any changes in the video capture settings consistent with the validated scene transitions; and

automatically modifying the ongoing video image capture relative to a video scene in accordance with both the identified changes in the video capture settings and a video capture scene adjustment algorithm;

wherein the intra-scene transitions include changes in the video signal or audio signal that reduce a communicative impact of the video communication event and wherein the inter-scene transitions include changes in the video signal or audio signal that suggest that a nature of the video communication event has changed.

16. The method according to claim 15, wherein the scene analysis algorithms assess activities occurring within a video scene relative to video context definitions to identify potential transitions from a current video context to a subsequent video context.

17. The method according to claim 16, wherein the video context definitions are based on knowledge of classifications or identities of individuals using the system.

18. The method according to claim 16, wherein the video context definitions are based upon a classification of events applicable to one or more video scenes.

19. The method according to claim 16, wherein the video context definitions are based upon quantitative assessments of the location and duration of activities within video scenes.

20. The method according to claim 16, wherein the video context definitions are based upon classification of gesture cues or voice cues provided by one or more individuals within video scenes.

21. The method according to claim 16, wherein the probability of a correct identification of a video context definition for a given video scene is calculated as a video context confidence value.

22. The method according to claim 21, wherein video capture transition algorithm uses video context confidence values to assess the validity of the identified changes in video capture settings.

23. The method according to claim 21, wherein the video capture transition algorithm initiates the use of interim video capture settings.

24. The method according to claim 15, wherein the video capture scene adjustment algorithm determines the rate and magnitude of the modifications in the video capture settings for ongoing video capture.

25. The method according to claim 24, wherein the video capture scene adjustment algorithm utilizes factors including scene adjustment rules and current video capture settings to determine the rate and magnitude of the modifications in video capture.

26. The method according to claim 15, wherein the video capture scene adjustment algorithm determines changes in the video capture settings to enhance image quality, related to factors including image focus, brightness, contrast, or color.

27. The method according to claim 15, wherein the video settings for video capture of a video scene are changed in response to either identified statistically significant intra-scene transitions or inter-scene transitions.

28. The method according to claim 27, wherein the scene analysis algorithms assess video scenes using metrics related to video context and content to identify scene transitions, while further providing transition confidence values indicative of the probability of a correct scene analysis.

29. The method according to claim 28, wherein the transition confidence values are used in testing the validity of identified intra-scene or inter-scene transitions.

30. The method according to claim 15, wherein changes in video capture settings include changes in an image field of view captured by an image capture device.

31. The method according to claim 30, wherein the field of view is modified based on the location and duration of the activities of the individual within the local environment.

32. The method according to claim 15, wherein the contextual interface modifies the video capture settings associated with an image capture device based upon the directions of either a local individual or the remote viewer.

33. A method for automatically controlling content acquired during a communication event using a video communication system at a local site, the method comprising the steps of:

defining a video context and an associated video capture mode appropriate for the activities of an individual or individuals within a current video scene that is occurring within the communication event at the local site;

acquiring video images according to the defined current video capture mode with the video communication system, and transmitting associated video signals to a remote video communication system;

monitoring the video images using metrics of intra-scene transitions and associated intra-scene transition confidence values to identify changes in activity that are consistent with the defined video context for the current video scene;

monitoring the video images with a transition test using metrics of inter-scene transitions and associated inter-scene transition confidence values to identify changes in activity that are indicative of a changes to an alternate video context;

selecting either the defined video context or the alternate video context depending on whether the intra-scene confidence values or the inter-scene confidence values statistically indicate a significant change in activity; and

automatically modifying the acquisition of subsequent video images based upon the selected video context and associated video capture mode settings.

34. The method according to claim 33, wherein the video images are monitored using one or more scene analysis algorithms which assess intra-scene transitions or inter-scene transitions using a Bayesian probabilistic inference method, a Markovian probabilistic inference method, or the two in combination.

35. The method according to claim 33, wherein both current and prior video images is monitored using scene analysis algorithms that assess potential intra-scene transitions or inter-scene transitions in the video context.