AUTOMATED VIDEOGRAPHY BASED COMMUNICATIONS

• US 20100245532A1
• Filed: 03/26/2009
• Published: 09/30/2010
• Est. Priority Date: 03/26/2009
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Kurtz, Andrew F., Blose, Andrew C.
• Granted Patent: US 8,237,771 B2
• Time in Patent Office: Days
• US Class Current: 348/14.30
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

A method for framing subjects captured on video includes receiving video of a subject and determining a current shot framing of the subject. A magnitude and a direction of movement of the subject is determined, relative to the current framing and a level of motion threshold. The subject movement is analyzed relative to the determined magnitude, direction of subject movement, and the current framing, to determine that the subject is properly framed by the current shot framing or whether modifications of the current shot framing are required to capture the moving subject, or to determine a new shot selection and new shot framing. The camera re-frames the subject if the subject is determined not to be properly framed, in accordance with the determined modifications of the current shot framing, the new shot selection and new shot framing. Video images are provided to a remote viewer.

Claims

1. A method for framing one or more subjects captured on video during a video communication event with a remote viewer, the method comprising the steps of:

receiving video of a first subject in an environment from a camera;

determining a current shot framing of the first subject in the video images, with a computer and an associated image processor, relative to a shot selection and subject positioning within the framed shot;

determining a magnitude and a direction of movement of the first subject in the received video, relative to the current framing and level of motion thresholds;

analyzing the subject movement of the first subject, relative to the determined magnitude and direction of subject movement, and the current framing, to determine that the subject is properly framed by the current shot framing, or to determine modifications of the current shot framing that are required to capture the moving first subject, or to determine a new shot selection and new shot framing that are required to capture the moving first subject;

,

instructing the image processor or camera to re-frame the first subject if the first subject is determined not to be properly framed by the current shot framing, in accordance with the determined modifications of the current shot framing or the determined new shot selection and new shot framing; and

providing video images to the remote viewer in accordance with the current shot framing, or the modified current shot framing, or the new shot framing.

4. The method according to claim 1 wherein subject motion is measured relative to a variety of factors, including a velocity, acceleration, frequency, position, direction, or a range, or combinations thereof, relative to the subject framing.

5. The method according to claim 1 wherein the subject movement is measured by a reframing frequency, which is a measurable rate of reframing the subject in video images, by means of zooming, panning, tilting of a camera, image cropping, or switching from images captured by one camera to images captured by another camera, or combinations thereof.

6. The method according to claim 1 wherein decreasing levels of subject motion are analyzed relative to decreasing level of motion thresholds to determine that the current shot framing can be changed to a tighter shot.

23. The system according to claim 1 wherein an intra-scene transition corresponds to a change in the framing of the subject in the video image, relative to subject position, size, or focus, while retaining the current framing, and wherein an inter-scene transition corresponds to a change to a new subject framing.

24. The system according to claim 23 wherein the scene analysis algorithm determines scene change confidence values, including an intra-scene change probability or an inter-scene change probability, are calculated and then used to determine that an intra-scene transition or an inter-scene transition, respectively.

2. The method according to claim I wherein the shot selection is determined based on the percentage area occupied by a face or head relative to the total image area, or on the basis of the number of subjects within the framed shot, or on the basis of subject activity, or combinations thereof.

3. The method according to claim 2 in which the selection of available shots includes a long shot, a wide shot, a medium shot, a medium wide shot a close-up shot, or intermediate shots thereof.

7. A method for providing video communications between at least one subject in a local environment and a remote viewer using a video communication system, comprising:

capturing video images with the video communication system using one or more cameras, during a video communication event of one or more video scenes, which involve a subject and a local environment;

applying current video capture settings which define subject subject framing and camera parameters, and transmitting captured video images to the remote viewer in accordance with current video capture settings;

analyzing the video images of a current video scene to assess image quality relative to changes in subject activity and the current subject framing;

determining a need to change subject framing to improve video image quality and presentation to the remote viewer based on the assessment of image quality;

determining whether the changes in subject activity, relative to the subject framing, correspond to an intra-scene transition or an inter-scene transition;

determining new subject framing and new video capture settings for a scene-transition, whether intra-scene or inter-scene;

automatically modifying ongoing video image capture in accordance with the newly determined video capture settings; and

providing video images to the remote viewer using the video capture settings.

8. The method according to claim 7 wherein an intra-scene transition corresponds to a change in the framing of the subject in the video image, relative to subject position, size, or focus, while retaining the current framing, and wherein an inter-scene transition corresponds to a change to a new subject framing.

9. The method according to claim 8 wherein scene change confidence values, including an intra-scene change probability or an inter-scene change probability, are calculated and then used to determine that an intra-scene transition or an inter-scene transition, respectively.

10. The method according to claim 7 in which the video images are framed as shots, and the shot selection is determined based on the percentage area occupied by a face or head relative to the total image area, or on the basis of the number of subjects within the framed shot, or on the basis of subject activity, or combinations thereof.

11. The method according to claim 10 in which the selection of available shots includes a long shot, a wide shot, a medium shot, a medium wide shot a close-up shot, or intermediate shots thereof.

12. The method according to claim 10 in which reframing of video capture from a current framed shot to an alternate shot framing is delayed by a defined re-centering and re-sizing time delay.

13. The method according to claim 12 wherein the defined re-centering and re-sizing time delays are applied to enhance the viewing experience of the remote viewer.

14. The method according to claim 7 in which changing levels of subject activity are measured by a subject movement factor or a re-framing frequency, which are tracked over time during the videography event.

15. The method according to claim 14 wherein the refraining frequency is a measurable rate of refraining, by means of zooming, panning, tilting of a camera, image cropping, or camera switching, or combinations thereof.

16. The method according to claim 14 wherein the subject movement factor is measured as a normalized product of subject movement relative to an imaged field of view, where the subject movement factor is determined relative to a variety of factors, including a velocity, acceleration, frequency, position, direction, or a range, or combinations thereof, relative to the subject framing.

17. The method according to claim 7 wherein the remote viewer provides instructions to the system to change the video capture settings of the system operating in the local environment.

18. A video communication system, comprising:

an image display device adapted to display video images;

at least one image capture device which acquires framed video images of a local environment having at least one subject therein during a video capture event, according to defined video capture settings, which include a current shot selection and current shot framing of the subject;

an audio system having an audio capture device operable to sense sound in the local environment in accordance with audio settings;

a computer cooperating with the at least one image capture device, the audio system, and an image processor, to generate video signals based upon the acquired video images, wherein said video signals are then transmitted across a communicative network;

said computer further providing a contextual interface including a scene analysis algorithm for identifying potential intra-scene transitions or inter-scene transitions in subject activity during the video capture event, as measured by subject activity metrics relative to the current shot framing; and

the contextual interface further including a video capture management algorithm for determining adjustments in video capture settings appropriate for identified intra-scene transitions or inter-scene transitions, wherein the identified adjustments in the video capture settings include changes in the shot framing of the subject within the video image or a shot transition delay time or a shot change transition time.

19. The system according to claim 18 wherein the subject activity metrics include a subject movement factor or a re-framing frequency, which are tracked over time during the video capture event.

20. The system according to claim 19 wherein the reframing frequency is defined as a measurable rate at which reframing, which includes zooming, panning, tilting of a camera, or image cropping, is undertaken.

21. The system according to claim 19 wherein the subject movement factor is defined as a normalized product of subject movement relative to an imaged field of view.

22. The system according to claim 18 wherein the video capture management algorithm determines shot change confidence values that are calculated as probabilities in determining a selection of a subsequent shot to follow a current shot framing.

25. The system according to claim 18 wherein the at least one image capture device includes at least one camera that captures a wide image field of view and one camera that captures a narrow image field of view.

26. The system according to claim 18 which further includes a user interface that enable subjects, or the remote viewer, to determine user defined settings, including privacy settings or selection of automatic capture modes that define operation of the contextual interface.

27. The system according to claim 18 in which the values for the shot transition delay time or a shot change transition time are measured in seconds or frames of elapsed video, and depend on a number of subjects and a shot selection.