APPARATUS FOR INTELLIGENT AND AUTONOMOUS VIDEO CONTENT GENERATION AND STREAMING

• US 20130182119A1
• Filed: 12/26/2012
• Published: 07/18/2013
• Est. Priority Date: 05/15/2008
• Status: Active Grant

• Current Assignee: Comcast Cable Communications LLC (Comcast Corporation)
• Original Assignee: SRI International, Inc.
• Inventors: ELEDATH, JAYAKRISHNAN KUMAR, PARIS-MASCICKI, SARAH, ENDRES, VINCENT, HANNA, BARBARA
• Granted Patent: US 9,596,399 B2
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system for automatically capturing an event of interest in a venue is disclosed, comprising a plurality of cameras for capturing video images of the event; and at least one processor communicatively connected to said plurality of cameras and configured to execute a plurality of modules, said modules comprising a rectification module for combining the video images to form a wide-angle view; at least one of a motion analysis module configured for tracking salient blobs that are not part of a background of the wide-angle view, an activity analysis module configured for extracting temporal and spatial patterns from the wide-angle view, and a shape and appearance module configured for selecting one or more objects in the wide-angle view based on descriptors that are scale and rotational invariant; and a region of interest selector for selecting a viewpoint from the wide-angle view based on output from at least one of the motion analysis module, the activity analysis module, and the shape and appearance module, wherein the region of interest selector outputs the selected viewpoint for display. The system further comprises at least one audio recording device for capturing audio from the event; and means for synchronizing the video images and audio.

Claims

1. A computer-implemented method for focusing cameras on an area of activity comprising:

selecting salient objects from a field of view in a video stream, generated by a plurality of cameras, where saliency is determined by shape, appearance and motion of the objects;

analyzing activity in the field of view to extract patterns from the field of view, where the patterns are at least one of temporal and spatial;

determining a region of interest based on the extracted patterns and the selected salient objects; and

focusing one or more of the plurality of cameras on the determined region of interest.

2. The method of claim 1 further comprising:

determining the region of interest based on a region in the field of view that is determined from the extracted patterns to have a relatively high degree of motion.

3. The method of claim 2 wherein the motion is particularly in one direction.

4. The method of claim 1 further comprising assembling the extracted patterns into larger patterns indicating a global activity.

5. The method of claim 4 further comprising:

determining the region of interest as the region in the field of view where the assembled larger patterns converge.

6. The method of claim 1 further comprising determining the region of interest as a region in the field of view with a relatively large number of salient objects.

7. The method of claim 1 further comprising synchronizing the video stream with a received audio stream.

8. The method of claim 1, further comprising extracting a background in the video stream from a wide-angle view.

9. The method of claim 1 further comprising:

determining the region of interest by applying one or more content specific rules associated with the activity.

10. The method of claim 9 wherein when the activity is a type of sporting event, and the content specific rules comprise one or more rules for focusing the plurality of cameras on a scenario of interest in said type of sporting event.

11. The method of claim 1 wherein selecting the salient objects further comprises:

selecting objects in the field of view based on descriptors that are scale and rotational invariant.

12. The method of claim 11, further comprising selecting one of silhouettes, contours, and silhouettes and contours of the objects as the descriptors.

13. A computer implemented method for determining video sensor focus comprising:

receiving a live video stream from a video sensor;

analyzing the live video stream to determine a changing region of most interest as compared to other regions, as the live video is streamed; and

modifying focus of the video sensor on the determined changing region of most interest as the live video is streamed.

14. The method of claim 13, wherein the region of most interest is determined as the region with the greatest amount of salient activity.

15. The method of claim 14, further comprising:

determining the salient activity according to shape, appearance and motion of detected objects in the video stream.

16. The method of claim 15, further comprising:

determining the salient activity by further analyzing activity in the video stream to extract patterns, where the patterns are at least one of temporal and spatial; and

assembling the extracted patterns into larger patterns indicating a global activity and determining the region of interest based on a region with the global activity and the salient objects;

17. The method of claim 16, further comprising determining the region of interest by applying content specific rules to the focusing where the global activity is determined to be a sporting event.

18. The method of claim 17 wherein when the sporting event is a football game, and the content specific rules comprise focusing the plurality of cameras on an automatically identified location of a play in the event, where a play is at least one of a “

pass,”

a “

run,”

a “

punt,”

a “

field goal attempt,”

or has resulted in a “

touchdown.”

19. The method of claim 17 wherein when the sporting event is a basketball game, and the content specific rules comprise focusing the plurality of cameras on a portion of the event based on selecting a player who is at a foul line and attempting a foul shot.

20. The method of claim 17 wherein when the sporting event is a volleyball game, and the content specific rules comprise focusing the plurality of cameras on a portion of the event based on selecting a player behind a service line.