Video coding

• US 9,049,447 B2
• Filed: 12/30/2010
• Issued: 06/02/2015
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: Pelco (Schneider Electric SE)
• Original Assignee: Pelco (Schneider Electric SE)
• Inventors: Millar, Greg, Aghdasi, Farzin, Wang, Lei, Huang, Chien-Min
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US12/982,602
• Publication Number:
  US 20120169923A1
• Time in Patent Office: 1,615 Days
• Field of Search: 375/240.01, 375/240.02, 375/240.24, 375/240.25, 375/240.26, 382/239, 382/243, 382/235, 382/233
• US Class Current: 1/1
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64769 for rate control flow control in packet networks H04L47/10
• CPC Class Codes: H04N 7/12 : Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal H04N7/24 takes precedence

Abstract

Techniques are discussed for providing mechanisms for coding and transmitting high definition video, e.g., over low bandwidth connections. In particular, foreground-objects are identified as distinct from the background of a scene represented in a plurality of video frames received from a video source, such as a camera. In identifying foreground-objects, semantically significant and semantically insignificant movement (e.g., repetitive versus non-repetitive movement) is differentiated. Processing of the foreground-objects and background proceed at different update rates or frequencies.

Claims

1. A transmission system comprising a processor configured to:

receive a plurality of frames;

analyze the plurality of frames;

determine a current bandwidth available for transmission;

determine, based on the current bandwidth, at least one of a pixel movement threshold or a pixel size threshold, the pixel movement threshold being a distance between a position in a first frame of a particular image object and a position in a second frame of the particular image object wherein the distance corresponds to a first number of pixels and the pixel size threshold being a difference between a dimension in the first frame of the particular image object and a dimension in the second frame of the particular image object wherein the difference corresponds to a second number of pixels;

identify foreground-object portions that are distinct from a background portion;

transmit, for each of the plurality of frames, foreground-object information for an identified foreground-object portion based on the identified foreground-object portion satisfying the at least one of the pixel movement threshold or the pixel size threshold; and

selectively transmit background information for less than all of the plurality of frames, wherein the background information is transmitted at a periodic rate based on the current bandwidth.

2. The system of claim 1, wherein the foreground-object portions include semantically significant moving objects.

3. The system of claim 1 wherein the processor is further configured to compare features of the particular image object with an identified object in a previously received frame, the features comprising position, speed, movement direction, color, and size.

4. A non-transitory computer readable medium encoded with one or more computer readable instructions that, when executed by a processor, cause the processor to:

analyze video data from a plurality of images;

determine a current bandwidth available for transmission;

determine, based on the current bandwidth, at least one of a pixel movement threshold or a pixel size threshold, the pixel movement threshold being a distance between a position in a first frame of a particular image object and a position in a second frame of the particular image object wherein the distance corresponds to a first number of pixels and the pixel size threshold being a difference between a dimension in the first frame of the particular image object and a dimension in the second frame of the particular image object wherein the difference corresponds to a second number of pixels;

identify the presence of foreground-object portions that are distinct from a background portion in the video data;

transmit first portions of the video data, of different images, associated with at least one identified foreground-object portion, based on the at least one identified foreground-object portion satisfying the at least one of the pixel movement threshold or the pixel size threshold, at a first update rate; and

transmit second portions of the video data, of different images, associated with the background portion at a second update rate, wherein the first update rate is greater than the second update rate.

5. The computer readable medium of claim 4, wherein the instructions that cause the processor to identify the presence of foreground-object portions cause the processor to identify semantically significant movement using a Gaussian model.

6. The computer readable medium of claim 5, wherein the instructions that cause the processor to identify the presence of foreground-object portions cause the processor to identify a region of semantically significant movement as a foreground-object portion.

7. The computer readable medium of claim 4, wherein the first update rate is thirty frames per second.

8. A video data transmission system comprising:

a processor configured to;

execute a scene analyzer algorithm for analyzing video data from a plurality of images;

determine a current bandwidth available for transmission;

determine, based on the current bandwidth, at least one of a pixel movement or a pixel size threshold, the pixel movement threshold being a distance between a position in a first frame of a particular image object and a position in a second frame of the particular image object wherein the distance corresponds to a first number of pixels and the pixel size threshold being a difference between a dimension in the first frame of the particular image object and a dimension in the second frame of the particular image object wherein the difference corresponds to a second number of pixels;

identify the presence of foreground-object portions that are distinct from a background portion in the video data; and

transmit first portions of the video data, of different images, associated with at least one identified foreground-object portion, based on the at least one identified foreground-object portion satisfying the at least one of the pixel movement threshold or the pixel size threshold, at a first update rate; and

to transmit second portions of the video data, of different images, associated with the background portion at a second update rate, wherein the first update rate is greater than the second update rate.

9. The system of claim 8, wherein the processor is configured to identify semantically significant movement using a Gaussian model.

10. The system of claim 9, wherein the processor is configured to identify a region of semantically significant movement as a foreground-object portion.

11. The system of claim 8, wherein the first update rate is thirty frames per second.

12. The system of claim 8 wherein the processor is configured to identify semantically significant moving objects as the foreground-object portions.