VIDEO CODING

• US 20120169923A1
• Filed: 12/30/2010
• Published: 07/05/2012
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: Pelco (Schneider Electric SE)
• Original Assignee: Pelco (Schneider Electric SE)
• Inventors: Millar, Greg, Aghdasi, Farzin, Wang, Lei, Huang, Chien-Min
• Granted Patent: US 9,049,447 B2
• Time in Patent Office: Days
• US Class Current: 348/399.1
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64769 for rate control flow control in packet networks H04L47/10
• CPC Class Codes: H04N 7/12 : Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal H04N7/24 takes precedence

Abstract

Techniques are discussed for providing mechanisms for coding and transmitting high definition video, e.g., over low bandwidth connections. In particular, foreground-objects are identified as distinct from the background of a scene represented in a plurality of video frames received from a video source, such as a camera. In identifying foreground-objects, semantically significant and semantically insignificant movement (e.g., repetitive versus non-repetitive movement) is differentiated. Processing of the foreground-objects and background proceed at different update rates or frequencies.

Claims

1. A transmission system comprising a processor configured to:

receive a plurality of frames;

analyze the plurality of frames;

identify foreground-object portions that are distinct from a background portion;

transmit foreground-object information for each of the plurality of frames; and

selectively transmit background information for less than all of the plurality of frames, wherein the background information is transmitted at a periodic rate.

2. The system of claim 1, wherein the foreground-object portions include semantically significant moving objects.

3. A method of transmitting video over a network to which computer devices can connect, the method comprising:

receiving a plurality of video data from a video source;

analyzing the plurality of video data;

identifying the presence of foreground-object portions that are distinct from background portions in the plurality of video data;

processing data associated with at least one identified foreground-object portion based on a first update rate;

processing data associated with the background portions based on a second update rate;

transmitting data associated with at least one identified foreground-object portion based on the first update rate; and

transmitting data associated with the background portions based on the second update rate, wherein the first update rate is greater than the second update rate.

4. The method of claim 3, wherein identifying the foreground-object portions includes identifying semantically significant movement based on a Gaussian mixture model.

5. The method of claim 4, wherein if a video data portion includes semantically significant movement, the portion is identified as a foreground-object portion.

6. The method of claim 3, wherein processing data associated with the at least one foreground-object portion includes determining whether the at least one foreground-object portion is noise.

7. The method of claim 3, wherein the first update rate is thirty frames per second.

8. The method of claim 3, wherein processing data associated with the at least one foreground-object portion includes:

computing a residual object image for the foreground-object portion;

encoding the residual object image to produce an encoded residual object image;

decoding the encoded residual object image to produce a decoded residual object image; and

generating an object image based on the decoded residual object image.

9. The method of claim 8, wherein the residual object image is based on a reference background image or a previously reconstructed frame.

10. The method of claim 3, wherein processing data associated with the background portions includes:

computing a residual background image for a background portion based on a first reference background image;

encoding the residual background image to produce an encoded residual background image;

decoding the encoded residual background image to produce a decoded residual background image;

generating a second reference background image based on the decoded residual background image and the first reference background image.

11. The method of claim 3, further comprising transmitting event data, wherein the event data indicates whether a foreground-object portion has entered or left a scene represented by the plurality of video data.

12. The method of claim 3, wherein transmitting data associated with the background portions includes:

dividing a residual background image for a background portion into a plurality of slices; and

transmitting each slice in the plurality of slices at intervals.

13. The method of claim 3, wherein transmitting data associated with the background portions includes transmitting a residual background region image and location information for the region image following a determination that the at least one foreground-object is stationary or a determination that a previously identified foreground-object has left a scene represented by the plurality of video data.

14. A non-transitory computer readable medium encoded with one or more computer readable instructions that, when executed by a processor, cause the processor to:

analyze video data from a plurality of images;

identify the presence of foreground-object portions that are distinct from a background portion in the video data;

transmit first portions of the video data, of different images, associated with at least one identified foreground-object portion at a first update rate; and

transmit second portions of the video data, of different images, associated with the background portion at a second update rate, wherein the first update rate is greater than the second update rate.

15. The computer readable medium of claim 14, wherein the instructions that cause the processor to identify the presence of foreground-object portions cause the processor to identify semantically significant movement using a Gaussian model.

16. The computer readable medium of claim 15, wherein the instructions that cause the processor to identify he presence of foreground-object portions cause the processor to identify a region of semantically significant movement as a foreground-object portion.

17. The computer readable medium of claim 14, wherein the first update rate is thirty frames per second.

18. The computer readable medium of claim 14, wherein the instructions that cause the processor to transmit portions of data associated with the at least one identified foreground-object portion cause the processor to:

compute a residual object image for the foreground-object portion;

encode the residual object image to produce an encoded residual object image;

decode the encoded residual object image to produce a decoded residual object image;

generate an object image based on the decoded residual object image; and

transmit the encoded residual object image.

19. The computer readable medium of claim 18, the residual object image is computed from a reference background image or a previously reconstructed frame.

20. The computer readable medium of claim 14, wherein the instructions that cause the processor to transmit portions of data associated with the background portion cause the processor to:

compute a residual background image for the background portion based on a first reference background image;

encode the residual background image to produce an encoded residual background image;

decode the encoded residual background image to produce a decoded residual background image;

generate a second reference background image based on the decoded residual background image and the first reference background image; and

transmit the encoded residual background image.

21. The computer readable medium of claim 20, wherein the instructions that cause the processor to transmit the encoded residual background image cause the processor to:

divide the encoded residual background image into a plurality of slices; and

transmit each slice in the plurality of slices at intervals.

22. A non-transitory computer readable medium encoded with one or more computer readable instructions that, when executed by a processor, cause the processor to:

determine a data type for each of a plurality of received data, wherein data in the plurality of the received data associated with a moving foreground-object data type is received at a first update rate and data in the plurality of the received data associated with a background data type is received at a second update rate, wherein the first update rate is greater than the second update rate;

process each of the plurality of received data based on a determined data type; and

generate at least one video frame using moving foreground-object data that is based on a first video frame associated with a first time and background data that is based on a second video frame associated with a second time, wherein the first time is later than the second time.

23. The computer readable medium of claim 22, wherein the instructions that cause the processor to process data based on a determined data type cause the processor to:

decode the data to produce a decoded object image; and

store the decoded object image in an objects list.