Video pre-processing

• US 20050185045A1
• Filed: 06/12/2003
• Published: 08/25/2005
• Est. Priority Date: 06/12/2002
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Kamariotis, Othon
• Granted Patent: US 7,425,981 B2
• Time in Patent Office: Days
• US Class Current: 348/14.120
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109

Abstract

A method of distinguishing between a foreground object part and a substantially static background part of each videoframe within a video sequence. The method comprises the steps of: dividing each video frame into a number of video blocks each of which comprises one or more pixels; generating a mask frame in respect of each video frame, each mask frame having a mask block corresponding to each video block in each respective video frame; and either setting each mask block to either an object value, indicating that the corresponding video block in the corresponding video frame includes one or more pixels depicting a foreground object part, or to another value.

Claims

1. A method of setting a region of interest in each of a sequence of video frames, each video frame comprising a plurality of pixel values, the method including, in respect of each video frame following a predetermined number of initial frames:

generating in respect of each video frame an instantaneous region of interest;

determining whether, and if so to what extent, to modify the region of interest set for the preceding video frame, in order to determine the region of interest for the currently considered video frame, according to a stepped procedure; and

setting the region of interest of the currently considered video frame in accordance with the result of the determination performed according to the stepped procedure;

wherein

the stepped procedure includes comparing a first boundary defining the instantaneous region of interest of the currently considered frame, or sections thereof, with a second boundary defining the region of interest set for the preceding frame, or corresponding sections thereof, and if the difference is greater than a threshold amount, modifying the second boundary, or sections thereof, in the direction towards the first boundary, or the corresponding sections thereof, by an amount which is constrained to be equal to or below a maximum step amount.

2. A method as claimed in claim 1 wherein each video frame is divided into a plurality of blocks each of which comprises one or more pixels, and wherein the set region of interest is set to correspond to a plurality of whole blocks, referred to as blocks of interest, by setting each block which falls on or inside the modified second boundary as a block of interest.

3. A method as claimed in claim 2 further comprising generating a corresponding mask frame in respect of each of the video frames, each mask frame comprising a plurality of block values corresponding to the blocks into which each video frame is divided, and setting each block value to either a block of interest value or another value, depending on whether the corresponding block in the corresponding video frame is a block of interest or not.

4. A method as claimed in claim 1 wherein an instantaneous region of interest and a modified set region of interest are calculated in respect of the second and each succeeding video frame of the sequence, and wherein a region of interest for the first video frame of the sequence is set to be the entire video frame.

5. A method as claimed in claim 1 wherein the comparison and modification of boundaries is performed on a section by section basis and wherein different maximum step amounts may be set for different sections or groups of sections and/or depending on whether the modification is away from or towards the centre of the video frame and/or depending on whether a particular condition has been met or not.

6. A method of transmitting a video sequence from a transmitting device to a receiving device across a limited bandwidth connection, the method comprising:

generating a mask frame in respect of each video frame in the sequence according to a method as claimed in claim 3,

processing the video frames and the mask frames in accordance with an object based encoding procedure by which objects may be encoded, and decoded at the receiving end, separately from a background part, to generate a plurality of encoded data, and

packetising and transmitting the data across the limited bandwidth connection to the receiving device.

10. A carrier medium carrying computer processor implementable instructions for causing a computer processor to carry out the method of claim 1 during implementation of the instructions.

7. Apparatus for setting a region of interest in each of a sequence of video frames, each video frame comprising a plurality of pixel values, the apparatus including:

generating means for generating in respect of a video frame an instantaneous region of interest;

determining means for determining whether, and if so to what extent, to modify the region of interest set for the preceding video frame, in order to determine the region of interest for a currently considered video frame, according to a stepped procedure; and

setting means for setting the region of interest of the currently considered video frame in accordance with the result of the determination performed by the determination means according to the stepped procedure;

wherein

the determining means includes comparison means for comparing a first boundary defining the instantaneous region of interest of the currently considered frame, or sections thereof, with a second boundary defining the region of interest set for the preceding frame, or corresponding sections thereof, and modification means for, if the difference is greater than a threshold amount, modifying the second boundary, or sections thereof, in the direction towards the first boundary, or the corresponding sections thereof, by an amount which is constrained to be equal to or below a maximum step amount.

8. Apparatus for transmitting a video sequence across a limited bandwidth connection to a receiving device, the apparatus including:

apparatus as claimed in claim 7 for setting a region of interest in each of the video frames of the video sequence,

encoding means for encoding the video sequence in accordance with an object based encoding procedure, wherein the region of interest in each frame is treated as an object and the remainder of each frame is treated as background, and

transmitting means for transmitting the encoded video sequence across the limited bandwidth connection to the receiving device.

9. A video-conference device including the apparatus of claim 7.