Encoding a video stream having a privacy mask

• US 10,277,901 B2
• Filed: 05/08/2017
• Issued: 04/30/2019
• Est. Priority Date: 05/08/2017
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Danielsson Fan, Xing
• Primary Examiner(s): Beck, Leron
• Application Number:
  US15/589,350
• Publication Number:
  US 20180324436A1
• Time in Patent Office: 722 Days
• Field of Search: 3752402
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

Methods and apparatus, including computer program products, implementing and using techniques for encoding a video sequence having a plurality of image frames, wherein at least some of the image frames include a privacy mask. An original total amount of motion is estimated in a received image frame. A reduced amount of motion is determined in the image frame, based on the estimated total amount of motion for the image frame and the size and position of the privacy mask. The image frames are encoded into a sequence of output image frames, wherein a temporal frame distance is adjusted based on the determined reduced amount of motion.

Claims

1. A method for encoding a video sequence comprising a plurality of image frames, the method comprising:

estimating a motion map of an image frame, the motion map dividing the image frame into blocks, wherein each block contains information about the amount of motion taking place in the block;

estimating an original total amount of motion in the image frame from the estimated motion in the individual block;

receiving data specifying a size and a position of a privacy mask overlay to be applied to the image frame to occlude an area of the image frame, wherein the privacy mask overlay coincides with one or more blocks of the motion map;

determining a reduced amount of motion in the image frame, based on the estimated total amount of motion for the image frame and the size and position of the privacy mask overlay, by deducting, from the estimated original total amount of motion, at least a portion of the motion occurring in the blocks coinciding with the privacy mask overlay; and

encoding the plurality of image frames into a sequence of output image frames, wherein a distance between image frames input to an encoder, or an intra frame distance in a sequence of output image frames from the encoder, is adjusted based on the determined reduced amount of motion.

2. The method of claim 1, further comprising:

receiving data specifying a type of the privacy mask overlay.

3. The method of claim 2, wherein the type of the privacy mask overlay is one of:

an opaque privacy mask overlay, a blurring privacy mask overlay, and a pixelation privacy mask overlay.

4. The method of claim 3, wherein, if the type of the privacy mask overlay is an opaque privacy mask overlay, determining a reduced amount of motion includes deducting from the estimated original total amount of motion all motion occurring behind the opaque privacy mask overlay.

5. The method of claim 3, wherein, if the type of the privacy mask overlay is a pixelation privacy mask overlay, determining a reduced amount of motion includes deducting from the estimated original total amount of motion a portion of the motion occurring behind the pixelation privacy mask.

6. The method of claim 1, wherein the temporal frame distance is adjusted dynamically in response to receiving and determining reduced amounts of motion for additional frames in the video sequence.

7. An encoder system for encoding a video sequence comprising a plurality of image frames, the encoder system comprising:

a motion estimation module configured to;

estimate a motion map of an image frame, the motion map dividing the image frame into blocks, wherein each block contains information about the amount of motion taking place in the block;

estimate an original total amount of motion in the image frame from the estimated motion in the individual blocks,

receive data specifying a size and a position of a privacy mask overlay to be applied to the image frame to occlude an area of the image frame, wherein the privacy mask overlay coincides with one or more blocks of the motion map, and

determine a reduced amount of motion in the image frame, based on the estimated total amount of motion for the image frame and the size and position of the privacy mask overlay, by deducting, from the estimated original total amount of motion, at least a portion of the motion occurring in the blocks coinciding with the privacy mask overlay; and

an encoder configured to;

encode the plurality of image frames into a sequence of output image frames, wherein a distance between image frames input to an encoder, or an intra frame distance in a sequence of output image frames from the encoder, is adjusted based on the determined reduced amount of motion.

8. A computer program product for encoding a video sequence comprising a plurality of image frames, wherein at least some of the image frames include a privacy mask, the computer program product comprising a non-transitory computer readable storage medium having program code embodied therewith, the program code being executable by a processor to perform a method comprising:

estimating a motion map of an image frame, the motion map dividing the image frame into blocks, wherein each block contains information about the amount of motion taking place in the block;

estimating an original total amount of motion in the image frame from the estimated motion in the individual blocks;

receiving data specifying a size and a position of a privacy mask overlay to be applied to the image frame to occlude an area of the image frame, wherein the privacy mask overlay coincides with one or more blocks of the motion map;

determining a reduced amount of motion in the image frame, based on the estimated total amount of motion for the image frame and the size and position of the privacy mask overlay, by deducting, from the estimated original total amount of motion, at least a portion of the motion occurring in the blocks coinciding with the privacy mask overlay; and

encoding the plurality of image frames into a sequence of output image frames, wherein a distance between image frames input to an encoder, or an intra frame distance in a sequence of output image frames from the encoder, is adjusted based on the determined reduced amount of motion.

9. The computer program product of claim 8, wherein the method further comprises:

receiving data specifying a type of the privacy mask overlay.

10. The computer program product of claim 8, wherein, if the type of the privacy mask overlay is an opaque privacy mask, determining a reduced amount of motion includes deducting from the estimated original total amount of motion all motion occurring behind the opaque privacy mask overlay.

11. The computer program product of claim 8, wherein, if the type of the privacy mask is a pixilation privacy mask overlay, determining a reduced amount of motion includes deducting from the estimated original total amount of motion a portion of the motion occurring behind the pixelation privacy mask.

12. The computer program product of claim 8, wherein the temporal frame distance is adjusted dynamically in response to receiving and determining reduced amounts of motion for additional frames in the video sequence.