VIDEO PROCESSING DEVICE AND METHOD FOR DETERMINING MOTION METADATA FOR AN ENCODED VIDEO

• US 20200137395A1
• Filed: 10/28/2019
• Published: 04/30/2020
• Est. Priority Date: 10/29/2018
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Andersson, Viktor A, Leonhardt, Moa, Hakansson, Jonas
• Granted Patent: US 11,057,626 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field

Abstract

A video processing device which generates motion metadata for encoded video comprises a decoder configured to decode frames of an encoded video into image frames; and a processing circuitry configured to execute a motion meta data deriving operation on image frames decoded by the decoder. The motion meta data deriving operation comprises: a dividing function configured to divide a current image frame into a mesh of cells, wherein each cell comprises multiple image pixels, a comparison function configured to determine a metric of change for each cell by comparing pixel data of each cell with pixel data of a correspondingly positioned cell of a previous and/or subsequent image frame, and a storing function configured to store the metric of change for each cell as the motion metadata related to the current image frame.

Claims

1. A method for identifying motion in an encoded video, the method comprising:

decoding frames of the encoded video, thereby deriving image frames;

for at least some of the image frames;

dividing a current image frame into a mesh of cells, each cell comprising multiple image pixels,

determining a metric of change for each cell by;

comparing pixel data of each cell with pixel data of a correspondingly positioned cell of a previous and/or subsequent image frame,

identifying cells within the image frame for which the metric of change exceeds a change threshold, and

for each identified cell, determining a number of adjacent cells for which the metric of change exceeds the change threshold, and

storing the metric of change for each cell as motion metadata related to the current image frame,

wherein the motion metadata further comprises, for each identified cell, the number of adjacent cells for which the metric of change exceeds the change threshold;

setting a first region of interest of a scene depicted by the encoded video;

setting a first motion threshold;

accessing the stored motion metadata;

identifying image frames for which the metric of change in at least one cell overlapping with the first region of interest exceeds the first motion threshold, thereby identifying motion within the first region of interest.

2. The method according to claim 1, further comprising:

changing the first motion threshold; and

identifying image frames for which the metric of change in cells overlapping with the first region of interest exceeds the changed first motion threshold, thereby identifying motion within the first region of interest.

3. The method according to claim 1, further comprising:

setting a second region of interest of the scene, in addition to or as a replacement for the first region of interest;

setting a second motion threshold, wherein the first and second motion threshold are the same or different; and

identifying image frames for which the metric of change in cells overlapping with the second region of interest exceeds the second motion threshold, thereby identifying motion within the second region of interest.

4. The method according to claim 1, wherein each cell spans less than five percent of the image frame.

5. The method according to claim 1, wherein the image frames for which motion metadata is determined corresponds to intra coded frames of the encoded video.

6. A non-transitory computer readable recording medium having recorded thereon program code which when executed at a device having processing capabilities is configured to:

decode frames of the encoded video, thereby deriving image frames;

for at least some of the image frames;

divide a current image frame into a mesh of cells, each cell comprising multiple image pixels,

determine a metric of change for each cell by;

compare pixel data of each cell with pixel data of a correspondingly positioned cell of a previous and/or subsequent image frame,

identify cells within the image frame for which the metric of change exceeds a change threshold, and

for each identified cell, determine a number of adjacent cells for which the metric of change exceeds the change threshold, and

store the metric of change for each cell as motion metadata related to the current image frame,

wherein the motion metadata further comprises, for each identified cell, the number of adjacent cells for which the metric of change exceeds the change threshold;

set a first region of interest of a scene depicted by the encoded video;

set a first motion threshold;

access the stored motion metadata;

identify image frames for which the metric of change in at least one cell overlapping with the first region of interest exceeds the first motion threshold, thereby identifying motion within the first region of interest.

7. A method for determining motion metadata for an encoded video, the method comprising:

decoding frames of the encoded video, thereby deriving image frames;

for at least some of the image frames;

dividing a current image frame into a mesh of cells, each cell comprising multiple image pixels,

determining a metric of change for each cell by;

comparing pixel data of each cell with pixel data of a correspondingly positioned cell of a previous and/or subsequent image frame,

identifying cells within the current image frame for which the metric of change exceeds a change threshold, and

for each identified cell, determining a number of adjacent cells for which the metric of change exceeds the change threshold; and

storing the metric of change for each cell as motion metadata related to the current image frame,

wherein the motion metadata further comprises, for each identified cell, the number of adjacent cells for which the metric of change exceeds the change threshold.

8. The method according to claim 7, wherein the image frames for which motion metadata is determined corresponds to intra coded frames of the encoded video.

9. A non-transitory computer readable recording medium having recorded thereon program code which when executed at a device having processing capabilities is configured to:

decode frames of the encoded video, thereby deriving image frames;

for at least some of the image frames;

divide a current image frame into a mesh of cells, each cell comprising multiple image pixels,

determine a metric of change for each cell by;

compare pixel data of each cell with pixel data of a correspondingly positioned cell of a previous and/or subsequent image frame,

identify cells within the current image frame for which the metric of change exceeds a change threshold, and

for each identified cell, determine a number of adjacent cells for which the metric of change exceeds the change threshold; and

store the metric of change for each cell as motion metadata related to the current image frame,

wherein the motion metadata further comprises, for each identified cell, the number of adjacent cells for which the metric of change exceeds the change threshold.

10. A video processing device comprising:

a decoder configured to decode frames of an encoded video into image frames; and

a processing circuitry configured to execute a motion meta data deriving operation on image frames decoded by the decoder, wherein the motion meta data deriving operation comprises;

a dividing function configured to divide a current image frame into a mesh of cells, wherein each cell comprises multiple image pixels,

a comparison function configured to determine a metric of change for each cell by comparing pixel data of each cell with pixel data of a correspondingly positioned cell of a previous and/or subsequent image frame,

a cell identification function configured to;

identify cells within the current image frame for which the metric of change exceeds a change threshold, and

for each identified, determining a number of adjacent cells for which the metric of change exceeds the change threshold; and

a storing function configured to store the metric of change for each cell as the motion metadata related to the current image frame;

wherein the cell identification function is further configured to add the determined number of adjacent cells for which the metric of change exceeds the change threshold as part of the motion metadata for the identified cell.

11. The video processing device according to claim 10, wherein the video processing device form part of a video storage server configured to store the encoded video, the video processing device further comprising:

a resource monitor configured to determine available processing power for the processing circuitry, and upon the available processing power being above a threshold instruct the motion meta data deriving operation to determine the motion metadata.

12. The video processing device according to claim 10, wherein the video processing device comprises at least part of a client computer, the video processing device further comprising:

a transceiver configured to download the encoded video from an external source device;

wherein the storing function is further configured to upload the motion metadata to the external source device.