Video feature extraction method and device

• US 11,055,536 B2
• Filed: 12/29/2018
• Issued: 07/06/2021
• Est. Priority Date: 03/29/2018
• Status: Active Grant

• Current Assignee: Beijing Bytedance Network Technology Co., Ltd.
• Original Assignee: Beijing Bytedance Network Technology Co., Ltd.
• Inventors: He, Yi, Li, Lei, Yang, Cheng, Li, Gen, Li, Yitan
• Primary Examiner(s): Haider, Syed
• Application Number:
  US16/768,527
• Publication Number:
  US 20200302183A1
• Time in Patent Office: 920 Days
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client

Abstract

A video feature extraction method and device are provided. The method includes extracting one or more frames from a video object to obtain one or more frames of images, obtaining one or more shift vectors for each of the one or more frames of images, using each of the one or more shift vectors, taking any pixel in each of the one or more frames of images as a starting point, determining a terminal point directed by the one or more shift vectors from the starting point, and determining an image feature of the one or more frames of images according to a difference between the starting point and the terminal point, and determining a video feature of the video object according to the image feature of the one or more frames of images.

Claims

1. A video feature extraction method, comprising:

extracting one or more frames from a video object to obtain one or more frames of images;

obtaining one or more shift vectors for each of the one or more frames of images, using each of the one or more shift vectors, taking any pixel in each of the one or more frames of images as a starting point, determining a terminal point directed by the one or more shift vectors from the starting point, and determining an image feature of the one or more frames of images according to a difference between the starting point and the terminal point, wherein the determining an image feature of the one or more frames of images according to a difference between the starting point and the terminal point further comprises;

determining the difference between each starting point and a corresponding terminal point in one of the one or more frames of images for each of the one or more shift vectors, and determining a feature bit of the one of the one or more frames of images based on an identical shift vector of the one or more shift vectors according to an overall situation of each difference based on the identical shift vector so as to obtain the feature bit of each of the one or more shift vectors, and

determining the image feature of the one of the one or more frames of images according to the feature bit of the one of the one or more frames of images based on the one or more shift vectors; and

determining a video feature of the video object according to the image feature of the one or more frames of images.

2. The video feature extraction method according to claim 1, wherein a length and a direction of the one or more shift vectors are arbitrary.

3. The video feature extraction method according to claim 2, wherein the difference between the starting point and the terminal point comprises a difference between brightness information of the starting point and brightness information of the terminal point.

4. The video feature extraction method according to claim 1, wherein the obtaining one or more shift vectors further comprises distributing a counter to each of the one or more shift vectors;

wherein the determining a feature bit of the one of the one or more frames of images based on an identical shift vector of the one or more shift vectors according to an overall situation of each difference based on the identical shift vector comprises;

increasing or reducing a value of the counter corresponding to the one or more shift vectors according to the each difference of the one or more shift vectors; and

comparing the value of the counter with a predetermined set value, and determining the feature bit of the one of the one or more frames of images based on the one or more shift vectors according to the comparison result.

5. The video feature extraction method according to claim 4, wherein the difference between the starting point and the terminal point comprises a difference between brightness information of the starting point and brightness information of the terminal point.

6. The video feature extraction method according to claim 1, wherein the difference between the starting point and the terminal point comprises a difference between brightness information of the starting point and brightness information of the terminal point.

7. The video feature extraction method according to claim 1, wherein the difference between the starting point and the terminal point comprises a difference between brightness information of the starting point and brightness information of the terminal point.

8. A device of extracting video feature, comprising:

at least one processor; and

at least one memory communicatively coupled to the at least one processor and storing instructions that upon execution by the at least one processor cause the device to;

extract one or more frames from a video object to obtain one or more frames of images;

obtain one or more shift vectors for each of the one or more frames of images, use each of the one or more shift vectors, take any pixel in each of the one or more frames of images as a starting point, determine a terminal point directed by the one or more shift vectors from the starting point, and determine an image feature of the one or more frames of images according to a difference between the starting point and the terminal point, wherein the instruction that upon execution causes the device to determine an image feature of the one or more frames of images according to a difference between the starting point and the terminal point further comprises instructions that upon execution by the at least one processor cause the device to;

determine the difference between each starting point and a corresponding terminal point in one of the one or more frames of images for each of the one or more shift vectors, and determine a feature bit of the one of the one or more frames of images based on an identical shift vector of the one or more shift vectors according to an overall situation of each difference based on the identical shift vector so as to obtain the feature bit of each of the one or more shift vectors, and

determine the image feature of the one of the one or more frames of images according to the feature bit of the one of the one or more frames of images based on the one or more shift vectors; and

determine a video feature of the video object according to the image feature of the one or more frames of images.

9. The device according to claim 8,

wherein a length and a direction of the one or more shift vectors are arbitrary.

10. The device according to claim 8,

wherein the instruction that upon execution causes the device to determine an image feature of the one or more frames of images according to a difference between the starting point and the terminal point further comprises instructions that upon execution by the at least one processor cause the device to:

determine the difference between each starting point and a corresponding terminal point in one of the one or more frames of images for each of the one or more shift vectors, and determine a feature bit of the one of the one or more frames of images based on an identical shift vector of the one or more shift vectors according to an overall situation of each difference based on the identical shift vector so as to obtain the feature bit of each of the one or more shift vectors; and

determine the image feature of the one of the one or more frames of images according to the feature bit of the one of the one or more frames of images based on the one or more shift vectors;

wherein the instruction that upon execution causes the device to obtain one or more shift vectors further comprises instructions that upon execution by the at least one processor cause the device to distribute a counter to each of the one or more shift vectors;

wherein the instruction that upon execution causes the device to determine the feature bit of the one of the one or more frames of images based on the identical shift vector of the one or more shift vectors according to the overall situation of the each difference based on the identical shift vector further comprises instructions that upon execution by the at least one processor cause the device to;

increase or reduce a value of the counter corresponding to the one or more shift vectors according to the each difference of the one or more shift vectors; and

compare the value of the counter with a predetermined set value, and determine the feature bit of the one of the one or more frames of images based on the one or more shift vectors according to the comparison result.

11. A non-transitory computer readable storage medium, configured for storing non-transitory computer readable instructions, when the non-transitory computer readable instructions are executed by a computer, causing the computer to perform operations, the operations comprising:

extracting one or more frames from a video object to obtain one or more frames of images;

obtaining one or more shift vectors for each of the one or more frames of images, using each of the one or more shift vectors, taking any pixel in each of the one or more frames of images as a starting point, determining a terminal point directed by the one or more shift vectors from the starting point, and determining an image feature of the one or more frames of images according to a difference between the starting point and the terminal point, wherein the determining an image feature of the one or more frames of images according to a difference between the starting point and the terminal point further comprises;

determining the difference between each starting point and a corresponding terminal point in one of the one or more frames of images for each of the one or more shift vectors, and determining a feature bit of the one of the one or more frames of images based on an identical shift vector of the one or more shift vectors according to an overall situation of each difference based on the identical shift vector so as to obtain the feature bit of each of the one or more shift vectors, and

determining the image feature of the one of the one or more frames of images according to the feature bit of the one of the one or more frames of images based on the one or more shift vectors; and

determining a video feature of the video object according to the image feature of the one or more frames of images.

12. The non-transitory computer readable storage medium according to claim 11, wherein a length and a direction of the one or more shift vectors are arbitrary.

13. The non-transitory computer readable storage medium according to claim 12, wherein the difference between the starting point and the terminal point comprises a difference between brightness information of the starting point and brightness information of the terminal point.

14. The non-transitory computer readable storage medium according to claim 11, wherein the obtaining one or more shift vectors further comprises distributing a counter to each of the one or more shift vectors;

wherein the determining the feature bit of the one of the one or more frames of images based on the identical shift vector of the one or more shift vectors according to the overall situation of the each difference based on the identical shift vector comprises;

increasing or reducing a value of the counter corresponding to the one or more shift vectors according to the each difference of the one or more shift vectors; and

comparing the value of the counter with a predetermined set value, and determining the feature bit of the one of the one or more frames of images based on the one or more shift vectors according to the comparison result.

15. The non-transitory computer readable storage medium according to claim 14, wherein the difference between the starting point and the terminal point comprises a difference between brightness information of the starting point and brightness information of the terminal point.

16. The non-transitory computer readable storage medium according to claim 11, wherein the difference between the starting point and the terminal point comprises a difference between brightness information of the starting point and brightness information of the terminal point.

17. The non-transitory computer readable storage medium according to claim 11, wherein the difference between the starting point and the terminal point comprises a difference between brightness information of the starting point and brightness information of the terminal point.