Recognition data generation device, image recognition device, and recognition data generation method

• US 10,616,613 B2
• Filed: 07/13/2015
• Issued: 04/07/2020
• Est. Priority Date: 07/17/2014
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Yabu, Hiroshi
• Primary Examiner(s): Le, Rong
• Application Number:
  US15/301,444
• Publication Number:
  US 20170034542A1
• Time in Patent Office: 1,730 Days
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/44 Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 21/8453 by locking or enabling a set of features, e.g. optional functionalities in an executable program

Abstract

A recognition data creation device includes an image acquirer and a data creator. The image acquirer acquires a plurality of image frames included in a frame sequence that composes video content. Based on an inter-frame change in an image in the plurality of image frames acquired by the image acquirer, the data creator creates recognition data representing the video content, the recognition data to be used as a fingerprint in recognition of the video content.

Claims

1. An image recognition device comprising:

an image acquirer that acquires a plurality of image frames included in a frame sequence that composes video content; and

a data creator that creates recognition data representing the video content, the recognition data including at least one of a static fingerprint and a dynamic fingerprint, the static fingerprint being based on a static region in which an inter-frame image variation between the plurality of image frames acquired by the image acquirer is smaller than a first threshold, the dynamic fingerprint being based on a dynamic region in which the inter-frame image variation is larger than a second threshold,

wherein the data creator includes;

a static fingerprint creation part that creates a static frame by filtering one of two temporally adjacent image frames in the static region and defines the static frame as the static fingerprint; and

a dynamic fingerprint creation part that creates a dynamic frame by filtering an image-changed frame in the dynamic region and defines the dynamic frame as the dynamic fingerprint, wherein the image recognition device further comprises;

an acquirer that acquires a plurality of pieces of recognition data which represent respective pieces of video content; and

a collator that collates the recognition data created in the data creator with the plurality of pieces of recognition data acquired by the acquirer, wherein the data creator creates recognition data including at least one static fingerprint and at least one dynamic fingerprint, wherein the collator collates each of the static fingerprints and the dynamic fingerprints which are included in the recognition data created in the data creator with the plurality of pieces of recognition data acquired by the acquirer to select recognition data similar to the static fingerprint or the dynamic fingerprint, and output, as a collation result, information corresponding to the selected recognition data, and wherein the collator selects the recognition data similar to at least two of the static fingerprints and the dynamic fingerprints which are included in the recognition data created in the data creator, and outputs, as the collation result, information corresponding to the selected recognition data.

2. The image recognition device according to claim 1, wherein the data creator includes:

a difference calculator that creates the image-changed frame by calculating, as a pixel-changed value, a pixel-value difference between the two temporally adjacent image frames among the plurality of image frames acquired by the image acquirer;

a decision section that decides at least one of a static region in which an absolute value of the pixel-changed value in the image-changed frame is smaller than the first threshold and the dynamic region in which the absolute value of the pixel-changed value in the image-changed frame is larger than the second threshold; and

a creation section that creates at least one of the static fingerprint and the dynamic fingerprint based on at least one of the static region and the dynamic region which are decided by the decision section.

3. The image recognition device according to claim 2, wherein the creation section creates the static fingerprint based on the static region in a case where the static region occupies a first ratio or more of the image-changed frame.

4. The image recognition device according to claim 2, wherein the creation section creates the dynamic fingerprint based on the dynamic region in a case where the dynamic region occupies a second ratio or more of the image-changed frame.

5. An image recognition method comprising:

a first acquisition step of acquiring a plurality of image frames included in a frame sequence that composes video content; and

a creation step of creating recognition data representing the video content, the recognition data including at least one of a static fingerprint and a dynamic fingerprint, the static fingerprint being based on a static region in which an inter-frame image variation between the plurality of image frames is smaller than a first threshold, the dynamic fingerprint being based on a dynamic region in which the inter-frame image variation is larger than a second threshold, wherein the creation step comprises;

creating a static frame by filtering one of two temporally adjacent image frames in the static region, and defining the static frame as the static fingerprint; and

creating a dynamic frame by filtering an image-changed frame in the dynamic region, and defining the dynamic frame as the dynamic fingerprint, wherein the image recognition method further comprises;

a second acquisition step of acquiring a plurality of pieces of recognition data which represent respective pieces of video content; and

a collating step of collating the recognition data created by the creation step with the plurality of pieces of recognition data acquired by the second acquisition step,

wherein the creation step creates recognition data including at least one static fingerprint and at least one dynamic fingerprint,

wherein the collating step collates each of the static fingerprints and the dynamic fingerprints which are included in the recognition data created in the creation step with the plurality of pieces of recognition data acquired by the second acquisition step to select recognition data similar to the static fingerprint or the dynamic fingerprint, and outputs, as a collation result, information corresponding to the selected recognition data, and wherein the collating step selects the recognition data similar to at least two of the static fingerprints and the dynamic fingerprints which are included in the recognition data created in the creation step, and outputs, as the collation result, information corresponding to the selected recognition data.

6. A non-transitory computer-readable recording medium storing a computer program for causing a computer to execute the image recognition method according to claim 5.