Object identification system and method

• US 9,122,931 B2
• Filed: 10/25/2013
• Issued: 09/01/2015
• Est. Priority Date: 10/25/2013
• Status: Active Grant

• Current Assignee: TCL Research America, Inc. (TCL Technology Group Corp.)
• Original Assignee: TCL Research America, Inc. (TCL Technology Group Corp.)
• Inventors: Wang, Haohong, Peng, Liang
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Kholdebarin, Iman K
• Application Number:
  US14/063,511
• Publication Number:
  US 20150117703A1
• Time in Patent Office: 676 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49

Abstract

An object identification method is provided. The method includes dividing an input video into a number of video shots, each containing one or more video frames. The method also includes detecting target-class object occurrences and related-class object occurrences in each video shot. Further, the method includes generating hint information including a small subset of frames representing the input video and performing object tracking and recognition based on the hint information. The method also includes fusing tracking and recognition results and outputting labeled objects based on the combined tracking and recognition results.

Claims

1. An object identification method, comprising:

dividing an input video into a number of video shots, each containing one or more video frames;

detecting target-class object occurrences and related-class object occurrences in each video shot;

generating hint information including a small subset of frames representing the input video;

performing object tracking and recognition based on the hint information;

combining tracking and recognition results; and

outputting labeled objects based on the combined tracking and recognition results;

wherein generating hint information including a small subset of frames representing the input video further includes;

applying a video summarization approach;

applying k-means clustering algorithms using identify-sensitive features on all detected objects; and

performing initial object identification using a knowledge database;

wherein the hint information possesses a desired identity coverage, a desired local representation, a desired pose variation coverage and a desired illumination variation coverage.

2. The method according to claim 1, wherein performing object tracking and recognition based on the hint information further includes:

classifying each of detected objects into one single class by using a set of static images.

3. The method according to claim 1, wherein applying k-means clustering algorithms using identify-sensitive features on all detected objects further includes:

selecting the frames which contain k centroid objects, where k equals to a total number of objects need to be identified in the input video.

4. The method according to claim 1, wherein:

provided that i, p, q are integers;

L is a feature vector that is sensitive to object identity;

Sim(·

) is a function that measures similarity, a local representation of the ith frame is define by;

5. The method according to claim 1, wherein:

the desired pose variation coverage is interpreted as that object regions of consecutive selected frames have a large dissimilarity in terms of pose; and

provided that i, j, p, q are integers;

G denotes a feature vector that is sensitive to different poses of an object, the pose similarity of key frames is defined by;

6. The method according to claim 1, wherein:

the desired illumination variation coverage is interpreted as that object regions of consecutive selected frames have large dissimilarity in terms of illumination; and

provided that i, j, p, q are integers;

Lum denotes the illumination from RGB image, the illumination similarity of summary frames is defined by;

7. An object identification system, comprising:

an object detection module configured to detect target-class object occurrences and related-class object occurrences in each video shot of an input video;

a hint information generation module configured to generate hint information including a small subset of frames representing the input video;

an object tracking and recognition module configured to perform object tracking and recognition based on the hint information and to combine tracking and recognition results; and

an output module configured to output labeled objects based on the combined tracking and recognition results;

wherein the hint information generation module is further configured to;

apply a video summarization approach;

apply k-means clustering algorithms using identify-sensitive features on all detected objects; and

perform initial object identification using a knowledge database

wherein the hint information possesses a desired identity coverage, a desired local representation, a desired pose variation coverage and a desired illumination variation coverage.

8. The system according to claim 7, wherein the object tracking and recognition module is further configured to:

classify each of detected objects into one single class by using a set of static images.

9. The system according to claim 7, wherein:

the k-means clustering algorithms are applied using identify-sensitive features on all detected objects to select the frames which contain the k centroid objects, where k equals to a total number of objects need to be identified in the video.

10. The system according to claim 7, wherein:

provided that i, p, q are integers;

L is a feature vector that is sensitive to object identity;

Sim(·

) is a function that measures similarity, the local representation of the ith frame is define by;

11. The system according to claim 7, wherein:

the desired pose variation coverage is interpreted as that object regions of consecutive selected frames have a large dissimilarity in terms of pose; and

provided that i, j, p, q are integers;

G denotes a feature vector that is sensitive to different poses of an object, the pose similarity of key frames is defined by;

12. The system according to claim 7, wherein:

the desired illumination variation coverage is interpreted as that object regions of consecutive selected frames have large dissimilarity in terms of illumination; and

provided that i, j, p, q are integers;

Lum denotes the illumination from RGB image, the illumination similarity of summary frames is defined by;