System and method of hybrid tracking for match moving

• US 10,636,152 B2
• Filed: 11/14/2017
• Issued: 04/28/2020
• Est. Priority Date: 11/15/2016
• Status: Active Grant

• Current Assignee: Grass Valley Canada (Belden Inc.)
• Original Assignee: Gvbb Holdings SARL (Vislink Technologies, Inc.)
• Inventors: Takada, Yousuke
• Primary Examiner(s): Bekele, Mekonen T
• Application Number:
  US15/811,995
• Publication Number:
  US 20180137632A1
• Time in Patent Office: 896 Days
• CPC Class Codes: G06T 13/00 Animation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 30/248 involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

A system and method that maximizes tracking speed of an object in a sequence of images by selecting a technique for tracking the object independently for each frame in a video. The system includes an object feature detector that detects object features in a reference frame of the video and a feature comparator that determines a number of object features in each frame in the sequence of images that match the detected object features in the reference frame. Moreover, a tracking pattern selector selects the type of object tracking to track the object in the current frame based on the determined matched object features between the reference frame and the current frame of the video.

Claims

1. A video system for maximizing tracking speed of an object in a sequence of images, the system comprising:

a camera configured to generate a plurality of images as a sequence of images of a video;

an object feature detector configured to detect a plurality of object features in a reference frame;

a feature correspondence determiner configured to determine a number of corresponding object features in at least one image in the sequence of images of the video that match the detected object features in the reference frame; and

a tracking pattern selector configured to select one of a plurality of types of object tracking by comparing a number of the determined corresponding object features to a predetermined object pattern threshold level,

wherein the tracking pattern selector selects a first type of object tracking to track the object in the at least one image of the sequence of images when the determined number of corresponding object features does not exceed the predetermined object pattern threshold level to achieve a first tracking object speed of the object,

wherein the tracking pattern selector selects a second type of object tracking to track the object in the at least one image of the sequence of images when the determined number of corresponding object features exceeds the predetermined object pattern threshold level to achieve a second tracking object speed of the object,

wherein the detected object features in the reference frame comprises a corner pattern of the object,

wherein the second type of object tracking comprises corner based tracking of the object,

wherein the first type of object tracking comprises template based tracking of the object, and

wherein the selecting of the one type of object tracking for each image in the sequence of images of the video by the tracking pattern selector excludes the selected type of object tracking for a previous image in the sequence of images of the video.

2. The video system of claim 1, wherein tracking pattern selector is further configured to select one of the plurality of types of object tracking separately for each image of the plurality of images in the sequence of images in the video.

3. The video system of claim 2, wherein the tracking pattern selector selects the one type object tracking for each image independently of the selected type of object tracking for the previous image in the sequence of images in the video.

4. The video system of claim 1, wherein tracking pattern selector is further configured to compare a number of the detected plurality of object features in the reference frame to a minimum detected feature threshold and to automatically select the first type of object tracking to track the object in the sequence of images when the number of detected object features is less than the minimum detected feature threshold.

5. The video system of claim 1, further comprising an object position determiner configured to generate position data of the tracked object based on the selected one type of object tracking.

6. The video system of claim 5, further comprising a current tracking output configured to output the sequence of images of the video and the generated position data for the tracked object to a digital video editor to modify the tracked object in the video using the generated position data.

7. A video system for maximizing tracking speed of an object, the system comprising:

a camera configured to generate an image frame; and

an object pattern tracker having a pattern detector configured to detect a pattern feature of an object in the image frame of a video,

wherein the object pattern tracker has a detected pattern feature processor configured to determine whether the detected pattern feature of the object exceeds a predetermined object pattern threshold level,

wherein the object pattern tracker is configured to apply a first type of video tracking to track the object in the image frame when the detected pattern feature of the object does not exceed the predetermined object pattern threshold level to achieve a first tracking object speed of the object,

wherein the object pattern tracker is configured to apply a second type of video tracking to track the object in the image frame when the detected pattern feature of the object exceeds the predetermined object pattern threshold level to achieve a second tracking object speed of the object, and

wherein the object pattern tracker applies one of the first and second types of video tracking to track the object in the image frame independently of a selected type of video tracking for a previous image frame in the video.

8. The system of claim 7, wherein the detected pattern feature of the object comprises a corner pattern of the object.

9. The system of claim 8, wherein the first type of video tracking comprises template based tracking of the object and the second type of video tracking comprises corner based tracking of the object.

10. The system of claim 9, wherein the corner based tracking of the object comprises corner tracking of the object in a current image frame to generate position data of the object.

11. The system of claim 10, wherein the template based tracking of the object comprises template tracking of the object in the current image frame to generate position data of the object.

12. The system of claim 11, further comprising a current tracking output configured to output the image frame and the generated position data of the tracked object to a digital video editor to modify the tracked object in the image frame using the generated position data.

13. A video system for maximizing tracking speed of an object in a sequence of images by selecting a tracking type independently for each image frame in a video, the system comprising:

an object feature detector configured to detect a plurality of object features in a reference frame of the video;

a feature comparator configured to determine a number of object features in a current image frame in a sequence of images of the video that match the detected object features in the reference frame; and

a tracking pattern selector configured to select one of a plurality of types of object tracking to track an object in the current image frame based on the determined matched object features between the reference frame and the current image frame of the video,

wherein the tracking pattern selector is further configured to select the one of the plurality of types of object tracking to track the object by comparing a number of the determined matched object features to a predetermined object pattern threshold level,

wherein the tracking pattern selector selects a first type of object tracking of the plurality of types of object tracking when the number of determined matched objects does not exceed the predetermined object pattern threshold level to achieve a first tracking object speed of the object, and

wherein the tracking pattern selector selects a second type of object tracking of the plurality of types of object tracking when the number of determined matched objects exceeds the predetermined object pattern threshold level to achieve a second tracking object speed of the object.

14. The video system of claim 13, wherein the detected object features in the reference frame comprises a corner pattern of the object.

15. The video system of claim 14, wherein first type of object tracking comprises template based tracking of the object and the second type of object tracking comprises corner based tracking of the object.

16. The video system of claim 15, wherein the tracking pattern selector is further configured to compare a number of the detected plurality of object features in the reference frame to a minimum detected feature threshold and to automatically select the first type of object tracking to track the object in the sequence of images when the number of detected object features is less than the minimum detected feature threshold.

17. The video system of claim 13, wherein the tracking pattern selector selects the one type object tracking for each image frame independently of the selected type of object tracking for a previous image frame in the sequence of images of the video.

18. The video system of claim 13, further comprising:

an object position determiner configured to generate position data of the tracked object based on the selected type of object tracking; and

a current tracking output configured to output the sequence of images of the video and the generated position data for the tracked object to a digital video editor to modify the tracked object in the video using the generated position data.