Confidence estimation for optical flow

• US 10,055,674 B2
• Filed: 03/18/2016
• Issued: 08/21/2018
• Est. Priority Date: 03/20/2015
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Mathew, Manu, Garud, Hrushikesh Tukaram, Nagori, Soyeb Noormohammed
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/073,940
• Publication Number:
  US 20160275377A1
• Time in Patent Office: 886 Days
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A confidence map for optical flow gradients is constructed calculating a set of gradients for each pixel of an image, filtering said gradients and extracting confidence values from said gradients using a plurality of decision tree classifiers. A confidence map is then generated from said confidence values.

Claims

1. A method of confidence estimation for optical flow comprising the steps of:

computing a set of features for each pixel of an input image, wherein the set of features includes an image feature, a matching cost, and a flow feature, wherein the image feature includes an image gradient feature, wherein the flow feature includes a flow gradient feature, and wherein computing the set of features comprises;

computing a gradient Gu of a U component of the optical flow;

computing a gradient Gv of a V component of the optical flow;

computing a cost (or distance) C0 corresponding to a best match;

computing a Delta matching cost Cd; and

computing a gradient magnitude Gm of the input image;

computing a smoothed version of each feature of the set of features;

constructing a feature vector for each pixel of the input based on the set of features and the smoothed versions of the set of features;

computing a classifier score from the feature vector using a set of decision tree classifiers; and

converting the classifier score into a confidence score, the confidence score representing a confidence of the optical flow.

2. The method of claim 1, wherein:

computing either the image gradient or the flow gradient is implemented as follows;

g=[abs(X(i,j)−

X(i−

1,j))+abs(X(i,j)−

X(i,j−

1))]G(i,j)=min((g<

<

GRADIENT_SHIFT),GRADIENT_MAX),

wherein X is an input for a location (i,j).

3. The method of claim 1, wherein:

computing the smoothed versions of the image gradient feature and the flow gradient feature comprises;

applying an FIR filter to the image gradient feature and the flow gradient feature in the horizontal dimension; and

applying an IIR filter to the image gradient feature and the flow gradient feature in the vertical dimension.

4. The method of claim 3, wherein:

Said FIR filter is implemented as follows;

SmoothH(I,j)=sum(X(i,j−

8+m));

m=0, 1, . . . 17

wherein X is an input gradient to a smoothing function and Smooth(i,j) is a horizontally filtered gradient.

5. The method of claim 3, wherein:

the IIR filter is implemented as follows;

S(i,j)=(SmoothH(i,j)*SMOOTH_ALPHA+S(i−

1,j)*SMOOTH_BETA),

wherein S(i,j) is an output of a smoothing operation, SmoothH(i,j) is an output of the FIR filter, and S(i−

1,j) is a corresponding output from previous line of the optical flow.

6. The method of claim 1, wherein:

computing the smoothed version of each feature of the set of features comprising;

computing a smoothed version Su of Gu;

computing a smoothed version Sv of Gv;

computing a smoothed version S0 of C0;

computing a smoothed version Sd of Cd; and

computing a smoothed version Sm of Gm;

the feature vector is constructed using Gu, Gv, C0, Cd, Gm, Su, Sv, S0, Sd, and Sm, and is processed by the set of decision tree classifiers to extract confidence values.

7. The method of claim 1, wherein:

the confidence score is determined based at least partially on a confidence map constructed from accumulated classifier scores from the set of decision tree classifiers.

8. A method of confidence estimation for optical flow comprising the steps of:

computing a set of features for each pixel of an input image, wherein the set of features includes an image feature, a matching cost, and a flow feature;

computing a smoothed version of each feature of the set of features by;

computing a smoothed version Su of Gu, wherein Gu is a gradient of a U component of the optical flow;

computing a smoothed version Sv of Gv, wherein Gv is a gradient of a V component of the optical flow;

computing a smoothed version S0 of C0, wherein C0 is a cost or distance corresponding to a best match;

computing a smoothed version Sd of Cd, wherein Cd is a delta matching cost; and

computing a smoothed version Sm of Gm, wherein Gm is a gradient magnitude of the input image;

constructing a feature vector for each pixel of the input image;

computing a classifier score from the feature vector using set of decision tree classifiers; and

converting the classifier score into a confidence score, the confidence score representing a confidence of the optical flow.

9. The method of claim 8, wherein:

the feature vector is constructed using Gu, Gv, C0, Cd, Gm, Su, Sv, S0, Sd, and Sm, and is processed by the set of decision tree classifier to extract confidence values.

10. The method of claim 8, wherein:

the confidence score is determined based at least partially on a confidence map constructed from accumulated classifier scores from the set of decision tree classifiers.