Fallback in frame rate conversion system

• US 9,602,762 B2
• Filed: 06/12/2015
• Issued: 03/21/2017
• Est. Priority Date: 06/12/2015
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Chen, Xu, Van Beek, Petrus J. L.
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US14/738,056
• Publication Number:
  US 20160366367A1
• Time in Patent Office: 648 Days
• Field of Search: 348/441, 348/443, 348/458, 348/459
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

A system for frame rate conversion of a video that includes detecting low quality of a received frame and determining an interpolated frame of the video based upon the detected low quality of the received frame.

Claims

1. A method for frame rate conversion of a video comprising:

(a) receiving a plurality of frames of said video;

(b) detecting a low quality of a received frame of said video;

(c) determining an interpolated frame of said video temporally proximate said frame of said video based upon said detecting said low quality of said received frame;

(d) wherein said determining said interpolated frame of said video temporally proximate said frame of said video is based upon a reliability factor of motion vectors of said frame;

(e) wherein said determining said interpolated frame based upon said reliability factor of said motion vectors is selectively used to (1) determine pixels of said interpolated frame using said motion vectors when said reliability factor of said motion vectors of said frame is sufficiently high, and (2) not determine said pixels of said interpolated frame using said motion vectors when said reliability factor of said motion vectors of said frame is not said sufficiently high.

2. The method of claim 1 wherein said detecting said low quality said received frame is based on a blur confidence measure.

3. The method of claim 1 wherein said blur estimation is based upon a presence of fast motion dominating a large portion of said frame.

4. The method of claim 2 wherein said blur confidence measure is based on an analysis of edges of said frame.

10. The method of claim 1 wherein said reliability factor of said motion vectors is a spatial error measure.

11. The method of claim 1 wherein said reliability factor of said motion vectors is a temporal error measure.

12. The method of claim 10 wherein said spatial error measure is a spatial motion vector variance.

13. The method of claim 11 wherein said temporal error measure is a temporal motion compensated error.

14. The method of claim 13 wherein said temporal motion compensated error is based upon a block based motion compensation.

5. A method for frame rate conversion of a video comprising:

(a) receiving a plurality of frames of said video;

(b) detecting a low quality of a received frame of said video;

(c) determining an interpolated frame of said video temporally proximate said frame of said video based upon said detecting said low quality of said received frame;

(d) wherein said determining said interpolated frame of said video temporally proximate said frame of said video is based upon a reliability factor of motion vectors of said frame;

(e) wherein said detecting said low quality said received frame is based on a blur confidence measure;

(f) wherein said blur confidence measure is based on an analysis of edges of said frame;

(g) wherein said blur confidence measure is based on an analysis of the sharpness of said edges of said frame.

6. A method for frame rate conversion of a video comprising:

(a) receiving a plurality of frames of said video;

(b) detecting a low quality of a received frame of said video;

(c) determining an interpolated frame of said video temporally proximate said frame of said video based upon said detecting said low quality of said received frame;

(d) wherein said determining said interpolated frame of said video temporally proximate said frame of said video is based upon a reliability factor of motion vectors of said frame;

(e) wherein said detecting said low quality said received frame is based on a blur confidence measure;

(f) wherein said blur confidence measure is based on an analysis of edges of said frame;

(g) wherein said blur confidence measure is based on a wavelet transform.

7. A method for frame rate conversion of a video comprising:

(a) receiving a plurality of frames of said video;

(b) detecting a low quality of a received frame of said video;

(c) determining an interpolated frame of said video temporally proximate said frame of said video based upon said detecting said low quality of said received frame;

(d) wherein said determining said interpolated frame of said video temporally proximate said frame of said video is based upon a reliability factor of motion vectors of said frame;

(e) wherein said detecting said low quality said received frame is based on a blur confidence measure;

(f) wherein said blur estimation is based upon a presence of fast motion dominating a large portion of said frame;

(g) wherein said fast motion is based upon a magnitude of motion vectors of said frame.

8. The method of claim 7 wherein said fast motion is based upon a horizontal component of said motion vectors of said frame.

9. The method of claim 8 wherein said fast motion is based upon a vertical component of said motion vectors of said frame.

15. A method for frame rate conversion of a video comprising:

(a) receiving a plurality of frames of said vide;

(b) detecting a low quality of a received frame of said video;

(c) determining an interpolated frame of said video temporally proximate said frame of said video based upon said detecting said low quality of said received frame;

(d) wherein said determining said interpolated frame of said video temporally proximate said frame of said video is based upon a reliability factor of motion vectors of said frame;

(e) wherein said reliability factor of said motion vectors is a temporal error measure;

(f) wherein said temporal error measure is a temporal motion compensated error;

(g) wherein said temporal motion compensated error is based upon a block based motion compensation;

(h) wherein said temporal motion compensated error includes weighting central blocks of said frame to a greater degree than non-central blocks of said frame.

16. A method for frame rate conversion of a video comprising:

(a) receiving a plurality of frames of said video;

(b) detecting a low quality of a received frame of said video;

(c) determining an interpolated frame of said video temporally proximate said frame of said video based upon said detecting said low quality of said received frame;

(d) wherein said determining said interpolated frame of said video temporally proximate said frame of said video is based upon a reliability factor of motion vectors of said frame;

(e) wherein said reliability factor of said motion vectors is a spatial error measure

(f) wherein said spatial error measure is a spatial motion vector variance;

(g) wherein said spatial motion vector variance includes a comparison between a local variance and a global mean of said local variance.