MOTION AND APPARATUS FOR SPATIO-TEMPORAL DEINTERLACING AIDED BY MOTION COMPENSATION FOR FIELD-BASED VIDEO

• US 20070206117A1
• Filed: 09/29/2006
• Published: 09/06/2007
• Est. Priority Date: 10/17/2005
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Shi, Fang, Raveendran, Vijayalakshmi, Tian, Tao
• Application Number:
  US11/536,894
• Publication Number:
  US 20070206117A1
• Time in Patent Office: Days
• US Class Current: 348/452
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229

Abstract

The invention comprises devices and methods for processing multimedia data to generate progressive frame data from interlaced frame data. In one aspect, a method of processing multimedia data includes generating spatio-temporal information for a selected frame of interlaced multimedia data, generating motion information for the selected frame, and deinterlacing fields of the selected frame based on the spatio-temporal information and the motion information to form a progressive frame associated with the selected frame. In another aspect an apparatus for processing multimedia data can include a spatial filter module configured to generate spatio-temporal information of a selected frame of interlaced multimedia data, a motion estimator configured to generate motion information for the selected frame, and a deinterlacer configured to deinterlace fields of the selected frame and form a progressive frame corresponding to the selected frame based on the spatio-temporal information and the motion information.

Claims

1. A method of processing multimedia data, the method comprising:

generating spatio-temporal information for a selected frame of interlaced multimedia data;

generating motion compensation information for the selected frame; and

deinterlacing fields of the selected frame based on the spatio-temporal information and the motion compensation information to form a progressive frame associated with the selected frame.

2. The method of claim 1, wherein generating spatio-temporal information comprises generating a spatio-temporal provisional deinterlaced frame, wherein generating motion information comprises generating a motion compensated provisional deinterlaced frame, and wherein deinterlacing fields of the selected frame further comprises combining said spatio-temporal provisional deinterlaced frame and said motion compensated provisional deinterlaced frame to form the progressive frame.

3. The method of claim 1, further comprising using motion vector candidates to generate said motion compensation information.

4. The method of claim 1, further comprising

receiving motion vector candidates;

determining motion vectors based on said motion vector candidates; and

using said motion vectors to generate the motion compensation information.

5. The method of claim 1, further comprising

determining a motion vector candidate for a block of video data in the selected frame from motion vector estimates of its neighboring blocks;

and

using said motion vector candidates to generate the motion compensation information.

6. The method of claim 1, wherein generating spatio-temporal information comprises:

generating at least one motion intensity map; and

generating a provisional deinterlaced frame based on the motion intensity map, wherein said deinterlacing comprises using the provisional deinterlaced frame and the motion information to generate the progressive frame.

7. The method of claim 6, wherein generating a provisional deinterlaced frame comprises spatial filtering the interlaced multimedia data if the at least one motion intensity map indicates a selected condition.

8. The method of claim 6, wherein generating at least one motion intensity map comprises classifying regions of the selected frame into different motion levels.

9. The method of claim 8, wherein generating at least one motion intensity map comprises spatial filtering the interlaced multimedia data based on the different motion levels.

10. The method of claim 1, wherein spatial filtering comprises processing the interlaced multimedia data using a weighted median filter.

11. The method of claim 6, wherein generating a provisional deinterlaced frame comprises spatial filtering across multiple fields of the interlaced multimedia data based on the motion intensity map.

12. The method of claim 1, wherein generating spatio-temporal information comprises spatio-temporal filtering across a temporal neighborhood of fields of a selected current field.

13. The method of claim 12, wherein the temporal neighborhood comprises a previous field that is temporally located previous to the current field, and comprises a next field that is temporally located subsequent to the current field.

14. The method of claim 12, wherein the temporal neighborhood comprises a plurality of previous fields that are temporally located previous to the current field, and comprises a plurality of next fields that are temporally located subsequent to the current field.

15. The method of claim 1, wherein generating spatio-temporal information comprises generating a provisional deinterlaced frame based on spatio-temporal filtering and filtering said provisional deinterlaced frame using a denoising filter.

16. The method of claim 15, wherein deinterlacing fields of the selected frame comprises combining the denoised provisional deinterlaced frame with motion information to form said progressive frame.

17. The method of claim 15, wherein said denoising filter comprises a wavelet shrinkage filter.

18. The method of claim 15, wherein said denoising filter comprises a Weiner filter.

19. The method of claim 1, wherein generating motion information comprises performing bi-directional motion estimation on the selected field to generate motion vectors, and performing motion compensation using the motion vectors.

20. The method of claim 1, further comprising:

generating a provisional deinterlaced frame associated with the selected frame based on the spatio-temporal information;

obtaining motion vectors on the provisional deinterlaced frame; and

performing motion compensation using the motion vectors to generate the motion information, wherein the motion information comprises a motion compensated frame, and

wherein deinterlacing comprises combining the motion compensated frame and the provisional deinterlaced frame.

21. The method of claim 20, further comprising:

generating a sequence of provisional deinterlaced frames in a temporal neighborhood around the selected frame based on the spatio-temporal information; and

generating motion vectors using the sequence of provisional deinterlaced frames.

22. The method of claim 20, wherein performing motion compensation comprises performing bidirectional motion compensation.

23. The method of claim 21, further comprising denoising filtering the provisional deinterlaced frame.

24. The method of claim 21, wherein the sequence of provisional interlaced frames comprises a provisional deinterlaced frame of the multimedia data previous to the provisional deinterlaced frame of the selected frame and a provisional deinterlaced frame of the multimedia data subsequent to the provisional deinterlaced frame of the selected frame.

30. The apparatus of claim 24, wherein the motion estimator is configured to use a previously generated progressive frame to generate at least a portion of the motion information.

25. An apparatus for processing multimedia data, comprising:

a filter module configured to generate spatio-temporal information of a selected frame of interlaced multimedia data;

a motion estimator configured to generate bidirectional motion information for the selected frame; and

a combiner configured to form a progressive frame associated with the selected frame using the spatio-temporal information and the motion information.

26. The apparatus of claim 25, further comprising a denoiser configured to remove noise from the spatio-temporal information.

27. The apparatus of claim 25, wherein said spatio-temporal information comprises a spatio-temporal provisional deinterlaced frame, wherein said motion information comprises a motion compensated provisional deinterlaced frame, and wherein said combiner is further configured to form the progressive frame by combining said spatio-temporal provisional deinterlaced frame and said motion compensated provisional deinterlaced frame

28. The apparatus of claim 25, wherein the motion information is bi-directional motion information.

29. The apparatus of claim 26, wherein said filter module is further configured to determine a motion intensity map of the selected frame and use the motion intensity map to generate a spatio-temporal provisional deinterlaced frame, and said combiner is configured to form the progressive frame by combining the motion information with the spatio-temporal provisional deinterlaced frame.

31. An apparatus for processing multimedia data comprising:

means for generating spatio-temporal information for a selected frame of interlaced multimedia data;

means for generating motion information for the selected frame; and

means for deinterlacing fields of the selected frame based on the spatio-temporal information and the motion information to form a progressive frame associated with the selected frame.

32. The apparatus of claim 31, wherein the spatio-temporal information comprises a spatio-temporal provisional deinterlaced frame, wherein the motion information comprises a motion compensated provisional deinterlaced frame, and wherein said deinterlacing means comprises means for combining the spatio-temporal provisional deinterlaced frame and the motion compensated provisional deinterlaced frame to form the progressive frame.

33. The apparatus of claim 31, wherein the deinterlacing means comprise a combiner configured to form the progressive frame by combining spatial temporal information and motion information.

34. The apparatus of claim 31, wherein the motion information comprises bi-directional motion information.

35. The apparatus of claim 32, wherein the generating spatio-temporal information means is configured to generate a motion intensity map of the selected frame and to use the motion intensity map to generate a spatio-temporal provisional deinterlaced frame, and wherein said combining means is configured to form the progressive frame by combining the motion information with the spatio-temporal provisional deinterlaced frame.

36. The apparatus of claim 31, wherein the generating spatio-temporal information means is configured to:

generate at least one motion intensity map; and

generate a provisional deinterlaced frame based on the motion intensity map,

wherein the deinterlacing means is configured to generate the progressive frame using the provisional deinterlaced frame and the motion information

37. The apparatus of claim 36, wherein generating a provisional deinterlaced frame comprises spatial filtering the interlaced multimedia data if the at least one motion intensity map indicates a selected condition.

38. The apparatus of claim 36, wherein generating at least one motion intensity map comprises classifying regions of the selected frame into different motion levels.

39. The apparatus of claim 38, wherein generating at least one motion intensity map comprises spatial filtering the interlaced multimedia data based on the different motion levels.

40. A machine readable medium comprising instructions for processing multimedia data, wherein the instructions upon execution cause a machine to:

generate spatio-temporal information for a selected frame of interlaced multimedia data;

generate bi-directional motion information for the selected frame; and

deinterlace fields of the frame based on the spatio-temporal information and the motion information to form a progressive frame corresponding to the selected frame.

41. A processor for processing multimedia data, said processor being configured to:

generate spatio-temporal information of a selected frame of interlaced multimedia data;

generate motion information for the selected frame; and

deinterlace fields of the selected frame to form a progressive frame associated with the selected frame based on the spatio-temporal information and the motion information.