SUBTITLE DETECTION FOR STEREOSCOPIC VIDEO CONTENTS

• US 20140333720A1
• Filed: 04/25/2014
• Published: 11/13/2014
• Est. Priority Date: 05/08/2013
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: HUANG, Chao, Incesu, Yalcin, Motoyama, Takuto
• Granted Patent: US 9,762,889 B2
• Time in Patent Office: Days
• US Class Current: 348/42
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: H04N 13/183 On-screen display [OSD] information, e.g. subtitles or menus
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 21/4884 for displaying subtitles

Abstract

A right image and a left image is provided. Then a disparity estimation map relating to both images is provided and a left subtitle detection map for the left image and a right subtitle detection map for the right image are generated. Each map indicates subtitle areas within an image. For said subtitle areas and based on said disparity estimation map a subtitle disparity value for X and Y directions common for all subtitle areas is determined. Said left and right subtitle maps and said subtitle X and Y disparity values are used in an image interpolation process.

Claims

1. Method comprising:

providing a right image and a left image,

providing a disparity estimation map relating to both images,

generating a left subtitle detection map for the left image and a right subtitle detection map for the right image, each map indicating subtitle areas within an image,

for said subtitle areas and based on said disparity estimation map determining a subtitle disparity value for X and Y directions common for all subtitle areas,

using said left and right subtitle maps and said subtitle X and Y disparity values in an image interpolation process.

2. Method of claim 1, wherein generating a subtitle detection map comprises:

applying a text edge detection on said image for providing a binary text edges map indicating text specific edges, and

applying a motion detection on said image and a previous image for providing a still area map indicating areas with motion and without motion.

3. Method of claim 2, wherein said binary text edges map and said still area map are AND gated to provide a binary final edge map.

4. Method of claim 2, wherein said text edge detection comprises detecting up-down edge pairs in said image indicating up-down changes of luminance level.

5. Method of claim 4, wherein said detecting comprises applying a gradient filter, preferably a sobel filter.

6. Method of claim 5, comprising classifying said luminance level changes.

7. Method of claim 3, comprising

applying a temporal stabilization to said binary final edge map.

8. Method of claim 7, wherein said temporal stabilization comprises evaluating the continuity of successive binary final edge maps.

9. Method of claim 1, wherein contiguous subtitle areas are merged to rectangular bounding areas.

10. Method of claim 9, comprising evaluating said bounding areas and discarding bounding areas not fulfilling a predetermined criterion.

11. Method of claim 10, wherein said criterion is the size of the rectangular bounding area.

12. Method of claim 11, wherein a bounding area is discarded as invalid if its area size is smaller than a predetermined value.

13. Method of claim 10, wherein determining a subtitle disparity value comprises:

evaluating the disparity estimation map in the bounding areas and

determining a common disparity value for X- and Y-direction on the basis of the evaluating.

14. Method of claim 13, wherein evaluating comprises applying a histogram analysis of the disparity values in the bounding areas and selecting the disparity value of the highest frequency.

15. Method of claim 13, comprising

evaluating the consistency of the common disparity values in the bounding areas.

16. Method of claim 15, wherein evaluating the consistency comprises

post-matching the bounding areas of left and right images using said common disparity values and

generating binary left subtitle detection map and said binary right subtitle detection map, a binary value, preferably a binary 1, indicating a match.

18. A computer program comprising program code means for causing a computer to perform the steps of said method as claimed in claim 1 when said computer program is carried out on a computer.

19. A non-transitory computer-readable recording medium that stores therein a computer program product, which, when executed by a processor, causes the method according to claim 1 to be performed.

17. Device for detecting subtitle areas in 3D image content comprising

a disparity estimation unit for providing a disparity estimation map relating to a left and a right image,

a subtitle detection unit adapted to detect subtitle text areas in left and right images for providing left and right subtitle detection maps, and

a disparity extracting unit adapted to extract a common disparity X- and Y-value for the detected text areas.