Automatic caption text detection and processing for digital images

• US 6,185,329 B1
• Filed: 10/13/1998
• Issued: 02/06/2001
• Est. Priority Date: 10/13/1998
• Status: Expired due to Term

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Zhang, Hong Jiang, Zhong, Yu
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Novik, Dmitry A.
• Application Number:
  US09/170,995
• Time in Patent Office: 847 Days
• Field of Search: 382/176, 382/233, 382/236, 382/172, 382/276, 382/282, 382/290, 382/291, 382/292, 382/173, 382/239, 358/462, 358/464, 348/403, 348/404, 348/468
• US Class Current: 382/176
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: H04N 1/40062 Discrimination between different image types, e.g. two-tone, continuous tone

Abstract

A texture-based text localization system proceeds directly in the compressed domain for DCT compressed JPEG images or MPEG videos. The DCT coefficient values in JPEG images and MPEG videos, which capture the directionality and periodicity of local image blocks, are used as texture feature measures to classify text areas. Each unit block in the compressed images is classified as either text or nontext. In addition, post-processing in both the compressed domain and the reconstructed candidate text areas can be used to refine the results. For video frames that contain text, the displacement of text between two consecutive frames is estimated which gives the velocity of the moving text. This temporal displacement information is also used to further refine the localization results. The text is then processed to provide content or speech output.

Claims

1. A text detection system for DCT compressed images comprising:

a calculation mechanism for calculating variations of a first energy in said DCT compressed images whereby predetermined values of said first energy indicate possible text areas in said DCT compressed images;

a threshold mechanism for screening for said predetermined values and outputting potential text areas;

a detecting mechanism for detecting variations of a second energy in said potential text areas indicative of the possibility of text in said potential text areas and outputting detected text areas; and

a reconstruction mechanism for decompressing said DCT compressed images detected text areas.

2. The text detection system as claimed in claim 1 wherein said calculation mechanism for calculating variations of said first energy calculates a horizontal text energy for each DCT compressed images area.

3. The text detection system as claimed in claim 2 wherein said threshold mechanism detects DCT compressed images areas below the predetermined horizontal text energy which is required for processing potential text areas to said detecting mechanism.

4. The text detection system as claimed in claim 1 wherein said potential text areas contain noise and are disconnected, and including a morphological mechanism for removing noisy areas and connecting disconnected text areas.

5. The text detection system as claimed in claim 1 wherein said detecting mechanism for detecting variations of said second energy detects vertical energy and outputs detected text areas having vertical energy over a predetermined value.

6. The text detection system as claimed in claim 1 wherein said reconstruction mechanism includes a calculation mechanism for calculating vertical edge components in said detected text areas.

7. The text detection system as claimed in claim 6 including:

second threshold means for screening probable text areas where the calculated vertical edge components provide a density of horizontally aligned edges greater than a predetermined threshold value.

8. The text detection system as claimed in claim 1 including:

temporal adjustment means mechanism to adjust consecutive frame images containing said detected text areas;

an estimation mechanism for estimating the displacement of text in two consecutive frame images of said detected text areas; and

an elimination mechanism for eliminating said two consecutive frames where there are no corresponding detected text areas.

9. The text detection system as claimed in claim 8 including:

a third threshold mechanism for screening for frame images having corresponding text areas before and after a predetermined frame image.

10. The text detection system as claimed in claim 1 including:

an output for outputting said text from said detected text areas; and

processing means for providing a text content from said output text.

11. The text detection system as claimed in claim 1 including:

an output for outputting said text from said detected text areas; and

synthesizing means for providing speech output from said output text.

12. A method of text detection for DCT compressed images comprising the steps of:

calculating variations of a first energy in said DCT compressed images whereby predetermined values of said first energy indicate possible text areas in said DCT compressed images;

screening for said predetermined values and outputting potential text areas related thereto;

deleting variations of a second energy in said potential text areas indicative of the possibility of text in said potential text areas and outputting detected text areas; and

decompressing said DCT compressed image detected text areas.

13. The method of text detection as claimed in claim 12 wherein said step of:

calculating variations of said first energy includes the step of calculating a horizontal text energy for each DCT compressed images area.

14. The method of text detection as claimed in claim 13 wherein said step:

of screening screens DCT compressed images areas below a predetermined horizontal text energy.

15. The method of text detection as claimed in claim 12 wherein said potential text areas contain noise and are disconnected, and including a morphological mechanism for removing noisy areas and connecting disconnected text areas.

16. The method of text detection as claimed in claim 12 wherein said step of:

screening includes screening for variations of said second energy as a vertical energy and outputting detected text areas having the vertical energy over a predetermined value.

17. The method of text detection as claimed in claim 12 wherein said step of:

decompressing said DCT compressed images detected text areas includes the step of calculating vertical edge components in said detected text areas.

18. The method of text detection as claimed in claim 17 including the step of:

screening probable text areas where the calculated vertical edge components provide a density of horizontally aligned edges greater than a predetermined threshold value.

19. The method of text detection as claimed in claim 12 including the steps of:

temporarily adjusting consecutive frame images containing said detected text areas;

estimating a displacement of text in two consecutive frame images of said detected text areas; and

eliminating said two consecutive frames where there are no corresponding detected text areas.

20. The method of text detection as claimed in claim 19 including the step of:

screening for frame images having corresponding text areas before and after a predetermined frame image.