Capture of stylized TV table data via OCR

• US 8,763,038 B2
• Filed: 01/26/2009
• Issued: 06/24/2014
• Est. Priority Date: 01/26/2009
• Status: Active Grant

• Current Assignee: Saturn Licensing LLC (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Candelore, Brant L.
• Primary Examiner(s): FLYNN, RANDY A
• Application Number:
  US12/321,856
• Publication Number:
  US 20100192178A1
• Time in Patent Office: 1,975 Days
• Field of Search: 725/39
• US Class Current: 725/39
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/329 on a disc [VTOC]
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/44 Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 5/4448 for frame-grabbing

Abstract

In certain implementations consistent with the present invention, a method of detecting text in a television video display table involves saving a frame of video to a memory device; determining that the frame of video contains a table having cells containing text; storing a working copy of the frame of video to a memory; isolating text in the table by: removing any table boundaries from the image; removing any cell boundaries from the image; determining if the image has three dimensional or shadow attributes and removing any three dimensional or shadow attributes identified; thereby producing text isolated against a contrasting color background; and processing the isolated text using an optical character recognition (OCR) engine to extract the text as data. This abstract is not to be considered limiting, since other embodiments may deviate from the features described in this abstract.

Claims

1. A method of detecting text in a television video display table, comprising:

saving a frame of video representing an image to a memory device;

determining that the frame of video contains a table having cells containing text;

storing a working copy of the frame of video to a memory;

isolating text in the table by;

removing any table boundaries from the image;

removing any cell boundaries from the image;

determining if the image has three dimensional or shadow attributes in the table boundaries or cell boundaries and removing any three dimensional or shadow attributes identified, wherein determining if the image has three dimensional or shadow attributes in the table boundaries or cell boundaries is carried out by finding line patterns adjacent and outside table or cell boundaries that track a table or cell boundary;

where determining if a cell has three dimensional or shadow attributes is carried out by subtracting a variable rectangular band of pixels from other cell values to see what size band maximizes cancellation in order to distinguish the cell from the text area;

thereby producing text isolated against a contrasting color background; and

processing the isolated text using an optical character recognition (OCR) engine to extract the text as data.

2. The method according to claim 1, further comprising converting the text isolated against a contrasting color background to black text on a white background.

3. The method according to claim 1, wherein determining that the frame of video contains a table having cells containing text is carried out by tracking remote control commands transmitted from a remote control to identify commands that result in display of a table having cells containing text.

4. The method according to claim 3, wherein the commands comprise commands that cause display of a program guide, list of content, PVR list, dialogue box, closed captioning, or setup table.

5. The method according to claim 1, wherein determining that the frame of video contains a table having cells containing text is carried out by detecting rectangular shapes of size adequate to contain legible text.

6. The method according to claim 1, wherein determining that the frame of video contains a table having cells containing text is carried out by matching a display to a template.

7. The method according to claim 1, wherein removing the boundaries of the cells and the tables is carried out by matching a display to a template in order to locate the boundaries.

8. The method according to claim 1, further comprising storing the extracted text as data to a metadata database.

9. The method according to claim 1, where the frame of video containing the table has cells containing text of a foreground color against a background color.

10. A method of detecting text in a television video display table, comprising:

saving a frame of video representing an image to a memory device;

determining that the frame of video contains a table having cells containing text by detecting rectangular shapes of size adequate to contain legible text;

storing a working copy of the frame of video to a memory;

isolating text in the table by;

removing any table boundaries from the image;

removing any cell boundaries from the image by matching a display to a template in order to locate the boundaries;

determining if the image has three dimensional or shadow attributes in the table boundaries or cell boundaries and removing any three dimensional or shadow attributes identified, wherein determining if the image has three dimensional or shadow attributes in the table boundaries or cell boundaries is carried out by finding line patterns adjacent and outside table or cell boundaries that track a table or cell boundary;

where determining if a cell has three dimensional or shadow attributes is carried out by subtracting a variable rectangular band of pixels from other cell values to see what size band maximizes cancellation in order to distinguish the cell from the text area;

thereby producing text isolated against a contrasting color background; and

processing the isolated text using an optical character recognition (OCR) engine to extract the text as data to a metadata database.

11. The method according to claim 10, further comprising converting the text isolated against a contrasting color background to black text on a white background.

12. The method according to claim 10, wherein determining that the frame of video contains a table having cells containing text is carried out by tracking remote control commands issued to identify commands that result in display of a table having cells containing text.

13. The method according to claim 12, wherein the commands comprise commands that cause display of a program guide, list of content, PVR list, dialogue box, closed captioning, or setup table.

14. The method according to claim 10, wherein determining that the frame of video contains a table having cells containing text further comprises matching a display to a template.

15. The method according to claim 10, where the frame of video containing the table has cells containing text of a foreground color against a background color.

16. A method of detecting text in a television video display table, comprising:

saving a frame of video representing an image to a memory device;

determining that the frame of video contains a table having cells containing text by detecting rectangular shapes of size adequate to contain legible text and then by matching a display to a template;

storing a working copy of the frame of video to a memory;

isolating text in the table by;

removing any table boundaries from the image;

removing any cell boundaries from the image by matching a display to a template in order to locate the boundaries;

determining if the image has three dimensional or shadow attributes in the table boundaries or cell boundaries and removing any three dimensional or shadow attributes identified, wherein determining if the image has three dimensional or shadow attributes in the table boundaries or cell boundaries is carried out by finding line patterns adjacent and outside table or cell boundaries that track a table or cell boundary;

where determining if a cell has three dimensional or shadow attributes is further carried out by subtracting a variable rectangular band of pixels from other cell values to see what size band maximizes cancellation in order to distinguish the cell from the text area;

thereby producing text isolated against a contrasting color background;

converting the text isolated against a contrasting color background to black text on a white background; and

processing the isolated text using an optical character recognition (OCR) engine to extract the text as data to a metadata database.

17. The method according to claim 16, wherein determining that the frame of video contains a table having cells containing text is carried out by tracking remote control commands issued to identify commands that result in display of a table having cells containing text, wherein the commands comprise commands that cause display of a program guide, list of content, PVR list, dialogue box, closed captioning, or setup table.