Method and system for analyzing video content using detected text in video frames

• US 6,608,930 B1
• Filed: 08/09/1999
• Issued: 08/19/2003
• Est. Priority Date: 08/09/1999
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Elenbaas, Jan H., Dimitrova, Nevenka, Agnihotri, Lalitha
• Primary Examiner(s): Do, Anh Hong
• Application Number:
  US09/370,931
• Time in Patent Office: 1,471 Days
• Field of Search: 382/176, 382/190, 382/173, 382/177, 382/178, 382/179, 382/198, 382/209, 348/465, 348/722, 709/217, 704/8
• US Class Current: 382/176
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/7854 using shape G06F16/7837 takes precedence
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program

Abstract

There is disclosed, for use in video text analysis system, a video processing device for searching video streams for one or more user-selected image text attributes. The video processing device comprises an image processor capable detecting and extracting image text from video frames, determining attributes of the extracted image text, comparing the extracted image text attributes and the user-selected image text attributes, and, if a match occurs, modifying, transferring, and/or labeling at least a portion of the video stream in accordance with user commands. The invention uses the user-selected image text attributes to search through an archive of video clips to 1) locate particular types of events, such as news programs or sports events; 2) locate programs featuring particular persons or groups; 3) locate programs by name; 4) save or remove all or some commercials, and to otherwise sort, edit, and save all of, or portions of, video clips according to image text that appears in the frames of the video clips.

Claims

1. For use in a system capable of analyzing image text in video frames, a video processing device capable of one of filtering and searching video streams in response to receipt of at least one selected image text attribute, wherein said selected image text attribute is a non-character attribute, said video processing device comprising:

an image processor capable of receiving a first video stream comprising a plurality of video frames, detecting and extracting image text from said plurality of video frames, determining at least one non-character attribute of said extracted image text, comparing said at least one extracted image text non-character attribute and said at least one selected image text attribute, and, in response to a match between said at least one extracted image text non-character attribute and said at least one selected image text attribute, at least one of;

modifying at least a portion of said first video stream;

transferring at least a portion of said first video stream; and

labeling at least a portion of said first video stream.

2. The video processing device set forth in claim 1 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is one of scrolling horizontally;

scrolling vertically;

fading;

zooming;

rippling;

flying; and

flashing.

3. The video processing device set forth in claim 1 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is one of a name of a person;

a name of a group;

a name of a location; and

a name of an event.

4. The video processing device set forth in claim 1 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is part of a commercial advertisement.

5. The video processing device set forth in claim 1 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is one of:

in-scene text; and

text appearing at one of a start of a program and an end of a program.

6. The video processing device set forth in claim 1 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is part of a program name.

7. The video processing device set forth in claim 1 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is part of a news program.

8. The video processing device set forth in claim 1 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is part of a sports program.

9. An image text analysis system comprising:

a video processing device capable of one of searching and filtering video streams in response to receipt of at least one selected image text attribute, wherein said selected image text attribute is a non-character attribute, said video processing device comprising;

an image processor capable of receiving a first video stream comprising a plurality of video frames, detecting and extracting image text from said plurality of video frames, determining at least one non-character attribute of said extracted image text, comparing said at least one extracted image text non-character attribute and said at least one selected image text attribute, and, in response to a match between said at least one extracted image text non-character attribute.and said at least one selected image text attribute, at least one of;

modifying at least a portion of said first video stream;

transferring at least a portion of said first video stream; and

labeling at least a portion of said first video stream;

a display monitor for displaying said at least a portion of said first video stream; and

a user input device for receiving user commands.

10. The video processing device set forth in claim 9 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is one of scrolling horizontally;

scrolling vertically;

fading;

zooming;

rippling;

flying; and

flashing.

11. The video processing device set forth in claim 9 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is one of a name of a person;

a name of a group;

a name of a location; and

a name of an event.

12. The video processing device set forth in claim 9 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is part of a commercial advertisement.

13. The video processing device set forth in claim 9 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is one of:

in-scene text; and

text appearing at one of a start of a program and an end of a program.

14. The video processing device set forth in claim 9 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is part of a program name.

15. The video processing device set forth in claim 9 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is part of a news program.

16. The video processing device set forth in claim 9 wherein said at least one extracted image text non-character attribute indicates that said image text in said plurality of video frames is part of a sports program.

17. For use in a system capable of analyzing image text in video frames, a method of one of searching and filtering video streams in response to receipt of at least one selected image text attribute, wherein the selected image text attribute is a non-character attribute, the method comprising the steps of:

receiving a first video stream comprising a plurality of video frames;

detecting and extracting image text from the plurality of video frames;

determining at least one non-character attribute of the extracted image text;

comparing the at least one extracted image text non-character attribute and the at least one selected image text attribute; and

in response to a match between the at least one extracted image text non-character attribute and the at least one selected image text attribute, at least one of;

modifying at least a portion of the first video stream;

transferring at least a portion of the first video stream; and

labeling at least a portion of the first video stream.

18. The method set forth in claim 17 wherein the at least one extracted image text non-character attribute indicates that the image text in the plurality of video frames is one of scrolling horizontally;

scrolling vertically;

fading;

zooming;

rippling;

flying; and

flashing.

19. The method set forth in claim 17 wherein the at least one extracted image text non-character attribute indicates that the image text in the plurality of video frames is one of a name of a person;

a name of a group;

a name of a location; and

a name of an event.

20. The method set forth in claim 17 wherein the at least one extracted image text non-character attribute indicates that the image text in the plurality of video frames is part of a commercial advertisement.

21. For use in a system capable of analyzing image text in video frames, computer executable process steps stored on a computer readable storage medium for performing one of searching and filtering video streams in response to receipt of at least one selected image text attribute, wherein the selected image text attribute is a non-character attribute, the computer executable process steps comprising the steps of:

receiving a first video stream comprising a plurality of video frames;

detecting and extracting image text from the plurality of video frames;

determining at least one non-character attribute of the extracted image text;

comparing the at least one extracted image text non-character attribute and the at least one selected image text attribute; and

in response to a match between the at least one extracted image text non-character attribute and the at least one selected image text attribute, at least one of;

modifying at least a portion of the first video stream;

transferring at least a portion of the first video stream; and

labeling at least a portion of the first video stream.

22. The computer readable storage medium set forth in claim 21 wherein the at least one extracted image text non-character attribute indicates that the image text in the plurality of video frames is one of scrolling horizontally;

scrolling vertically;

fading;

zooming;

rippling;

flying; and

flashing.

23. The computer readable storage medium set forth in claim 21 wherein the at least one extracted image text non-character attribute indicates that the image text in the plurality of video frames is one of a name of a person;

a name of a group;

a name of a location; and

a name of an event.

24. The computer readable storage medium set forth in claim 21 wherein the at least one extracted image text non-character attribute indicates that the image text in the plurality of video frames is part of a commercial advertisement.