Method of indexing and searching images of text in video

• US 6,937,766 B1
• Filed: 04/13/2000
• Issued: 08/30/2005
• Est. Priority Date: 04/15/1999
• Status: Expired due to Term

• Current Assignee: Anxin Mate Holding Ltd. (Anxin-China Holdings Ltd.)
• Original Assignee: Mate-Media Access Technologies Limited
• Inventors: Menadeva, Ovadya, Greenspan, Hayit, Ladkani, Joseph, Wilf, Itzhak
• Primary Examiner(s): Au, Amelia M.
• Assistant Examiner(s): LE, BRIAN Q
• Application Number:
  US09/549,229
• Time in Patent Office: 1,965 Days
• Field of Search: 707/3, 707/104.1, 707/516, 707/529, 707/530, 707/526, 345/723, 345/724, 382/229, 348/468
• US Class Current: 382/229
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A method for generating an index of the text of a video image sequence is provided. The method includes the steps of determining the image text objects in each of a plurality of frames of the video image sequence; comparing the image text objects in each of the plurality of frames of the video image sequence to obtain a record of frame sequences having matching image text objects; extracting the content for each of the similar image text objects in text string format; and storing the text string for each image text object as a video text object in a retrievable medium.

Claims

1. A method of generating an index of the text of a video image sequence, the method comprising the steps of:

determining the image text objects in each of a plurality of frames of the video image sequence;

comparing the image text objects in each of said plurality of frames of the video image sequence to obtain a record of frame sequences having matching image text objects;

extracting the content for each of said similar image text objects in text string format;

storing the text string for each image text object as a video text object in a retrievable medium; and

categorizing the text string as at least one of a group including static text, scrolling text, overlayed text and in-scene text, wherein said step of categorizing comprises the steps of;

tracking the text string motion, or lack of it, over time;

tracking the image background motion or lack of it, over time; and

comparing the text string motion and the background motion.

2. A method of generating an index of the text of a video image sequence, the method comprising the steps of:

determining the image text objects in each of a plurality of frames of the video image sequence;

comparing the image text objects in each of said plurality of frames of the video image sequence to obtain a record of frame sequences having matching image text objects;

extracting the content for each of said similar image text objects in text string format;

storing the text string for each image text object as a video text object in a retrievable medium; and

storing the frame location of said text string for each image text object as a video text object in a retrievable medium.

3. A method of generating an index of the text of a video image sequence, the method comprising the steps of:

determining the image text objects in each of a plurality of frames of the video image sequence;

comparing the image text objects in each of said plurality of frames of the video image sequence to obtain a record of frame sequences having matching image text objects;

extracting the content for each of said similar image text objects in text string format; and

storing the text string for each image text object as a video text object in a retrievable medium, wherein said step of determining the image text objects comprises the steps of;

a) selecting a consecutive pair of frames consisting of a first current frame and its previous frame;

b) selecting an image text object in said current frame;

c) comparing the selected image text object with each image text object of said previous frame;

d) repeating steps b) and c) for each image text object in said current frame;

e) selecting a subsequent consecutive pair of frames consisting of a second current frame and the previous frame of said second current frame, wherein said previous frame of said second current frame is the first current frame; and

f) repeating steps b)-e) for each consecutive pair of frames of a video sequence.

4. The method of claim 3, wherein said step of determining further comprises the steps of:

quantizing the color space of each of said plurality of frames to a pre-determined number of individual colors;

forming a map of connected pixels for each of said pre-determined number of individual colors;

determining which of said connected pixels are potentially alpha-numeric characters;

grouping the potentially alpha-numeric characters as potential text string candidates, each of said potential text strings having a single text color;

forming a binary image of said potential text strings; and

computing horizontal and vertical projections of the binary image and conducting a profile analysis to whether the potential string has text shape characteristics

5. The method of claim 4, wherein said step of determining which of said connected pixels are potentially alpha-numeric characters comprises the steps of:

determining the dimensions, aspect ratio and fill factor for the color space.

6. The method of claim 4, wherein said step of grouping comprises the steps of:

grouping according to the location of said potentially alpha-numeric characters.

7. The method of claim 3, wherein said step of comparing the selected image text object comprises the step of:

comparing the text content of said text objects.

8. The method of claim 3, wherein said step of comparing the selected image text object comprises the step of:

comparing the location of said text objects.

9. The method of claim 3, wherein said step of comparing the selected image text object comprises the step of:

computing a score characteristic of the image similarity of the regions occupied by said text objects.