Image retrieval processing to obtain static image data from video data

• US 7,848,598 B2
• Filed: 09/15/2003
• Issued: 12/07/2010
• Est. Priority Date: 09/19/2002
• Status: Expired due to Fees

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Taguchi, Shinya, Ogawa, Masakazu, Nagatani, Shunsuke, Egawa, Yutaka, Yamazoe, Nobuyuki, Suzuki, Michitoshi, Kanno, Eisuke
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): MOTSINGER, SEAN T
• Application Number:
  US10/661,590
• Publication Number:
  US 20040056881A1
• Time in Patent Office: 2,640 Days
• Field of Search: 715723-726, 715243-253, 715753-758, 715730-732, 382/305, 382/306
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5846 using extracted text

Abstract

Desired data can be easily retrieved from material image data associated with video data. For material image data associated with video data, character strings contained in relevant material images are extracted, and keywords input to keyword input parts of a user interface are matched with the extracted characters to retrieve data of the relevant material images. The retrieved material images are changed in size according to their importance before being displayed on a screen.

Claims

1. An image retrieval apparatus, comprising:

a storage unit that stores static image data associated with time positions in a video data, the static image data being displayed with the video during time positions with which the static image data are associated;

an input unit that accepts an input keyword from a user;

an extraction unit that extracts a character string contained in the static image data by at least one of (1) extracting text data from the static image data which has the text data, and (2) performing character recognition processing on the static image data and extracting text data which is a result of the processing;

a retrieval unit that matches the extracted character string with the input keyword to retrieve relevant static image data, the retrieval unit retrieving meta-data and voice index data that include the input keyword;

a retrieval result display unit that displays the retrieved relevant static image data, varying size of the displayed relevant static image data based on (1) a time length of a corresponding video scene in which the static image data is associated, and (2) an amount of change in scene contents of a corresponding video scene in which the static image data is associated; and

a video display unit that, according to the user'"'"'s operation for selecting at least one of the displayed relevant static image data, reproduces and displays associated video data during time positions with which the selected static image data are associated.

4. The image retrieval apparatus of claim 1, wherein the video display unit displays a time scale including a slidable plug, and the user'"'"'s operation for selecting at least one of the displayed relevant static image data comprises moving the slidable plug to select a slide image that is associated with the time position of the video data to be replayed.

7. The image retrieval apparatus according to claim 1, wherein the input keyword is converted into voice waveform data, the voice waveform data being matched by the retrieval unit during the retrieval of the voice index data.

2. An image retrieval method comprising:

performing an extraction, with an extraction unit, that extracts a character string contained in the static image data by at least one of (1) extracting text data from the static image data which has the text data, and (2) performing character recognition processing on the static image data and extracting text data which is a result of the processing;

matching an input keyword input by the user with the extracted character string, with a retrieval unit, to retrieve relevant static image data and retrieving meta-data and voice index data that include the input keyword;

displaying the retrieved relevant static image data, varying the size of the displayed relevant static image data based on (1) a time length of a corresponding video scene in which the static image data is associated, and (2) an amount of change in scene contents of a corresponding video scene in which the static image data is associated; and

reproducing and displaying on a video display unit, according to the user'"'"'s operation for selecting at least one of the displayed relevant static image data, associated video data during time positions with which the selected static image data are associated.

5. The method according to claim 2, wherein the user'"'"'s operation for selecting at least one of the displayed relevant static image data comprises moving a slidable plug on a displayed time scale to select a slide image that is associated with the time position of the video data to be replayed.

8. The method according to claim 2, further comprising:

converting the input keyword input by the user into voice waveform data and matching the voice waveform data with the voice index data.

3. A storage non-transitory medium readable by a computer, the non-transitory storage medium storing a program of instructions executable by the computer to perform a function for retrieving image data, the function comprising:

accepting an input keyword from a user;

performing an extraction that extracts a character string contained in the static image data by at least one of (1) extracting text data from the static image data which has the text data, and (2) performing character recognition processing on the static image data and extracting text data which is a result of the processing;

matching the extracted character string with the input keyword to retrieve relevant static image data and retrieving meta-data and voice index data that include the input keyword;

displaying the retrieved relevant static image data, varying the size of the displayed relevant static image data based on (1) a time length of a corresponding video scene in which the static image data is associated, and (2) an amount of change in scene contents of a corresponding video scene in which the static image data is associated; and

reproducing and displaying, according to the user'"'"'s operation for selecting at least one of the displayed relevant static image data, associated video data during time positions with which the selected static image data are associated.

6. The non-transitory storage medium according to claim 3, wherein the user'"'"'s operation for selecting at least one of the displayed relevant static image data comprises moving a slidable plug on a displayed time scale to select a slide image that is associated with the time position of the video data to be replayed.

9. The function for retrieving image data according to claim 3, further comprising:

converting the input keyword input by the user into voice waveform data and matching the voice waveform data with the voice index data.