Image retrieving method and apparatuses therefor

• US 6,400,890 B1
• Filed: 05/11/1998
• Issued: 06/04/2002
• Est. Priority Date: 05/16/1997
• Status: Expired due to Term

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Miyatake, Takafumi, Nagasaka, Akio
• Primary Examiner(s): Tran, Thai
• Application Number:
  US09/075,521
• Time in Patent Office: 1,485 Days
• Field of Search: H04/N.576, H04/N.592, H04/N.979, H04/N.5781, H04/N.514, 386/1, 386/45, 386/69-70, 386/125-126, 360/72.1, 360/72.2, 360/72.3, 369/32, 345/719, 345/721, 345/722, 345/723, 707/1, 707/2, 707/3, 707/4, 707/5, 707/101-102, 707/103.R, 707/103.Y, 707/103, 707/103.Z, 707/104.1, 382/305-306, 348/474, 348/700, 348/701, 348/702, 348/552
• US Class Current: 386/241
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7328 Query by example, e.g. a complete video frame or video sequence graphical querying G06F16/7335
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/7854 using shape G06F16/7837 takes precedence
• CPC Class Codes: G06F 16/7857 using texture G06F16/7837 takes precedence
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation

Abstract

By sequentially inputting images for each frame, sequentially extracting features from the inputted frame images, converting the features sequentially extracted into a feature series corresponding to the inputted frame image series, compressing the feature series in the direction of the time axis, storing the compressed feature series in the storage, sequentially extracting features separately from the images to be retrieved for each inputted frame, sequentially comparing the features of the images to be retrieved for each frame with the stored compressed feature series, storing the progress state of this comparison, updating the stored progress state of the comparison on the basis of a comparison result with the frame features of the succeeding images to be retrieved, and retrieving image scenes matching with the updated progress state from the images to be retrieved on the basis of the comparison result between the updated progress state and the features of the images to be retrieved for each frame, the present invention can retrieve video images on the air or video images in the data base at high speed and enables self organization of video to be classified and arranged on the basis of the identity of partial images of video.

Claims

1. A method for describing a video comprising the steps of:

extracting a feature from the video in each of a plurality of predetermined periods;

comparing extracted features of each of the periods to obtain a representative feature representing successive ones of the periods in which the extracted features are within an allowable range,

recording the representative feature with length information indicating the periods that are represented by the representative feature.

2. A method for describing a video according to claim 1, wherein the step of extracting includes extracting color as the extracted feature.

3. A method for describing a video according to claim 1, wherein the step of extracting includes extracting a shape of a boundary line as the extracted feature.

4. A method for describing a video according to claim 1, wherein the step of extracting includes extracting texture as the extracted feature.

5. A method for describing a video according to claim 1, wherein the step of extracting includes extracting the feature from at least one frame of the video in each of the predetermined periods.

6. A method for describing a video according to claim 1, wherein the step of extracting includes extracting the feature from each frame of the video as the predetermined period, and

the step of recording the representative feature with length information includes recording the length information as a frame number.

7. A method for describing a video comprising the steps of:

extracting features from the video within specified intervals;

setting a first feature as a representative feature and an interval of the first feature as a first segment,

comparing a current one of the extracted features with the representative feature, and

when a difference is within a threshold, adding the interval of the current extracted feature to the first segment,

when the difference is lager than the threshold, setting the current extracted feature as another representative feature, and the interval of the current extracted feature as another segment,

recording each representative feature with length information indicating the segment represented by the representative feature.

8. A method for describing a video according to claim 7, wherein the step of extracting includes extracting color as the extracted feature.

9. A method for describing a video according to claim 7, wherein the step of extracting includes extracting the feature from at least one frame of the video in each of the predetermined periods.

10. A method for describing a video according to claim 7, wherein the step of extracting includes extracting the feature from each frame of the video as the predetermined period, and

the step of recording the representative feature with length information includes recording the length information as a frame number.

11. A program stored on a computer readable storage medium executing a video describing method on a computer, comprising the instructions of:

extracting a feature from the video in each of a plurality of predetermined periods;

comparing extracted features of each of the periods to obtain a representative feature representing successive ones of the periods in which the extracted features are within an allowable range,

recording the representative feature with length information indicating the periods that are represented by the representative feature.

12. A program stored on a computer readable storage medium according to claim 11, wherein the step of extracting includes extracting color as the extracted feature.

13. A program stored on a computer readable storage medium according to claim 11, wherein the step of extracting includes extracting a shape of a boundary line as the extracted feature.

14. A program stored on a computer readable storage medium according to claim 11, wherein the step of extracting includes extracting texture as the extracted feature.

15. A program stored on a computer readable storage medium according to claim 11, wherein the step of extracting includes extracting the feature from at least one frame of the video in each of the predetermined periods.

16. A program stored on a computer readable storage medium according to claim 11, wherein the step of extracting includes extracting the feature from each frame of the video as the predetermined period, and

the step of recording the representative feature with length information includes recording the length information as a frame number.

17. A program stored on a computer readable storage medium executing a video describing method on a computer, comprising instructions of:

extracting features from the video within specified intervals;

setting a first feature as a representative feature and an interval of the first feature as a first segment,

comparing a current one of the extracted features with the representative feature, and

when a difference is within a threshold, adding the interval of the current extracted feature to the first segment,

when the difference is larger than the threshold, setting the current extract ed feature as another representative feature, and the interval of the current extracted feature as another segment,

recording each representative feature with length information indicating the segment represented by the representative feature.

18. A program stored on a computer readable storage medium according to claim 17, wherein the step of extracting includes extracting color as the extracted feature.

19. A program stored on a computer readable storage medium according to claim 17, wherein the step of extracting includes extracting the feature from at least one frame of the video in each of the predetermined periods.

20. A program stored on a computer readable storage medium according to claim 17, wherein the step of extracting includes extracting the feature from each frame of the video as the predetermined period, and

the step of recording the representative feature with length information includes recording the length information as a frame number.

21. A computer readable storage medium on which is stored executable instructions representing a video describing program, wherein execution of the instructions causes a computer to perform the steps of:

extracting a feature from the video in each of a plurality of predetermined periods;

comparing extracted features of each of the periods to obtain a representative feature representing successive ones of the periods in which the extracted features are within an allowable range,

recording the representative feature with length information indicating the periods that are represented by the representative feature.

22. A computer readable storage medium according to claim 21, wherein the step of extracting includes extracting color as the extracted feature.

23. A computer readable storage medium according to claim 21, wherein the step of extracting includes extracting a shape of a boundary line as the extracted feature.

24. A computer readable storage medium according to claim 21, wherein t he step of extracting includes extracting texture as the extracted feature.

25. A computer readable storage medium according to claim 21, wherein the step of extracting includes extracting the feature from at least one frame of the video in each of the predetermined periods.

26. A computer readable storage medium according to claim 21, wherein the step of extracting includes extracting the feature from each frame of the video as the predetermined period, and

the step of recording the representative feature with length information includes recording the length information as a frame number.

27. A computer readable storage medium on which is stored executable instructions representing a video describing program, wherein execution of the instructions causes a computer to perform the steps of:

extracting features from the video with in specified intervals;

setting a first feature as a representative feature and an interval of the first feature as a first segment,

comparing a current one of the extracted features with the representative feature, and

when a difference is within a threshold, adding the interval of the current extracted feature to the first segment,

when the difference is lager than the threshold, setting the current extracted feature as another representative feature, and the interval of the current extracted feature as another segment,

recording each representative feature with length information indicating the segment represented by the representative feature.

28. A computer readable storage medium according to claim 27, wherein the step of extracting includes extracting color as the extracted feature.

29. A computer readable storage medium according to claim 27, wherein the step of extracting includes extracting the feature from at least one frame of the video in each of the predetermined periods.

30. A computer readable storage medium according to claim 27, wherein the step of extracting includes extracting the feature from each frame of the video as the predetermined period, and

the step of recording the representative feature with length information includes recording the length information as a frame number.