Picture feature extraction device, picture retrieving device, and methods thereof for picture feature extraction and retrieving picture

• US 20030123735A1
• Filed: 02/24/2003
• Published: 07/03/2003
• Est. Priority Date: 01/06/1999
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kasutani, Eiji
• Granted Patent: US 6,868,182 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients

Abstract

In a picture feature extraction device, picture data having been determined as a moving picture by a picture data identifying section is subjected to being taken out the data corresponding to the entire frames or a part of the frames by a feature data extracting frame selecting means, upon which a resized picture generating means generates a resized picture from the data of each frame, and a frequency analyzing means analyzes a frequency of the resized picture having been generated. Then a DC component/partial AC component extracting means takes out DC components and a part of AC components, after which a feature extraction means generates feature data of the whole frames having been selected. The picture data identified as still picture data is subjected to having a resized picture generated by the resized picture generating means from the data of each frame, after which the frequency analyzing means executes a frequency analysis on the resized picture having being generated. Then as the DC component/partial AC component extracting means takes out DC components and a part of AC components obtained as a result of the analysis in order to generate a feature data.

Claims

1. A picture feature extraction device comprising:

a feature data extracting frame selecting means taking out some data corresponding to the entire frames or a portion of the frames from the moving picture data, selecting the data subjected to processing;

a resized picture generating means generating a resized picture from data corresponding to each frame having been taken out;

a frequency analyzing means executing a frequency analysis on said resized picture having been generated by said resized picture generating means;

a DC/partial AC component extracting means taking out DC components and a part of AC components obtained as a result of the analysis by said frequency analyzing means as a frame feature data; and

a moving picture feature data producing means producing a moving picture feature data by collecting said frame feature data outputted from said DC/partial AC component extracting means.

2. A picture feature extraction device as claimed in claim 1, comprising:

a moving picture data base including a moving picture data storing means storing said moving picture data, and a moving picture feature data storing means storing said moving picture feature data generated for each moving picture data stored in said moving picture data storing means.

3. A picture feature extraction device as claimed in claim 2, wherein p1 said resized picture generating means divides a still picture into blocks, and takes out color data representative of each block from each block.

4. A picture feature extraction device as claimed in claim 2, wherein

in case when said picture data is compressed by blocks, said resized picture generating means takes out DC components from each block.

5. A picture retrieving device comprising:

a moving picture feature extraction means including a feature data extracting frame selecting means taking out some data corresponding to the entire frames or a portion of the frames from the moving picture data, selecting the data subjected to processing, a resized picture generating means generating a resized picture from data corresponding to each frame having been taken out, a frequency analyzing means executing a frequency analysis on said resized picture having been generated by said resized picture generating means, a DC/partial AC component extracting means taking out DC components and a part of AC components obtained as a result of the analysis by said frequency analyzing means as a frame feature data, and a moving picture feature data producing means producing a moving picture feature data by collecting said frame feature data outputted from said DC/partial AC component extracting means;

a moving picture data base including a moving picture feature data storing means storing the moving picture feature data previously being generated by said moving picture feature extraction means on the basis of the moving picture data, and a moving picture data storing means storing said moving picture data; and

a moving picture feature data correlation measuring means calculating a correlation value between the moving picture feature data generated by said moving picture feature extraction means on the basis of the inputted moving picture data and the moving picture feature data stored in said moving picture data base, the moving picture feature data correlation measuring means retrieving the moving picture segment within the moving picture data base having a high correlation value as a candidate of similar moving picture segment.

6. A picture retrieving device as claimed in claim 5, wherein

said moving picture feature data correlation measuring means divides the moving picture feature data having been generated by said moving picture feature extraction means on the basis of said inputted moving picture data, and calculates in turn a partial correlation value between a part of the moving picture feature data of the moving picture included in said moving picture data base and a moving picture feature data of each division of the divided moving picture feature data, upon which eliminating the moving picture segment included in said moving picture data base from the candidates of similar moving pictures when it has a feature data with a partial correlation value lower than a predetermined threshold value.

7. A picture retrieving device as claimed in claim 6, wherein

said resized picture generating means divides a picture into blocks, and takes out from each block color information representative of each block.

8. A picture retrieving device as claimed in claim 6, wherein

said resized picture generating means takes out DC components from said each block when said picture data is compressed by blocks.

9. A picture retrieving device as claimed in claim 5, wherein

said moving picture feature data correlation measuring means divides the moving picture feature data having been generated by said moving picture feature extraction means in respect with said inputted moving picture data, and calculates in turn a partial correlation value between a part of the moving picture feature data of the moving picture included in said moving picture data base and a partial moving picture feature data of each division of the divided moving picture feature data, upon which eliminating the moving picture segment included in said moving picture data base from the candidates of similar moving pictures at the time when it is acknowledged that a number of partial moving picture feature data having a feature data with a partial correlation value lower than a predetermined threshold value surpasses a predetermined threshold value.

10. A picture retrieving device as claimed in claim 9, wherein

said resized picture generating means divides a picture into blocks, and takes out from each block color information representative of each block.

11. A picture retrieving device as claimed in claim 9, wherein

said resized picture generating means takes out DC components from said each block when said picture data is compressed by blocks.

12. A picture retrieving device as claimed in claim 5, wherein

said resized picture generating means divides a picture into blocks, and takes out from each block color information representative of each block.

13. A picture retrieving device as claimed in claim 5, wherein

said resized picture generating means takes out DC components from said each block when said picture data is compressed by blocks.

14. A picture feature extraction method comprising:

a feature data extracting frame selecting step for taking out some data corresponding to the entire frames or a portion of the frames from the moving picture data, selecting the data subjected to processing;

a resized picture generating step for generating a resized picture from data corresponding to each frame having been taken out;

a frequency analyzing step for executing a frequency analysis on said resized picture having been generated at said resized picture generating step;

a DC/partial AC component extracting step for taking out DC components and a part of AC components obtained as a result of the analysis at said frequency analyzing step as a frame feature data; and

a moving picture feature data producing step for producing a moving picture feature data by collecting said frame feature data outputted from said DC/partial AC component extracting step.

15. A picture feature extraction method as claimed in claim 14, comprising:

a moving picture data storing step for storing said moving picture data; and

a moving picture feature data storing step for storing said moving picture feature data generated for each moving picture data stored at said moving picture data storing step.

16. A picture feature extraction method as claimed in claim 15, wherein

said resized picture generating step divides a still picture into blocks, and takes out from each block color data representative of each block.

17. A picture feature extraction method as claimed in claim 15, wherein

in case when said picture data is compressed by blocks, said resized picture generating step takes out from each block DC components.

18. A picture feature extraction method comprising:

a picture data identifying step for identifying whether the inputted picture data is of still pictures or of moving pictures;

a still picture feature extraction step including a resized picture generating step for generating a resized picture from the picture data identified as still picture data at said picture data identifying step, a frequency analyzing step for executing a frequency analysis on said resized picture having been generated at said resized picture generating step, and a DC/partial AC component extracting step for taking out DC components and a part of AC components obtained as a result of the analysis at said frequency analyzing step as a still picture feature data; and

a moving picture feature extraction step including a feature data extracting frame selecting step for taking out some data corresponding to the entire frames or a portion of the frames from the picture data identified as moving picture data at said picture data identifying step, selecting the data subjected to processing, a resized picture generating step for generating a resized picture from data corresponding to each frame having been taken out, a frequency analyzing step for executing a frequency analysis on said resized picture having been generated at said resized picture generating step, a DC/partial AC component extracting step for taking out DC components and a part of AC components obtained as a result of the analysis at said frequency analyzing step as a frame feature data, and a moving picture feature data producing step for generating a moving picture feature data by collecting said frame feature data outputted at said DC/partial AC component extracting step.

19. A picture feature extraction method as claimed in claim 18, comprising:

a picture data storing step for storing said picture data; and

a picture feature data storing step using each picture data stored at said picture data storing step for storing the still picture feature data generated at said still picture feature extraction step and the moving picture feature data generated at said moving picture feature extraction step as a picture feature data.

20. A picture feature extraction method as claimed in claim 18, wherein

said resized picture generating step divides a still picture into blocks, and takes out from each block color data representative of each block.

21. A picture feature extraction method as claimed in claim 18, wherein

in case when said picture data is compressed by blocks, said resized picture generating step takes out from each block DC components.

22. A picture retrieving method comprising:

a moving picture feature extraction step including a feature data extracting frame selecting step for taking out some data corresponding to the entire frames or a portion of the frames from the moving picture data, selecting the data subjected to processing, a resized picture generating step for generating a resized picture from data corresponding to each frame having been taken out, a frequency analyzing step for executing a frequency analysis on said resized picture having been generated by said resized picture generating step, a DC/partial AC component extracting step for taking out DC components and a part of AC components obtained as a result of the analysis at said frequency analyzing step as a frame feature data, and a moving picture feature data producing step for producing a moving picture feature data by collecting said frame feature data outputted at said DC/partial AC component extracting step;

a moving picture feature data correlation measuring step for calculating a correlation value between the moving picture feature data of the moving picture within a moving picture data base where the moving picture feature data of the moving picture data having been previously generated at said moving picture feature extraction step is stored together with the moving picture data and the moving picture feature data generated at said moving picture feature extraction step on the basis of the inputted moving picture data; and

a similar moving picture retrieving step for retrieving the moving picture segment within said moving picture data base having a high correlation value with respect to the moving picture feature data generated on the basis of the inputted moving picture data as a candidate of similar moving picture.

23. A picture retrieving method as claimed in claim 22, wherein

said moving picture feature data correlation measuring step has a step of dividing the moving picture feature data having been generated by said moving picture feature extraction step on the basis of said inputted moving picture data, and a step of calculating in turn a partial correlation value between a part of the moving picture feature data of the moving picture included in said moving picture data base and a moving picture feature data of each division of the divided moving picture feature data, and

said similar picture retrieving step eliminates the moving picture segment included in said moving picture data base from the candidates of similar moving pictures when it has a feature data with a partial correlation value lower than a predetermined threshold value.

24. A picture retrieving method as claimed in claim 23, wherein

said second picture feature data correlation measuring step has a step of dividing the moving picture feature data having been generated by said moving picture feature extraction step on the basis of said inputted moving picture data, and a step of calculating in turn a partial correlation value between a part of the moving picture feature data of the moving picture included in said moving picture data base and a moving picture feature data of each division of the divided moving picture feature data, and

said similar picture retrieving step eliminates the moving picture segment included in said moving picture data base from the candidates of similar moving pictures when it has a feature data with a partial correlation value lower than a predetermined threshold value.

25. A picture retrieving method as claimed in claim 24, wherein

said resized picture generating step divides a picture into blocks, and takes out from each block color information representative of each block.

26. A picture retrieving method as claimed in claim 24, wherein

said resized picture generating step takes out DC components from said each block when said picture data is compressed by blocks.

27. A picture retrieving method as claimed in claim 23, wherein

said second picture feature data correlation measuring step has a step of dividing the moving picture feature data having been generated by said moving picture feature extraction step on the basis of said inputted moving picture data, and a step of calculating in turn a partial correlation value between a part of the moving picture feature data of the moving picture included in said moving picture data base and a partial moving picture feature data of each division of the divided moving picture feature data, and

said similar picture retrieving step eliminates the moving picture segment included in said moving picture data base from the candidates of similar moving pictures at the time when it is acknowledged that a number of partial moving picture feature data having a feature data with a partial correlation value lower than a predetermined threshold value surpasses a predetermined threshold value.

28. A picture retrieving method as claimed in claim 27, wherein

said resized picture generating step divides a picture into blocks, and takes out from each block color information representative of each block.

29. A picture retrieving method as claimed in claim 27, wherein

said resized picture generating step takes out DC components from said each block when said picture data is compressed by blocks.

30. A picture retrieving method as claimed in claim 22, wherein

said moving picture feature data correlation measuring step has a step of dividing the moving picture feature data having been generated by said moving picture feature extraction step on the basis of said inputted moving picture data, and a step of calculating in turn a partial correlation value between a part of the moving picture feature data of the moving picture included in said moving picture data base and a partial moving picture feature data of each division of the divided moving picture feature data, and

said similar picture retrieving step eliminates the moving picture segment included in said moving picture data base from the candidates of similar moving pictures at the time when it is acknowledged that a number of partial moving picture feature data having a feature data with a partial correlation value lower than a predetermined threshold value surpasses a predetermined threshold value.

31. A picture retrieving method as claimed in claim 30, wherein

said resized picture generating step divides a picture into blocks, and takes out from each block color information representative of each block.

32. A picture retrieving method as claimed in claim 30, wherein

said resized picture generating step takes out DC components from said each block when said picture data is compressed by blocks.

33. A picture retrieving method comprising:

a first picture feature data correlation measuring step for calculating a correlation value between a still picture feature data of a still picture in inputted still picture data generated by a still picture feature extraction step including a resized picture generating step for generating a resized picture from the inputted picture data identified as still picture data, a frequency analyzing step for executing a frequency analysis on said resized picture having been generated at said resized picture generating step, and a DC/partial AC component extracting step for taking out DC components and a part of AC components obtained as a result of the analysis at said frequency analyzing step as a still picture feature data and a picture feature data of a picture within a picture data base where a still picture feature data having been generated at said still picture feature extraction step on the basis of a still picture data is stored together with said still picture data;

a similar still picture retrieving step for retrieving the picture within said picture data base having a high correlation value with respect to the picture feature data generated on the basis of the inputted picture data as a candidate of similar still picture;

a second picture feature data correlation measuring step for calculating a correlation value between a moving picture feature data of a moving picture in inputted moving picture data generated by a moving picture feature extraction step including a feature data extracting frame selecting step for taking out some data corresponding to the entire frames or a portion of the frames from the picture data identified as moving picture data, selecting the data subjected to processing, a resized picture generating step for generating a resized picture from data corresponding to each frame having been taken out, a frequency analyzing step for executing a frequency analysis on said resized picture having been generated by said resized picture generating step, a DC/partial AC component extracting step for taking out DC components and a part of AC components obtained as a result of the analysis at said frequency analyzing step as a frame feature data, and a moving picture feature data producing step for generating a moving picture feature data by collecting said frame feature data outputted at said DC/partial AC component extracting step, and a picture feature data of a picture within a picture data base where a moving picture feature data having been generated at said moving picture feature extraction step on the basis of a moving picture data is stored together with said moving picture data; and

a similar moving picture retrieving step for retrieving a moving picture segment within said picture data base having a high correlation value with respect to the moving picture feature data generated on the basis of the inputted moving picture data as a candidate of similar moving picture.

34. A picture retrieving method as claimed in claim 33, wherein

said resized picture generating step divides a picture into blocks, and takes out from each block color information representative of each block.

35. A picture retrieving method as claimed in claim 33, wherein

said resized picture generating step takes out DC components from said each block when said picture data is compressed by blocks.