RETRIEVING SCENES FROM MOVING IMAGE DATA

• US 20090324014A1
• Filed: 05/29/2009
• Published: 12/31/2009
• Est. Priority Date: 06/30/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Shimizu, Tetsuya, Kato, Shingo, Matsumoto, Yoshitaka, Masuno, Naruaki
• Granted Patent: US 8,428,304 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 20/42 of sport video content

Abstract

A computer system, method and computer program that retrieves, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved. The computer system includes a storage unit that stores a locus of a model of the moving image to be retrieved and velocity variation of the model; a first calculation unit that calculates a first vector including the locus and the velocity variation of the model; a second calculation unit that calculates a second vector regarding the moving image content to be retrieved included in the at least one piece of moving image data; a third calculation unit that calculates a degree of similarity between the first and second vectors; and a selection unit that selects, at least one scene which includes the moving image content to be retrieved, on the basis of the degree of similarity.

Claims

1. A computer system that retrieves, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved, the computer system comprising:

a storage unit that stores a locus of a model of the moving image content to be retrieved and velocity variation of the model;

a first calculation unit that calculates a first vector regarding the model, the first vector including the locus of the model and the velocity variation of the model;

a second calculation unit that calculates a second vector regarding the moving image content to be retrieved included in the at least one piece of moving image data, the second vector being calculated on the basis of information on a motion of the moving image content;

a third calculation unit that calculates a degree of similarity between the first and second vectors by comparing the first and second vectors; and

a selection unit that selects, from the at least one piece of moving image data, the at least one scene, which includes the moving image content to be retrieved, on the basis of the degree of similarity.

2. The computer system according to claim 1, wherein the model is a three-dimensional model, the locus of the model is a locus of the three-dimensional model, and the velocity variation of the model is velocity variation of the three-dimensional model.

3. The computer system according to claim 2, further comprising a visual point determining unit that allows a user to determine a visual point in three dimensions.

4. The computer system according to claim 3, wherein the first calculation unit, which calculates the first vector, calculates a first vector as viewed from the visual point in three dimensions determined by the user.

5. The computer system according to claim 3, wherein, when prior knowledge about a locus or velocity variation of the moving image content to be retrieved included in the moving image data exists, the visual point determining unit further determines visual points that are close to the prior knowledge as being candidate visual points.

6. The computer system according to claim 3, wherein the visual point determining unit further selects visual points at which velocity or acceleration obtained from the first vector is large in descending order of the velocity or the acceleration.

7. The computer system according to claim 1, further comprising a mapping unit that obtains, regarding moving images that include the moving image content to be retrieved, information on a motion of the moving image content.

8. The computer system according to claim 1, wherein the third calculation unit, which calculates a degree of similarity, includes means for calculating a degree of similarity between frames of moving images of the model and frames of moving images that include the moving image content to be retrieved included in the moving image data.

9. The computer system according to claim 1, wherein the third calculation unit, which calculates a degree of similarity, includes means for accommodating difference in velocity by dividing, into a constant number of frames, frames of moving images of the model or frames of moving images that include the moving image content to be retrieved included in the moving image data.

10. The computer system according to claim 3, wherein the third calculation unit, which calculates a degree of similarity, includes means for calculating a degree of similarity according to a weight assigned to each visual point.

11. The computer system according to claim 1, wherein each piece of moving image data that includes the moving image content to be retrieved includes a tag, further comprising:

search means for searching for tags to narrow down moving images that include the moving image content.

12. The computer system according to claim 1, wherein the third calculation unit, which calculates a degree of similarity, compares the first and second vectors, compares a velocity vector obtained from the first vector and a velocity vector obtained from the second vector, and calculates a degree of similarity from the respective results of the comparison.

13. A method for retrieving, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved, the method comprising:

a step of storing a locus of a model of the moving image content to be retrieved and velocity variation of the model;

a step of calculating a first vector regarding the model, the first vector including the locus of the model and the velocity variation of the model;

a step of calculating a second vector regarding the moving image content to be retrieved included in the at least one piece of moving image data, the second vector being calculated on the basis of information on a motion of the moving image content;

a step of calculating a degree of similarity between the first and second vectors by comparing the first and second vectors; and

a step of selecting, from the at least one piece of moving image data, the at least one scene, which includes the moving image content to be retrieved, on the basis of the degree of similarity.

14. The method according to claim 13, wherein the model is a three-dimensional model, the locus of the model is a locus of the three-dimensional model, and the velocity variation of the model is velocity variation of the three-dimensional model.

15. The method according to claim 14, further comprising a step of allowing a user to determine a visual point in three dimensions.

16. The method according to claim 14, wherein the step of calculating the first vector includes a step of calculating a first vector as viewed from the visual point in three dimensions determined by the user.

17. The method according to claim 15, wherein the step of allowing determination of a visual point includes a step of determining, when prior knowledge about a locus or velocity variation of the moving image content to be retrieved included in the moving image data exists, visual points that are close to the prior knowledge as being candidate visual points.

18. The method according to claim 16, wherein the step of allowing determination of a visual point includes a step of selecting visual points at which velocity or acceleration obtained from the first vector is large in descending order of the velocity or the acceleration.

19. The method according to claim 13, further comprising a step of obtaining, regarding moving images that include the moving image content to be retrieved, information on a motion of the moving image content.

20. The method according to claim 13, wherein the step of calculating a degree of similarity includes a step of calculating a degree of similarity between frames of moving images of the model and frames of moving images that include the moving image content to be retrieved included in the moving image data.

21. The method according to claim 13, wherein the step of calculating a degree of similarity includes a step of accommodating difference in velocity by dividing, into a constant number of frames, frames of moving images of the model or frames of moving images that include the moving image content to be retrieved included in the moving image data.

22. The method according to claim 15, wherein the step of calculating a degree of similarity includes a step of calculating a degree of similarity according to a weight assigned to each visual point.

23. The method according to claim 13, wherein each piece of moving image data that includes the moving image content to be retrieved includes a tag, further comprising:

a step of searching for tags to narrow down moving images that include the moving image content to be retrieved.

25. A computer program residing on a tangible computer media for retrieving, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved, the computer program causing a computer system to execute the method steps of claim 13.

24. A method for retrieving, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved, the method comprising:

a step of storing a locus of a three-dimensional model of the moving image content to be retrieved and velocity variation of the three-dimensional model;

a step of allowing a user to determine a visual point in three dimensions;

a step of calculating, regarding the three-dimensional model, a first vector as viewed from the determined visual point in three dimensions, the first vector including the locus of the three-dimensional model and the velocity variation of the three-dimensional model;

a step of calculating a second vector regarding the moving image content to be retrieved included in the at least one piece of moving image data, the second vector being calculated on the basis of information on a motion of the moving image content;

a step of calculating a degree of similarity between the first and second vectors by comparing the first and second vectors; and

a step of selecting, from the at least one piece of moving image data, the at least one scene, which includes the moving image content to be retrieved, on the basis of the degree of similarity.