Retrieving scenes from moving image data

• US 8,428,304 B2
• Filed: 05/29/2009
• Issued: 04/23/2013
• Est. Priority Date: 06/30/2008
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kato, Shingo, Masuno, Naruaki, Matsumoto, Yoshitaka, Shimizu, Tetsuya
• Primary Examiner(s): Akhavannik, Hadi
• Assistant Examiner(s): RASHIDIAN, MOHAMMAD M
• Application Number:
  US12/474,835
• Publication Number:
  US 20090324014A1
• Time in Patent Office: 1,425 Days
• Field of Search: 382/100, 382103-104, 382/107, 382115-118, 382/168, 382/172, 382124-127, 382/260, 356/27, 345/473, 348/143, 348/154, 348/107.99, 3482081-2084, 348/208.12, 348/208.99, 348699-700
• US Class Current: 382/103
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 20/42 of sport video content

Abstract

A computer system, method and computer program that retrieves, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved. The computer system includes a storage unit that stores a locus of a model of the moving image to be retrieved and velocity variation of the model; a first calculation unit that calculates a first vector including the locus and the velocity variation of the model; a second calculation unit that calculates a second vector regarding the moving image content to be retrieved included in the at least one piece of moving image data; a third calculation unit that calculates a degree of similarity between the first and second vectors; and a selection unit that selects, at least one scene which includes the moving image content to be retrieved, on the basis of the degree of similarity.

Claims

1. A computer system that retrieves, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved, the computer system comprising:

a storage unit that stores;

(a) a locus of a three-dimensional model of the moving image content to be retrieved, and (b) a velocity variation of the three-dimensional model, the three-dimensional model being a model in three spatial dimensions, the locus being a locus within the three spatial dimensions, the velocity variation being a velocity variation in the three spatial dimensions;

a first calculation unit that calculates a first vector regarding the three-dimensional model, the first vector including the locus of the three-dimensional model and the velocity variation of the three-dimensional model, wherein the first calculation unit calculates the first vector regarding the three-dimensional model as viewed from a selected visual point in the three spatial dimensions;

a second calculation unit that calculates a second vector regarding the moving image content to be retrieved included in the at least one piece of moving image data, the second vector being calculated on the basis of information on a motion of the moving image content;

a third calculation unit that calculates a degree of similarity between the first and second vectors by comparing the first and second vectors; and

a selection unit that selects, from the at least one piece of moving image data, the at least one scene, which includes the moving image content to be retrieved, on the basis of the degree of similarity.

2. The computer system according to claim 1, further comprising a visual point determining unit that allows a user to determine the visual point in the three spatial dimensions.

3. The computer system according to claim 2, wherein, when prior knowledge about a locus or velocity variation of the moving image content to be retrieved included in the moving image data exists, the visual point determining unit further determines visual points that are close to the prior knowledge as being candidate visual points.

4. The computer system according to claim 2, wherein the visual point determining unit further selects visual points at which velocity or acceleration obtained from the first vector is large in descending order of the velocity or the acceleration.

5. The computer system according to claim 1, further comprising a mapping unit that obtains, regarding moving images that include the moving image content to be retrieved, information on a motion of the moving image content.

6. The computer system according to claim 1, wherein the third calculation unit, which calculates a degree of similarity, includes means for calculating a degree of similarity between frames of moving images of the model and frames of moving images that include the moving image content to be retrieved included in the moving image data.

7. The computer system according to claim 1, wherein the third calculation unit, which calculates a degree of similarity, includes means for accommodating difference in velocity by dividing, into a constant number of frames, frames of moving images of the model or frames of moving images that include the moving image content to be retrieved included in the moving image data.

8. The computer system according to claim 2, wherein the third calculation unit, which calculates a degree of similarity, includes means for calculating a degree of similarity according to a weight assigned to each visual point.

9. The computer system according to claim 1, wherein each piece of moving image data that includes the moving image content to be retrieved includes a tag, further comprising:

search means for searching for tags to narrow down moving images that include the moving image content.

10. The computer system according to claim 1, wherein the third calculation unit, which calculates a degree of similarity, compares the first and second vectors, compares a velocity vector obtained from the first vector and a velocity vector obtained from the second vector, and calculates a degree of similarity from the respective results of the comparison.

11. A method for retrieving, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved, the method comprising:

a step of storing;

(a) a locus of a three-dimensional model of the moving image content to be retrieved, and (b) a velocity variation of the three-dimensional model, the three-dimensional model being a model in three spatial dimensions, the locus being a locus within the three spatial dimensions, the velocity variation being a velocity variation in the three spatial dimensions;

a step of calculating a first vector regarding the three-dimensional model, the first vector including the locus of the three-dimensional model and the velocity variation of the three-dimensional model, wherein the first calculation unit calculates the first vector regarding the three-dimensional model as viewed from a selected visual point in the three spatial dimensions;

a step of calculating a second vector regarding the moving image content to be retrieved included in the at least one piece of moving image data, the second vector being calculated on the basis of information on a motion of the moving image content;

a step of calculating a degree of similarity between the first and second vectors by comparing the first and second vectors; and

a step of selecting, from the at least one piece of moving image data, the at least one scene, which includes the moving image content to be retrieved, on the basis of the degree of similarity.

12. The method according to claim 11, further comprising a step of allowing a user to determine a visual point in the three spatial dimensions.

13. The method according to claim 12, wherein the step of allowing determination of a visual point includes a step of determining, when prior knowledge about a locus or velocity variation of the moving image content to be retrieved included in the moving image data exists, visual points that are close to the prior knowledge as being candidate visual points.

14. The method according to claim 12, wherein the step of allowing determination of a visual point includes a step of selecting visual points at which velocity or acceleration obtained from the first vector is large in descending order of the velocity or the acceleration.

15. The method according to claim 11, further comprising a step of obtaining, regarding moving images that include the moving image content to be retrieved, information on a motion of the moving image content.

16. The method according to claim 11, wherein the step of calculating a degree of similarity includes a step of calculating a degree of similarity between frames of moving images of the model and frames of moving images that include the moving image content to be retrieved included in the moving image data.

17. The method according to claim 11, wherein the step of calculating a degree of similarity includes a step of accommodating difference in velocity by dividing, into a constant number of frames, frames of moving images of the model or frames of moving images that include the moving image content to be retrieved included in the moving image data.

18. The method according to claim 12, wherein the step of calculating a degree of similarity includes a step of calculating a degree of similarity according to a weight assigned to each visual point.

19. The method according to claim 11, wherein each piece of moving image data that includes the moving image content to be retrieved includes a tag, further comprising:

a step of searching for tags to narrow down moving images that include the moving image content to be retrieved.

21. A computer program residing on a tangible computer media for retrieving, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved, the computer program causing a computer system to execute the method steps of claim 11.

20. A method for retrieving, from at least one piece of moving image data, at least one scene that includes moving image content to be retrieved, the method comprising:

a step of storing a locus of a three-dimensional model of the moving image content to be retrieved and velocity variation of the three-dimensional model;

a step of storing;

(a) a locus of a three-dimensional model of the moving image content to be retrieved, and (b) a velocity variation of the three-dimensional model, the three-dimensional model being a model in three spatial dimensions, the locus being a locus within the three spatial dimensions, the velocity variation being a velocity variation in the three spatial dimensions;

a step of allowing a user to determine a visual point in the three spatial dimensions;

a step of calculating, regarding the three-dimensional model, a first vector as viewed from the determined visual point in the three spatial dimensions, the first vector including the locus of the three-dimensional model and the velocity variation of the three-dimensional model;

a step of calculating a second vector regarding the moving image content to be retrieved included in the at least one piece of moving image data, the second vector being calculated on the basis of information on a motion of the moving image content;

a step of calculating a degree of similarity between the first and second vectors by comparing the first and second vectors; and

a step of selecting, from the at least one piece of moving image data, the at least one scene, which includes the moving image content to be retrieved, on the basis of the degree of similarity.