Image processing apparatus, image processing method, and program

• US 7,949,188 B2
• Filed: 07/13/2007
• Issued: 05/24/2011
• Est. Priority Date: 07/14/2006
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Wu, Weiguo, Han, Bo, Yoshigahara, Takayuki
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US11/777,775
• Publication Number:
  US 20080118153A1
• Time in Patent Office: 1,411 Days
• Field of Search: 382/100, 382/181, 382/190, 382/195, 382/203, 382/209, 707/705, 707/708, 707/713
• US Class Current: 382/181
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

An image processing apparatus is disclosed which processes moving images each divisible into a plurality of shots, the image processing apparatus including: a holding unit configured to hold discrimination models acquired by learning beforehand a first rule and a second rule from a moving image formed by a plurality of known shot groups which are made up of at least one shot each and from which a highlight is to be extracted in accordance with the discrimination models, the first rule governing relevance between the shots, the second rule governing relevance between frames within each of the shots; and an extraction unit configured to extract from a newly input moving image a shot group recognized as the highlight in accordance with the discrimination models held in the holding unit.

Claims

1. An image processing apparatus for processing moving images each divisible into a plurality of shots, said image processing apparatus comprising:

a holding unit configured to hold discrimination models acquired by learning beforehand a first rule and a second rule from a moving image formed by a plurality of known shot groups which are made up of at least one shot each and from which a highlight is to be extracted in accordance with said discrimination models, said first rule governing relevance between said shots, said second rule governing relevance between frames within each of said shots; and

an extraction unit configured to extract from a newly input moving image a shot group recognized as said highlight in accordance with said discrimination models held in said holding unit.

2. The image processing apparatus according to claim 1, wherein the discrimination model based on said first rule is a time-series model acquired by learning a plurality of shot types as a state each;

and

wherein, from said newly input moving image, said extraction unit extracts the shot group recognized as said highlight representative of state transition between said shot types in accordance with said time-series model.

3. The image processing apparatus according to claim 2, wherein said time-series model is a hidden Markov model known as HMM.

4. The image processing apparatus according to claim 1, wherein the discrimination model based on said second rule is a time-series model acquired by learning a plurality of frame types as a state each;

and

wherein, from said newly input moving image, said extraction unit extracts the shot group including shots recognized as said highlight representative of state transition between said frame types in accordance with said time-series model.

5. The image processing apparatus according to claim 4, wherein said time-series model is a hidden Markov model known as HMM.

6. The image processing apparatus according to claim 5, wherein said plurality of frame types are each defined on the basis of at least one object feature includable in a frame;

wherein said image processing apparatus further comprises a feature extraction unit configured to attempt extracting each of said at least one object feature from a plurality of frames constituting said newly input moving image; and

wherein said extraction unit extracts said highlight on the basis of what is recognized as each of said frame types in said plurality of frames constituting said newly input moving image, the extractions being based on the attempt made by said feature extraction unit.

7. An image processing method for use with an image processing apparatus for processing moving images each divisible into a plurality of shots, said image processing method comprising the steps of:

holding, in said image processing apparatus, discrimination models acquired by learning beforehand a first rule and a second rule from a moving image formed by a plurality of known shot groups which are made up of at least one shot each and from which a highlight is to be extracted in accordance with said discrimination models, said first rule governing relevance between said shots, said second rule governing relevance between frames within each of said shots; and

extracting from a newly input moving image a shot group recognized as said highlight in accordance with said discrimination models being held.

8. A computer program stored on a non-transitory computer readable medium for control over the processing of moving images each divisible into a plurality of shots, said program comprising the steps of:

allowing said computer to hold discrimination models acquired by learning beforehand a first rule and a second rule from a moving image formed by a plurality of known shot groups which are made up of at least one shot each and from which a highlight is to be extracted in accordance with said discrimination model, said first rule governing relevance between said shots, said second rule governing relevance between frames within each of said shots; and

causing said computer to extract from a newly input moving image a shot group recognized as said highlight in accordance with said discrimination models being held.