INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, AND PROGRAM

• US 20120057775A1
• Filed: 03/31/2011
• Published: 03/08/2012
• Est. Priority Date: 04/09/2010
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Suzuki, Hirotaka, Ito, Masato, Sabe, Kohtaro
• Application Number:
  US13/076,744
• Publication Number:
  US 20120057775A1
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/775 between a recording apparatus and a television receiver
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 5/783 Adaptations for reproducing at a rate different from the recording rate
• CPC Class Codes: H04N 9/8211 the additional signal being a sound signal H04N9/835 takes precedence

Abstract

An information processing device includes a feature amount extracting unit configured to extract the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene; a clustering unit configured to use cluster information that is the information of the cluster obtained by performing cluster learning; a highlight label generating unit configured to generate a highlight label sequence; and a highlight detector learning unit configured to perform learning of the highlight detector.

Claims

1. An information processing device comprising:

feature amount extracting means configured to extract the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene;

clustering means configured to use cluster information that is the information of said cluster obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs;

highlight label generating means configured to generate a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations; and

highlight detector learning means configured to perform learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence.

2. The information processing device according to claim 1, further comprising:

highlight detecting means configured to extract the feature amount of each frame of an image of a content for highlight detection of interest that is a content from which a highlight scene is to be detected,

to convert the time sequence of the feature amount of said content for highlight detection of interest into said code sequence by subjecting the feature amount of each frame of said content for highlight detection of interest to clustering into one cluster of said plurality of clusters using said cluster information,

to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that a label sequence for detection that is a pair of said code sequence obtained from said content for highlight detection of interest, and the highlight label sequence of a highlight label representing a highlight scene or non-highlight scene will be observed in said highlight detector,

to detect the frame of a highlight scene from said content for highlight detection of interest based on the observation probability of said highlight label of each state of a highlight relation state sequence that is said maximum likelihood state sequence obtained from said label sequence for detection, and

to generate a digest content that is the digest of said content for highlight detection of interest using the frame of said highlight scene.

3. The information processing device according to claim 2, wherein said highlight detecting means detect, in the event that difference between the observation probability of a highlight label representing a highlight scene, and the observation probability of a highlight label representing a non-highlight scene in a predetermined point-in-time state of said highlight relation state sequence is greater than a predetermined threshold, the frame of said content for highlight detection of interest corresponding to said predetermined point-in-time state as the frame of a highlight scene.

4. The information processing device according to claim 1, further comprising:

scrapbook generating means configured to extract the feature amount of each frame of an image of a content,

to subject the feature amount of said content to clustering using said cluster information to convert into a code sequence,

to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that the code sequence of said content will be observed with a code model that is a state transition probability model after said model learning obtained by performing model learning that is learning of a state transition probability model using the code sequence of said content for learning,

to extract, of the states of said maximum likelihood state sequence, a frame corresponding to a state matching the state instructed by the user, from said content, and

to register the frame extracted from said content in a scrapbook in which said highlight scene is registered.

5. The information processing device according to claim 1, further comprising:

inter-state distance calculating means configured to obtain inter-state distance from one state to another state of said code model based on state transition probability from said one state to said another state;

coordinates calculating means configured to obtain, so as to reduce error between Euclidean distance from said one state to said another state and said inter-state distance on a model map that is a two-dimensional or three-dimensional map where a state of said code model is disposed, state coordinates that are the coordinates of the position of said state on said model map; and

display control means configured to perform display control for displaying said model map where said corresponding state is disposed in the position of said state coordinates.

6. The information processing device according to claim 5, wherein said coordinates calculating means obtain said state coordinates so as to minimize a Sammon Map error function in proportion to statistical error between said Euclidean distance and said inter-state distance, and in the event that the Euclidean distance from said one state to said another state is greater than a predetermined threshold, set the Euclidean distance from said one state to said another state to distance equal to said inter-state distance from said one state to said another state, and perform calculation of said error function.

7. The information processing device according to claim 5, further comprising:

scrapbook generating means configured to extract the feature amount of each frame of an image of a content,

to subject the feature amount of said content to clustering using said cluster information to convert into a code sequence,

to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that the code sequence of said content will be observed with a code model that is a state transition probability model after said model learning obtained by performing model learning that is learning of a state transition probability model using the code sequence of said content for learning,

to extract, of the states of said maximum likelihood state sequence, a frame corresponding to a state matching a state on said model map, instructed by the user, from said content, and

to register the frame extracted from said content in a scrapbook in which said highlight scene is registered.

8. The information processing device according to claim 1, wherein the feature amount of said frame is obtained by dividing said frame into sub regions that are a plurality of small regions, extracting the feature amount of each of said plurality of sub regions, and combining the feature amount of each of said plurality of sub regions.

9. The information processing device according to claim 1, wherein the feature amount of said frame is obtained by combining a mean value and dispersion of audio energy, zero crossing rate, or spectrum center of gravity within predetermined time corresponding to said frame.

10. The information processing device according to claim 1, wherein the feature amount of said frame is obtained by detecting the display region of an object within said frame, dividing said frame into sub regions that are a plurality of small regions, extracting the percentage of the number of pixels of the display region of said object in said sub regions as to the number of pixels in each of said plurality of sub regions, as feature amount, and combining the feature amount of each of said plurality of sub regions.

11. The information processing device according to claim 1, further comprising:

cluster information and code model learning means configured to obtain said cluster information by performing cluster learning for dividing said feature amount space into a plurality of clusters using the feature amount of said content for learning, and also

to generate said code model by performing model learning of a state transition probability model using a code sequence obtained by subjecting the feature amount of said content for learning to clustering using said cluster information.

12. An information processing method using an information processing device, comprising the steps of:

extracting the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene;

using cluster information that is the information of said cluster obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs;

generating a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations; and

performing learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence.

13. A program causing a computer to serve as:

feature amount extracting means configured to extract the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene;

clustering means configured to use cluster information that is the information of said clusters obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs;

highlight label generating means configured to generate a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations; and

highlight detector learning means configured to perform learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence.

14. An information processing device comprising:

obtaining means configured to obtain said highlight detector obtained by

extracting the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene,

using cluster information that is the information of said clusters obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs,

generating a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations, and

performing learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence;

feature amount extracting means configured to extract the feature amount of each frame of an image of a content for highlight detection of interest that is a content from which a highlight scene is to be detected;

clustering means configured to convert the time sequence of the feature amount of said content for highlight detection of interest into said code sequence by subjecting the feature amount of each frame of said content for highlight detection of interest to clustering into one cluster of said plurality of clusters using said cluster information;

maximum likelihood state sequence estimating means configured to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that a label sequence for detection that is a pair of said code sequence obtained from said content for highlight detection of interest, and the highlight label sequence of a highlight label representing a highlight scene or non-highlight scene will be observed in said highlight detector;

highlight scene detecting means configured to detect the frame of a highlight scene from said content for highlight detection of interest based on the observation probability of said highlight label of each state of a highlight relation state sequence that is said maximum likelihood state sequence obtained from said label sequence for detection; and

digest contents generating means configured to generate a digest content that is the digest of said content for highlight detection of interest using the frame of said highlight scene.

15. The information processing device according to claim 14, wherein said highlight detecting means detect, in the event that difference between the observation probability of a highlight label representing a highlight scene, and the observation probability of a highlight label representing a non-highlight scene in a predetermined point-in-time state of said highlight relation state sequence is greater than a predetermined threshold, the frame of said content for highlight detection of interest corresponding to said predetermined point-in-time state as the frame of a highlight scene.

16. The information processing device according to claim 14, further comprising:

scrapbook generating means configured to extract the feature amount of each frame of an image of a content,

to subject the feature amount of said content to clustering using said cluster information to convert into a code sequence,

to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that the code sequence of said content will be observed with a code model that is a state transition probability model after said model learning obtained by performing model learning that is learning of a state transition probability model using the code sequence of said content for learning,

to extract, of the states of said maximum likelihood state sequence, a frame corresponding to a state matching the state instructed by the user, from said content, and

to register the frame extracted from said content in a scrapbook in which said highlight scene is registered.

17. The information processing device according to claim 14, further comprising:

inter-state distance calculating means configured to obtain inter-state distance from one state to another state of said code model based on state transition probability from said one state to said another state;

coordinates calculating means configured to obtain, so as to reduce error between Euclidean distance from said one state to said another state and said inter-state distance on a model map that is a two-dimensional or three-dimensional map where a state of said code model is disposed, state coordinates that are the coordinates of the position of said state on said model map; and

display control means configured to perform display control for displaying said model map where said corresponding state is disposed in the position of said state coordinates.

18. The information processing device according to claim 17, wherein said coordinates calculating means obtain said state coordinates so as to minimize a Sammon Map error function in proportion to statistical error between said Euclidean distance and said inter-state distance, and in the event that the Euclidean distance from said one state to said another state is greater than a predetermined threshold, set the Euclidean distance from said one state to said another state to distance equal to said inter-state distance from said one state to said another state, and perform calculation of said error function.

19. The information processing device according to claim 17, further comprising:

scrapbook generating means configured to extract the feature amount of each frame of an image of a content,

to subject the feature amount of said content to clustering using said cluster information to convert into a code sequence,

to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that the code sequence of said content will be observed with a code model that is a state transition probability model after said model learning obtained by performing model learning that is learning of a state transition probability model using the code sequence of said content for learning,

to extract, of the states of said maximum likelihood state sequence, a frame corresponding to a state matching a state on said model map, instructed by the user, from said content, and

to register the frame extracted from said content in a scrapbook in which said highlight scene is registered.

20. The information processing device according to claim 14, wherein the feature amount of said frame is obtained by dividing said frame into sub regions that are a plurality of small regions, extracting the feature amount of each of said plurality of sub regions, and combining the feature amount of each of said plurality of sub regions.

21. The information processing device according to claim 14, wherein the feature amount of said frame is obtained by combining a mean value and dispersion of audio energy, zero crossing rate, or spectrum center of gravity within predetermined time corresponding to said frame.

22. The information processing device according to claim 14, wherein the feature amount of said frame is obtained by detecting the display region of an object within said frame, dividing said frame into sub regions that are a plurality of small regions, extracting the percentage of the number of pixels of the display region of said object in said sub regions as to the number of pixels in each of said plurality of sub regions, as feature amount, and combining the feature amount of each of said plurality of sub regions.

23. An information processing method using an information processing device, comprising the steps of:

obtaining said highlight detector to be obtained by

extracting the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene,

using cluster information that is the information of said clusters obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs,

generating a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations, and

performing learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence;

extracting the feature amount of each frame of an image of a content for highlight detection of interest that is a content from which a highlight scene is to be detected;

converting the time sequence of the feature amount of said content for highlight detection of interest into said code sequence by subjecting the feature amount of each frame of said content for highlight detection of interest to clustering into one cluster of said plurality of clusters using said cluster information;

estimating the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that a label sequence for detection that is a pair of said code sequence obtained from said content for highlight detection of interest, and the highlight label sequence of a highlight label representing a highlight scene or non-highlight scene will be observed in said highlight detector;

detecting the frame of a highlight scene from said content for highlight detection of interest based on the observation probability of said highlight label of each state of a highlight relation state sequence that is said maximum likelihood state sequence obtained from said label sequence for detection; and

generating a digest content that is the digest of said content for highlight detection of interest using the frame of said highlight scene.

24. A program causing a computer to serve as:

obtaining means configured to obtain said highlight detector obtained by

extracting the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene,

using cluster information that is the information of said clusters obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs,

generating a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations, and

performing learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence;

feature amount extracting means configure to extract the feature amount of each frame of an image of a content for highlight detection of interest that is a content from which a highlight scene is to be detected;

clustering means configured to convert the time sequence of the feature amount of said content for highlight detection of interest into said code sequence by subjecting the feature amount of each frame of said content for highlight detection of interest to clustering into one cluster of said plurality of clusters using said cluster information;

maximum likelihood state sequence estimating means configured to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that a label sequence for detection that is a pair of said code sequence obtained from said content for highlight detection of interest, and the highlight label sequence of a highlight label representing a highlight scene or non-highlight scene will be observed in said highlight detector;

highlight scene detecting means configured to detect the frame of a highlight scene from said content for highlight detection of interest based on the observation probability of said highlight label of each state of a highlight relation state sequence that is said maximum likelihood state sequence obtained from said label sequence for detection; and

digest contents generating means configured to generate a digest content that is the digest of said content for highlight detection of interest using the frame of said highlight scene.

25. An information processing device comprising:

a feature amount extracting unit configured to extract the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene;

a clustering unit configured to use cluster information that is the information of said cluster obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs;

a highlight label generating unit configured to generate a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations; and

a highlight detector learning unit configured to perform learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence.

26. A program causing a computer to serve as:

a feature amount extracting unit configured to extract the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene;

a clustering unit configured to use cluster information that is the information of said clusters obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs;

a highlight label generating unit configured to generate a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations; and

a highlight detector learning unit configured to perform learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence.

27. An information processing device comprising:

an obtaining unit configured to obtain said highlight detector obtained by

extracting the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene,

using cluster information that is the information of said clusters obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs,

generating a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations, and

performing learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence;

a feature amount extracting unit configured to extract the feature amount of each frame of an image of a content for highlight detection of interest that is a content from which a highlight scene is to be detected;

a clustering unit configured to convert the time sequence of the feature amount of said content for highlight detection of interest into said code sequence by subjecting the feature amount of each frame of said content for highlight detection of interest to clustering into one cluster of said plurality of clusters using said cluster information;

a maximum likelihood state sequence estimating unit configured to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that a label sequence for detection that is a pair of said code sequence obtained from said content for highlight detection of interest, and the highlight label sequence of a highlight label representing a highlight scene or non-highlight scene will be observed in said highlight detector;

a highlight scene detecting unit configured to detect the frame of a highlight scene from said content for highlight detection of interest based on the observation probability of said highlight label of each state of a highlight relation state sequence that is said maximum likelihood state sequence obtained from said label sequence for detection; and

a digest contents generating unit configured to generate a digest content that is the digest of said content for highlight detection of interest using the frame of said highlight scene.

28. A program causing a computer to serve as:

an obtaining unit configured to obtain said highlight detector obtained by

extracting the feature amount of each frame of an image of a content for detector learning of interest that is a content to be used for learning of a highlight detector which is a model for detecting a scene in which the user is interested as a highlight scene,

using cluster information that is the information of said clusters obtained by performing cluster learning for extracting the feature amount of each frame of an image of a content for learning that is a content to be used for cluster learning for dividing feature amount space that is the space of said feature amount into a plurality of clusters, and dividing said feature amount space into a plurality of clusters using the feature amount of each frame of said content for learning to subject the feature amount of each frame of said content for detector learning of interest to clustering into one cluster of said plurality of clusters, thereby converting the time sequence of the feature amount of said content for detector learning of interest into the code sequence of a code representing a cluster to which the feature amount of said content for detector learning of interest belongs,

generating a highlight label sequence regarding said content for detector learning of interest by labeling each frame of said content for detector learning of interest using a highlight label representing whether or not said highlight scene in accordance with the user'"'"'s operations, and

performing learning of said highlight detector which is a state transition probability model stipulated by state transition probability that a state will proceed, and observation probability that a predetermined observation value will be observed from said state, using a label sequence for learning that is a pair of said code sequence obtained from said content for detector learning of interest, and said highlight label sequence;

a feature amount extracting unit configure to extract the feature amount of each frame of an image of a content for highlight detection of interest that is a content from which a highlight scene is to be detected;

a clustering unit configured to convert the time sequence of the feature amount of said content for highlight detection of interest into said code sequence by subjecting the feature amount of each frame of said content for highlight detection of interest to clustering into one cluster of said plurality of clusters using said cluster information;

a maximum likelihood state sequence estimating unit configured to estimate the maximum likelihood state sequence that is a state sequence causing state transition to occur where likelihood is the highest that a label sequence for detection that is a pair of said code sequence obtained from said content for highlight detection of interest, and the highlight label sequence of a highlight label representing a highlight scene or non-highlight scene will be observed in said highlight detector;

a highlight scene detecting unit configured to detect the frame of a highlight scene from said content for highlight detection of interest based on the observation probability of said highlight label of each state of a highlight relation state sequence that is said maximum likelihood state sequence obtained from said label sequence for detection; and

a digest contents generating unit configured to generate a digest content that is the digest of said content for highlight detection of interest using the frame of said highlight scene.