Intelligent video thumbnail selection and generation

• US 9,830,515 B2
• Filed: 11/01/2016
• Issued: 11/28/2017
• Est. Priority Date: 10/10/2012
• Status: Active Grant

• Current Assignee: BroadbandTV, Corp. (BBTV Holdings, Inc.)
• Original Assignee: BroadbandTV, Corp. (BBTV Holdings, Inc.)
• Inventors: Rafati, Shahrzad, Fatourechi, Mehrdad, Coria, Lino
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US15/340,546
• Publication Number:
  US 20170053170A1
• Time in Patent Office: 392 Days
• Field of Search: None
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

In accordance with one embodiment, an intelligent video thumbnail selection and generation tool may select a relevant and visually stimulating image from a video file and generate a thumbnail including the image. The image may be selected by computing a relevancy metric for an image in the file based on one or more selected relevant features, and comparing that relevancy metric with the metric of at least one other image in the file. In another embodiment, a series of images in a video file may be divided into shots. One of the shots may be selected based on a shot relevancy metric and a key image from the shot may be selected as a thumbnail based on a key image relevancy metric, where the shot relevancy metric and the key image relevancy metrics may be computed based on one or more relevant content features.

Claims

1. A method for recommending a thumbnail image representative of a collection of images, the method comprising:

obtaining a relevancy metric for each image of the collection of images, wherein the relevancy metric is computed based on at least one image characteristic selected from the group consisting of;

a size of a face in the image, wherein the relevancy metric is influenced in a first direction more when the face is large than when the face is small;

a number of eyes in the image, wherein the relevancy metric is influenced in the first direction more when the number of eyes is large than when the number of eyes is small;

a number of open eyes in the image, wherein the relevancy metric is influenced in the first direction more when the number of open eyes is large than when the number of open eyes is small;

an object or face in the image, wherein the relevancy metric is influenced in the first direction more when the object or face is identified as corresponding to a database entry than when the object or face is unidentified; and

a number of skin-colored pixels in the image, wherein the relevancy metric is influenced in the first direction more when the number of skin-colored pixels is large than when the number of skin-colored pixels is small;

identifying a subset of the collection of images that each have a relevancy metric influenced in the first direction by an amount that satisfies a predetermined threshold; and

transmitting for presentation through a user interface the identified subset of the collection of images.

2. The method of claim 1 further comprising:

presenting the identified subset of the collection of images alongside an original thumbnail image; and

receiving user feedback identifying a select image of the presented images.

3. The method of claim 2 further comprising:

transmitting the select image to a video distribution site for use as a thumbnail image for an associated video.

4. The method of claim 2, further comprising adding text, graphics, or an additional image to the select image.

5. The method of claim 1 further comprising:

wherein the relevancy metric for an image is influenced more heavily in the first direction when a number of faces in the first targeted image is large than when a number of faces in the first targeted image is small.

6. The method of claim 1 wherein the relevancy metric for the first targeted image is influenced more heavily in the first direction more when a degree of contrast is high than when a degree of contrast is low.

7. The method of claim 1 and further comprising at least one of downsizing, down sampling, or filtering of the collection of images.

8. The method of claim 1, wherein the relevancy metric of each image in the collection of images is based on at least one of texture or brightness.

9. The method of claim 1 wherein the relevancy metric for an image is influenced more in the first direction when a region of interest is located in an off-center position in the image than when a region of interest is located in a central position in the image.

20. The method of claim 1, wherein the relevancy metric of each image in the collection of images is based on at least one of texture or brightness.

10. One or more computer-readable storage media of a tangible article of manufacture encoding computer-executable instructions for executing on a computer system a computer process, the computer process comprising:

selecting at least one image from a collection of images to be a selected image based on at least one image characteristic selected from the group consisting of;

a size of a face in the image, wherein an image of the collection is more likely to be the selected image when the size of the face is large than when the size of the face is small;

a number of eyes in an image, wherein an image of the collection is more likely to be the selected image when the number of eyes is large than when the number of eyes is small;

a number of “

skin-colored”

pixels in the image, wherein an image of the collection is more likely to be the selected image when the number of skin-colored pixels is large than when the number of skin-colored pixels is small;

an object or face in the image, wherein an image of the collection is more likely to be the selected image when the object or face is identified as corresponding to a database entry than when the object or face is unidentified; and

a number of open eyes in the first targeted image, wherein an image of the collection is more likely to be the selected image when the number of open eyes is large than when the number of open eyes is small.

11. The one or more computer-readable storage media of claim 10, wherein the computer process further comprises:

dividing a group of images into a plurality of subgroups, each subgroup comprising a plurality of subgroup images arranged in chronological order, wherein the collection of images is a subgroup selected from the plurality of subgroups based on a visual similarity metric.

12. The one or more computer-readable storage media of claim 11, wherein the visual similarity metric is based on a number of images in the selected subgroup.

13. The one or more computer-readable storage media of claim 10, wherein the visual similarity metric is based on at least one of a number of faces in an image, a size of a face in an image, a number of eyes in an image, a number of “

skin-colored”

pixels in an image, texture in an image, brightness in an image, and contrast of an image.

14. The method of claim 10, wherein an image of the selected subgroup is more likely to be the selected image when a region of interest is located in an off-center position in the image than when a region of interest is located in a central position in the image.

15. A system comprising:

memory;

a module stored in the memory and executable by a processor, the module configured to;

compute a relevancy metric for each image of a collection of images based on an image characteristic selected from the group consisting of;

a size of a face in the image, wherein the relevancy metric is influenced in a first direction more when the face is large than when the face is small;

a number of eyes in the image, wherein the relevancy metric is influenced in the first direction more when the number of eyes is large than when the number of eyes is small;

a number of open eyes in the image, wherein the relevancy metric is influenced in the first direction more when the number of open eyes is large than when the number of open eyes is small;

an object or face in the image, wherein the relevancy metric is influenced in the first direction more when the object or face is identified as corresponding to a database entry than when then object or face is unidentified;

a number of skin-colored pixels in the image, wherein the relevancy metric is influenced in the first direction more when the number of skin-colored pixels is large than when the number of skin-colored pixels is small;

identify a subset of the collection of images, the subset including images that each have a relevancy metric influenced in the first direction by an amount that satisfies a predetermined threshold;

transmit for presentation through a user interface the identified subset of one or more images.

16. The system of claim 15, wherein the module is further configured to:

present the identified subset of the collection of images alongside an original thumbnail image; and

receive user feedback specifying a select image of the presented images.

17. The system of claim 16, wherein the module is further configured to:

transmit the select image to a video distribution site for use as a thumbnail image for an associated video.

18. The system of claim 17, wherein the module is further configured to:

add text, graphics, or an additional image to the select image.

19. The system of claim 17, wherein the module is further configured to perform at least one task from the collection of tasks comprising:

downsizing, down sampling, and filtering the collection of images.