Intelligent video thumbnail selection and generation

• US 9,514,536 B2
• Filed: 10/10/2013
• Issued: 12/06/2016
• Est. Priority Date: 10/10/2012
• Status: Active Grant

• Current Assignee: BroadbandTV, Corp. (BBTV Holdings, Inc.)
• Original Assignee: BroadbandTV, Corp. (BBTV Holdings, Inc.)
• Inventors: Rafati, Shahrzad, Fatourechi, Mehrdad, Coria, Lino
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US14/051,285
• Publication Number:
  US 20140099034A1
• Time in Patent Office: 1,153 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

In accordance with one embodiment, an intelligent video thumbnail selection and generation tool may select a relevant and visually stimulating image from a video file and generate a thumbnail including the image. The image may be selected by computing a relevancy metric for an image in the file based on one or more selected relevant features, and comparing that relevancy metric with the metric of at least one other image in the file. In another embodiment, a series of images in a video file may be divided into shots. One of the shots may be selected based on a shot relevancy metric and a key image from the shot may be selected as a thumbnail based on a key image relevancy metric, where the shot relevancy metric and the key image relevancy metrics may be computed based on one or more relevant content features.

Claims

1. A method comprising:

obtaining a relevancy metric for a first targeted image of a collection of images, wherein the relevancy metric is computed based on an image characteristic selected from the group consisting of;

a size of a face in the first targeted image, wherein the relevancy metric is higher when the face is large than when the face is small;

a number of eyes in the first targeted image, wherein the relevancy metric is higher when the number of eyes is large than when the number of eyes is small; and

a number of skin-colored pixels in the first targeted image, wherein the relevancy metric is higher when the number of skin-colored pixels is large than when the number of skin-colored pixels is small;

comparing with a computer the relevancy metric of the first targeted image with the relevancy metric of at least a second targeted image of the collection of images;

transmitting for presentation through a user interface the image having the higher relevancy metric.

2. The method of claim 1 wherein the relevancy metric for the first targeted image is based on one or more content features.

3. The method of claim 1 wherein the relevancy metric for the first targeted image indicates a relevancy of the first targeted image to the subject matter of the collection of images.

4. The method of claim 1 and further comprising at least one of downsizing, down sampling, or filtering the collection of images.

5. The method of claim 1, wherein the relevancy metric is based on at least one of texture of the first targeted image, brightness of the first targeted image, or contrast of the first targeted image.

6. The method of claim 1 wherein the relevancy metric for the first targeted image is higher when a region of interest is located in an off-center position in the first targeted image than when a region of interest is located in a central position in the first targeted image.

7. The method of claim 1 and further comprising adding text, graphics, or an additional image to a selected image.

30. The method of claim 1, wherein the relevancy metric is based on a number of open eyes in the first targeted image and the relevancy metric is higher when the number of open eyes is large than when the number of open eyes is small.

8. A method comprising:

dividing a group of images into a plurality of subgroups, each subgroup comprising a plurality of subgroup images arranged in chronological order;

selecting one of the plurality of subgroups based on a visual similarity metric; and

selecting at least one image from the selected subgroup to be a selected image based at least one image characteristic selected from the group consisting of;

a size of a face in the image, wherein an image of the selected subgroup is more likely to be the selected image when the size of the face is large than when the size of the face is small;

a number of eyes in an image, wherein an image of the selected subgroup is more likely to be the selected image when the number of eyes is large than when the number of eyes is small; and

a number of “

skin-colored”

pixels in the image, wherein an image of the selected subgroup is more likely to be the selected image when the number of skin-colored pixels is large than when the number of skin-colored pixels is small.

9. The method of claim 8, wherein the visual similarity metric is based on a number of images in the selected subgroup.

10. The method of claim 8, wherein the visual similarity metric is based on at least one of a number of faces in an image, a size of a face in an image, a number of eyes in an image, a number of “

skin-colored”

pixels in an image, texture in an image, brightness in an image, and contrast of an image.

11. The method of claim 8 wherein an image of the selected subgroup is more likely to be the selected image when a region of interest is located in an off-center position in the image than when a region of interest is located in a central position in the image.

12. The method of claim 8 and further comprising adding text, graphics, or an additional image to the selected image.

13. The method of claim 8, further comprising at least one of resizing or filtering the selected image.

31. The method of claim 8, wherein selecting the at least image from the selected subgroup further comprises selecting the at least one image based on a number of open eyes in each image of the selected subgroup, wherein an image of the selected subgroup is more likely to be the selected image when the number of open eyes is large than when the number of open eyes is small.

14. A system comprising:

a relevancy metric computed for a first targeted image of a collection of images based on an image characteristic selected from the group consisting of;

a size of a face in the first targeted image, wherein the relevancy metric is higher when the face is large than when the face is small,

a number of eyes in the first targeted image, wherein the relevancy metric is higher when the number of eyes is large than when the number of eyes is small, or

a number of skin-colored pixels in the first targeted image, wherein the relevancy metric is higher when the number of skin-colored pixels is large than when the number of skin-colored pixels is small;

a computer configured to compare the relevancy metric of the first targeted image with the relevancy metric of at least a second targeted image of the collection of images;

a user interface configured to transmit for presentation the image having the higher relevancy metric.

15. The system of claim 14 wherein the relevancy metric for the first targeted image is based on one or more content features.

16. The system of claim 14 wherein the relevancy metric for the first targeted image indicates a relevancy of the first targeted image to the subject matter of the collection of images.

17. The system of claim 14 wherein the computer is configured at least to downsize, down sample, or filter the collection of images.

18. The system of claim 14, wherein the relevancy metric is based on at least one of texture of the first targeted image, brightness of the first targeted image, or contrast of the first targeted image.

19. The system of claim 14 wherein the relevancy metric for the first targeted image is higher when a region of interest is located in an off-center position in the first targeted image than when a region of interest is located in a central position in the first targeted image.

20. The system of claim 14, further comprising adding text, graphics, or an additional image to a selected image.

21. A system comprising:

a visual similarity metric;

a computer configured to;

divide a group of images into a plurality of subgroups, each subgroup comprising a plurality of subgroup images arranged in chronological order;

select one of the plurality of subgroups based on a visual similarity metric; and

select at least one image from the selected subgroup to be a selected image based on at least one image characteristic selected from the group consisting of;

a size of a face in the image, wherein an image of the selected subgroup is more likely to be the selected image when the size of the face is large than when the size of the face is small;

a number of eyes in the image, wherein an image of the selected subgroup is more likely to be the selected image when the number of eyes is large than when the number of eyes is small;

or

a number of “

skin-colored”

pixels in the image, wherein an image of the selected subgroup is more likely to be the selected image when the number of skin-colored pixels is large than when the number of skin-colored pixels is small.

22. The system of claim 21, wherein the visual similarity metric is based on a number of images in the selected subgroup.

23. The system of claim 21, wherein the visual similarity metric is based on at least one of a number of faces in an image, a size of a face in an image, a number of eyes in an image, a number of “

skin-colored”

pixels in an image, texture in an image, brightness in an image, and contrast of an image.

24. The system of claim 21 wherein an image of the selected subgroup is more likely to be the selected image when a region of interest is located in an off-center position in the image than when a region of interest is located in a central position in the image.

25. The system of claim 21 wherein the computer is further configured to add text, graphics, or an additional image to the selected image.

26. The system of claim 21, wherein the computer is further configured to perform at least one of resizing or filtering the selected image.

27. One or more non-transitory computer-readable storage media encoding computer-executable instructions for executing on a computer system a computer process, the computer process comprising:

obtaining a relevancy metric for a first targeted image of a collection of images, wherein the relevancy metric is computed based on an image characteristic selected from the group consisting of;

a size of a face in the first targeted image, wherein the relevancy metric is higher when the face is large than when the face is small,

a number of eyes in the first targeted image, wherein the relevancy metric is higher when the number of eyes is large than when the number of eyes is small, or

a number of skin-colored pixels in the first targeted image wherein the relevancy metric is higher when the number of skin-colored pixels is large than when the number of skin-colored pixels is small;

comparing with a computer the relevancy metric of the first targeted image with the relevancy metric of at least a second targeted image of the collection of images;

transmitting for presentation through a user interface the image having the higher relevancy metric.

28. One or more non-transitory computer-readable storage media encoding computer-executable instructions for executing on a computer system a computer process, the computer process comprising:

dividing a group of images into a plurality of subgroups, each subgroup comprising a plurality of subgroup images arranged in chronological order;

selecting one of the plurality of subgroups based on a visual similarity metric; and

selecting at least one image from the selected subgroup to be a selected image based on at least one image characteristic selected from the group consisting of;

a size of a face in the image, wherein an image of the selected subgroup is more likely to be the selected image when the size of the face is large than when the size of the face is small;

a number of eyes in the image, wherein an image of the selected subgroup is more likely to be the selected image when the number of eyes is large than when the number of eyes is small;

or

a number of “

skin-colored”

pixels in the image, wherein an image of the selected subgroup is more likely to be the selected image when the number of skin-colored pixels is large than when the number of skin-colored pixels is small.

29. A method comprising:

obtaining a computed metric for a first targeted image of a collection of images, wherein the computed metric is indicative of how visually appealing the targeted image is and is based on an image characteristic selected from the group consisting of;

a size of a face in the targeted image, wherein the metric is higher when the face is large than when the face is small;

a number of eyes in the targeted image, wherein the metric is higher when the number of eyes is large than when the number of eyes is small;

or a number of “

skin-colored”

pixels in the targeted image, wherein the metric is higher when the number of skin-colored pixels is large than when the number of skin-colored pixels is small;

comparing with a computer the metric of the first targeted image with the metric of at least a second targeted image of the collection of images;

transmitting for presentation through a user interface the image having the higher metric.