Temporal occlusion costing applied to video editing

• US 8,295,683 B2
• Filed: 04/23/2007
• Issued: 10/23/2012
• Est. Priority Date: 04/23/2007
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Widdowson, Simon
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): Khalid, Omer
• Application Number:
  US11/789,201
• Publication Number:
  US 20080260347A1
• Time in Patent Office: 2,010 Days
• Field of Search: 348/445, 348448-452, 348/556, 715/723
• US Class Current: 386/278
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence

Abstract

In a method for determining a temporal occlusion cost for a video crop, video image saliency data for a video is received. A video crop is received for a presenting an edited version of the video within a viewing region. The video image saliency data is used to determine a temporal occlusion cost for the video crop.

Claims

1. A method for determining a temporal occlusion cost for a video crop, said method comprising:

receiving, at a computer system, video image saliency data for a video;

receiving, at said computer system, a video crop for a presenting an edited version of said video within a viewing region; and

utilizing, at said computer system, said video image saliency data to determine a temporal occlusion cost for said video crop, wherein said temporal occlusion cost comprises determining a value which represents said temporal occlusion cost, wherein said value varies in accordance with video running time removed by said video crop.

2. The method as recited in claim 1, wherein said receiving video image saliency data for a video comprises:

receiving a video image saliency map for said video.

3. The method as recited in claim 2, wherein said receiving a video image saliency map for said video comprises:

receiving a plurality of grayscale saliency maps, each of said grayscale saliency maps providing image saliency information for a frame of said video.

4. The method as recited in claim 1, wherein said receiving video image saliency data for a video comprises:

utilizing a user-specified saliency parameter to control generation of said video image saliency data; and

receiving said video image saliency data, wherein said video image saliency data is generated according to said user-specified image saliency parameter.

5. The method as recited in claim 1, wherein said receiving a video crop for a presenting an edited version of said video within a viewing region comprises:

utilizing a user-specified video cropping constraint to control generation of said video crop; and

receiving said video crop, wherein said video crop is generated according to said user-specified video cropping constraint.

6. The method as recited in claim 1, wherein said receiving a video crop for a presenting an edited version of said video within a viewing region comprises:

receiving video crop comprising an aspect ratio conversion of said video.

7. The method as recited in claim 1, wherein said receiving a video crop for a presenting an edited version of said video within a viewing region comprises:

receiving a cuboidal volume of voxels defined within a three-dimensional space, said three-dimensional space defined by a height and a width of a frame of said video along with a running time of said video.

8. The method as recited in claim 7, wherein said receiving a cuboidal volume of voxels defined within a three-dimensional space comprises:

receiving said cuboidal volume of voxels, wherein cross-sectional areas of said cuboidal volume remain fixed over time.

9. The method as recited in claim 7, wherein said receiving a cuboidal volume of voxels defined within a three-dimensional space comprises:

receiving said cuboidal volume of voxels, wherein cross-sectional areas of said cuboidal volume vary over time.

10. The method as recited in claim 1, wherein said determining said value which represents said temporal occlusion cost, wherein said value varies in accordance with saliency eliminated by said video crop and video frame area eliminated by said video crop.

11. A non-transitory computer-readable medium having computer-executable instructions stored thereon for causing a computer system to execute a method for cropping a video, said method comprising:

receiving video image saliency data for a video;

receiving a plurality of video crops, each of said plurality of video crops comprising a possible presentation of an edited version of said video within a viewing region;

utilizing said video image saliency data to determine a plurality of temporal occlusion costs, each of said temporal occlusion costs associated with one of said plurality of possible video crops, wherein each of said temporal occlusion costs comprise determining values for temporal occlusion costs associated with a plurality of said video crops, wherein a value of said values varies in accordance with video running time eliminated by said video crop;

determining a video crop associated with a lowest temporal occlusion cost of said plurality of temporal occlusion costs; and

cropping said video in accordance with said video crop associated with said lowest temporal occlusion cost.

12. The non-transitory computer readable medium of claim 11, wherein said receiving video image saliency data for a video to comprises:

receiving a video image saliency map for said video.

13. The non-transitory computer readable medium claim 12, wherein said receiving a video image saliency map for said video comprises:

receiving a plurality of grayscale saliency maps, each of said grayscale saliency maps providing image saliency information for a frame of said video.

14. The non-transitory computer readable medium claim 11, wherein said receiving video image saliency data for a video further comprises:

utilizing a user-specified saliency parameter to control generation of said video image saliency data; and

receiving said video image saliency data, wherein said video image saliency data is generated according to said user-specified image saliency parameter.

15. The non-transitory computer readable medium claim 11, wherein said receiving a plurality of video crops further comprises:

utilizing a user-specified video cropping constraint to control generation of said video crops; and

receiving said video crops, wherein said video crops are generated according to said user-specified video cropping constraint.

16. The non-transitory computer readable medium claim 11, wherein said receiving a plurality of video crops comprises:

receiving a cuboidal volume of voxels defined within a three-dimensional space, said three-dimensional space defined by a height and a width of a frame of said video along with a running time of said video, wherein said cuboidal volume of voxels represents one of said plurality of video crops.

17. The non-transitory computer readable medium claim 11, wherein said determining said values for said temporal occlusion costs associated with said plurality of said video crops, wherein a value of said values varies in accordance with saliency eliminated by said video crop and video frame area eliminated by said video crop.

18. A system for determining a temporal occlusion cost for a video crop, said system comprising:

an video image saliency receiver configured for receiving video image saliency data associated with a video;

a video crop receiver configured for receiving a video crop to said video; and

a temporal occlusion cost generator coupled to said video image saliency receiver and said video crop receiver, said temporal occlusion cost generator configured for utilizing said video image saliency data to determine a temporal occlusion cost associated with said video crop.

19. The system of claim 18 further comprising:

a cost minimization engine coupled to said temporal occlusion cost generator, said cost minimization engine configured for selecting a minimized temporal occlusion cost from a plurality of temporal occlusion costs associated with a plurality of video crops evaluated by said temporal occlusion cost generator.

20. The system of claim 18 further comprising:

a video image saliency generator controller coupled to said video image saliency receiver, wherein said video image saliency generator controller incorporates a user-specifiable image saliency parameter for controlling generation of said video image saliency data; and

a video crop generator controller coupled to said video crop receiver, wherein said video crop generator controller incorporates a user-specifiable video cropping constraint for controlling generation of said video crop.