Content-based zooming and panning for video curation
First Claim
1. A computer-implemented method of simulating panning and zooming in video data, the method comprising:
- receiving panoramic video data comprising video frames having a first aspect ratio, the panoramic video data showing a plurality of directional views of a scene;
identifying an object of interest represented in the panoramic video data;
identifying an object within a first number of pixels of the object of interest in the first video frame;
determining a beginning of an event of interest involving the object, the beginning corresponding to a first video frame of the panoramic video data, the first video frame showing a first directional view of the plurality of directional views;
determining that the object does not move during the event of interest;
identifying a person of interest within the first number of pixels of the object of interest in the first video frame;
determining first pixel coordinates associated with the object in the first video frame, the first pixel coordinates including the object of interest and the person of interest;
determining a first cropped window from the first video frame, the first cropped window comprising a portion of the first video frame including the first pixel coordinates, the first cropped window having a second aspect ratio less than the first aspect ratio and the first cropped window having a first size and a first position within the first video frame;
determining an end of the event in a second video frame of the panoramic video data, the second video frame subsequent to the first video frame and the second video frame showing a second directional view of the plurality of directional views;
determining second pixel coordinates associated with the object in the second video frame;
determining a second cropped window from the second video frame, the second cropped window comprising a portion of the second video frame including the second pixel coordinates and the second cropped window having a second size and a second position within the second video frame; and
determining output video data including the first cropped window and the second cropped window.
1 Assignment
0 Petitions
Accused Products
Abstract
Devices, systems and methods are disclosed for identifying content in video data and creating content-based zooming and panning effects to emphasize the content. Contents may be detected and analyzed in the video data using computer vision, machine learning algorithms or specified through a user interface. Panning and zooming controls may be associated with the contents, panning or zooming based on a location and size of content within the video data. The device may determine a number of pixels associated with content and may frame the content to be a certain percentage of the edited video data, such as a close-up shot where a subject is displayed as 50% of the viewing frame. The device may identify an event of interest, may determine multiple frames associated with the event of interest and may pan and zoom between the multiple frames based on a size/location of the content within the multiple frames.
-
Citations
20 Claims
-
1. A computer-implemented method of simulating panning and zooming in video data, the method comprising:
-
receiving panoramic video data comprising video frames having a first aspect ratio, the panoramic video data showing a plurality of directional views of a scene; identifying an object of interest represented in the panoramic video data; identifying an object within a first number of pixels of the object of interest in the first video frame; determining a beginning of an event of interest involving the object, the beginning corresponding to a first video frame of the panoramic video data, the first video frame showing a first directional view of the plurality of directional views; determining that the object does not move during the event of interest; identifying a person of interest within the first number of pixels of the object of interest in the first video frame; determining first pixel coordinates associated with the object in the first video frame, the first pixel coordinates including the object of interest and the person of interest; determining a first cropped window from the first video frame, the first cropped window comprising a portion of the first video frame including the first pixel coordinates, the first cropped window having a second aspect ratio less than the first aspect ratio and the first cropped window having a first size and a first position within the first video frame; determining an end of the event in a second video frame of the panoramic video data, the second video frame subsequent to the first video frame and the second video frame showing a second directional view of the plurality of directional views; determining second pixel coordinates associated with the object in the second video frame; determining a second cropped window from the second video frame, the second cropped window comprising a portion of the second video frame including the second pixel coordinates and the second cropped window having a second size and a second position within the second video frame; and determining output video data including the first cropped window and the second cropped window. - View Dependent Claims (2, 3, 4)
-
-
5. A computer-implemented method comprising:
-
receiving input video data comprising video frames; identifying a first person represented in the video data; identifying a second person represented in the video data; determining, at a first time, that a first number of pixels between the first person and the second person in the video data exceeds a threshold; determining, at a second time following the first time, that a second number of pixels between the first person and the second person in the video data is less than the threshold, wherein the second time is associated with a beginning of an event of interest; determining first pixel coordinates in a first video frame associated with the beginning of the event; determining a first cropped window from the first video frame, the first cropped window comprising a portion of the first video frame including the first pixel coordinates; determining an end of the event in a second video frame of the video data; determining second pixel coordinates in the second video frame associated with the end of the event, the second pixel coordinates different than the first pixel coordinates; determining a second cropped window from the second video frame, the second cropped window comprising a portion of the second video frame including the second pixel coordinates; and determining output data corresponding to the first cropped window and the second cropped window. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20)
-
-
13. A system, comprising:
-
at least one processor; a memory including instructions that, when executed by the at least one processor, cause the system to perform a set of actions comprising; receiving input video data comprising video frames; identifying a first person represented in the video data; identifying a second person represented in the video data; determining, at a first time, that a first number of pixels between the first person and the second person in the video data exceeds a threshold; determining, at a second time following the first time, that a second number of pixels between the first person and the second person in the video data is less than the threshold, wherein the second time is associated with a beginning of an event of interest; determining first pixel coordinates in a first video frame associated with the beginning of the event; determining a first cropped window from the first video frame, the first cropped window comprising a portion of the first video frame including the first pixel coordinates; determining an end of the event in a second video frame of the video data; determining second pixel coordinates in the second video frame associated with the end of the event, the second pixel coordinates different than the first pixel coordinates; determining a second cropped window from the second video frame, the second cropped window comprising a portion of the second video frame including the second pixel coordinates; and determining output data corresponding to the first cropped window and the second cropped window.
-
Specification