Video ingestion and clip creation

• US 10,230,866 B1
• Filed: 09/30/2015
• Issued: 03/12/2019
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Townsend, Matthew Alan, Oks, Eduard, Mysore Vijaya Kumar, Rohith, Chaudhri, Apoorv, Rao, Yadunandana Nagaraja, Tyagi, Ambrish
• Primary Examiner(s): Sosanya, Obafemi O
• Application Number:
  US14/870,653
• Time in Patent Office: 1,259 Days
• CPC Class Codes: H04N 1/2108 for one picture only

Abstract

Devices, systems and methods are disclosed for reducing a perceived latency associated with uploading and annotating video data. For example, video data may be divided into video sections that are uploaded individually so that the video sections may be annotated as they are received. This reduces a latency associated with the annotation process, as a portion of the video data is annotated before an entirety of the video data is uploaded. In addition, the annotation data may be used to generate a master clip table and extract individual video clips from the video data.

Claims

1. A computer-implemented method for creating video clip selections from video data, comprising:

receiving, from an image capture device, first video data;

determining that a first face is represented in the first video data at a pixel location and at a timestamp;

receiving, from the image capture device, second video data;

determining that the first face is represented in the second video data;

based at least in part on determining that the first face is represented in the first video data and determining that the first face is represented in the second video data, combining sequentially the first video data with the second video data;

generating third video data corresponding to the first video data and the second video data; and

determining a first video clip from the second video data using.

2. The computer-implemented method of claim 1, wherein generating the second video data further comprises:

determining a first portion of the first video data having a width of 2600 pixels and a height of 1080 pixels, the first portion extending from (0,

     0) pixel coordinates to (2600,

     1080) pixel coordinates;

determining a second portion of the first video data, the second portion having a width of 2600 pixels and a height of 1080 pixels and extending from (0,

     1116) pixel coordinates to (2600,

     2196) pixel coordinates;

separating the first portion from the second portion; and

generating the second video data, the first portion extending from (0,

     0) pixel coordinates to (2600,

     1080) pixel coordinates and the second portion extending from (2601,

     0) pixel coordinates to (5200,

     1080) pixel coordinates.

3. The computer-implemented method of claim 1, further comprising:

determining the first face represented in the first video clip;

determining the timestamp associated with the first face, the timestamp indicating a time within the second video data;

determining the pixel location within the first video clip, the pixel location associated with the first face at the timestamp;

determining a second video clip from the second video data;

determining a second face represented in the second video clip;

determining a second timestamp associated with the second face, the second timestamp indicating a time within the second video data;

determining a second pixel location within the second video clip, the second pixel location associated with the second face at the second timestamp; and

generating a data structure, the data structure including a list of the first video clip and the second video clip along with the first face, the timestamp, the pixel location, the second face, the second timestamp and the second pixel location.

4. The computer-implemented method of claim 1, further comprising:

adding a first event to a queue upon receiving the second video data, the second video data;

removing the first event from the queue upon determining that the first face is represented in the second video data;

determining that the queue is empty; and

based on determining that the queue is empty, generating a data structure comprising a list including the first video clip.

5. A computer-implemented method, comprising:

receiving, during a first time period, first video data having a first resolution, the first video data including a first portion captured by a first image-capture device facing a first direction and a second portion captured by a second image-capture device facing a second direction;

determining at least one characteristic represented in the first portion;

receiving, during a second time period after the first time period, second video data including a third portion captured by the first image-capture device and a fourth portion captured by the second image-capture device;

determining that the at least one characteristic is represented in the fourth portion; and

generating third video data having a second resolution different than the first resolution, the third video data including the first portion and the fourth portion.

6. The computer-implemented method of claim 5, further comprising:

receiving a first request for a first uniform resource identifier corresponding to the first portion of the first video data;

receiving a second request for a second uniform resource identifier corresponding to the second portion of the first video data;

receiving, using the first uniform resource identifier, the first portion of the first video data; and

receiving, using the second uniform resource identifier, the second portion of the first video data.

7. The computer-implemented method of claim 5, wherein combining the first portion and the fourth portion further comprises:

determining a first time associated with an end of the first portion;

determining a second time associated with a beginning of the fourth portion, the second time being after the first time;

determining that a difference between the first time and the second time is within a threshold; and

combining the first portion and the fourth portion,

wherein generating the third video data further comprises separating the first portion from the second portion and separating the third portion from the fourth portion such that a horizontal resolution of the third video data is twice the horizontal resolution of the first video data.

8. The computer-implemented method of claim 7, further comprising:

generating fourth video data using the second video data, the fourth video data having a third resolution different than the second resolution; and

sending the fourth video data to a remote device.

9. The computer-implemented method of claim 5, further comprising:

determining a first priority metric associated with a first video frame of the first portion, the first priority metric corresponding to a measure of interest in the first video frame;

determining a second priority metric associated with a second video frame of the fourth portion;

determining that the first priority metric exceeds a first threshold;

determining that the second priority metric exceeds a second threshold; and

generating a video from the third video data, the video including the first video frame and the second video frame.

10. The computer-implemented method of claim 5, further comprising:

determining first characteristics associated with the third video data;

determining a first timestamp associated with the first characteristics;

determining a first pixel location within the third video data, the first pixel location associated with the first characteristics at the first timestamp;

determining a video from the third video data;

determining second characteristics associated with the video;

determining a second timestamp associated with the second characteristics;

determining a second pixel location within the video, the second pixel location associated with the second characteristics at the second timestamp; and

generating a data structure, the data structure including a list of the third video data and the video along with the first characteristics, the first timestamp, the first pixel location, the second characteristics, the second timestamp, and the second pixel location.

11. The computer-implemented method of claim 5, further comprising:

adding a first event to a queue upon receiving the third video data;

removing the first event from the queue upon determining that the at least one characteristic is represented in the fourth portion;

determining that the fourth portion corresponds to an end of the third video data;

determining that the queue is empty; and

generating a data structure comprising a list including the first video data.

12. The computer-implemented method of claim 5, further comprising:

identifying one or more faces represented in a first video frame of the first portion;

identifying one or more objects represented in the first video frame;

determining a location associated with the first video frame;

determining motion data associated with the first video frame;

determining a scene associated with the first video frame;

determining an existence of speech in audio data associated with the first video frame; and

associating the one or more faces, the one or more objects, the location, the motion data, the scene and the existence of speech with the first video frame.

13. A system, comprising:

at least one processor;

at least one memory including instructions that, when executed by the at least one processor, configure the system to perform;

receiving, during a first time period, first video data having a first resolution, the first video data including a first portion captured by a first image-capture device facing a first direction and a second portion captured by a second image-capture device facing a second direction;

determining at least one characteristic represented in the first portion;

receiving, during a second time period after the first time period, second video data including a third portion captured by the first image-capture device and a fourth portion captured by the second image-capture device;

determining that the at least one characteristic is represented in the fourth portion; and

generating third video data having a second resolution different than the first resolution, the third video data including the first portion and the second portion.

14. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further configure the system for:

receiving a first request for a first uniform resource identifier corresponding to the first portion of the first video data;

receiving a second request for a second uniform resource identifier corresponding to the second portion of the first video data;

receiving, using the first uniform resource identifier, the first portion of the first video data; and

receiving, using the second uniform resource identifier, the second portion of the first video data.

15. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further configure the system for:

determining a first time associated with an end of the first portion;

determining a second time associated with a beginning of the fourth portion, the second time being after the first time;

determining that a difference between the first time and the second time is within a threshold;

combining the first portion and the fourth portion,

wherein generating the third video data further comprises separating the first portion from the second portion and separating the third portion from the fourth portion such that a horizontal second resolution of the third video data is twice the horizontal resolution of the first video data.

16. The system of claim 15, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further configure the system for:

generating fourth video data using the second video data, the fourth video data having a third resolution different than the second resolution; and

sending the fourth video data to a remote device.

17. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further configure the system for:

determining a first priority metric associated with a first video frame of the first portion, the first priority metric corresponding to a measure of interest in the first video frame;

determining a second priority metric associated with a second video frame of the fourth portion;

determining that the first priority metric exceeds a first threshold;

determining that the second priority metric exceeds a second threshold; and

generating a video from the third video data, the video including the first video frame and the second video frame.

18. The system of claim 13, wherein at least one memory further includes the instructions that, when executed by the at least one processor, further configure the system for:

determining first characteristics associated with the third video data;

determining a first timestamp associated with the first characteristics;

determining a first pixel location within the third video data, the first pixel location associated with the first characteristics at the first timestamp;

determining a video from the third video data;

determining second characteristics associated with the video;

determining a second timestamp associated with the second characteristics;

determining a second pixel location within the video, the second pixel location associated with the second characteristics at the second timestamp; and

generating a data structure, the data structure including a list of the third video data and the video along with the first characteristics, the first timestamp, the first pixel location, the second characteristics, the second timestamp, and the second pixel location.

19. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further configure the system for:

adding a first event to a queue upon receiving the third video data;

removing the first event from the queue upon determining that the at least one characteristic is represented in the fourth portion;

determining that the fourth portion is an end of the third video data;

determining that the queue is empty; and

generating a data structure including the first video data.

20. The system of claim 13, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further configure the system for:

identifying one or more faces represented in a first video frame of the first portion;

identifying one or more objects represented in the first video frame;

determining a location associated with the first video frame;

determining motion data associated with the first video frame;

determining a scene associated with the first video frame;

determining an existence of speech in audio data associated with the first video frame; and

associating the one or more faces, the one or more objects, the location, the motion data, the scene and the existence of speech with the first video frame.