Temporal video streaming and summaries

• US 9,805,567 B2
• Filed: 09/14/2015
• Issued: 10/31/2017
• Est. Priority Date: 09/14/2015
• Status: Active Grant

• Current Assignee: Logitech Europe S A (Logitech International SA)
• Original Assignee: Logitech Europe S A (Logitech International SA)
• Inventors: Borel, Vincent, Standridge, Aaron, Nater, Fabian, Grabner, Helmut
• Primary Examiner(s): Shibru, Helen
• Application Number:
  US14/853,943
• Publication Number:
  US 20170076571A1
• Time in Patent Office: 778 Days
• Field of Search: 386226, 386248
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: G08B 13/19669 Event triggers storage or change of storage policy
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/437 Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server flow control in data networks H04L47/10; streaming protocols, e.g. RTP or RTCP, H04L65/65; scheduling or organising the servicing of application requests in data packet switching networks H04L67/60
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8153 : comprising still images, e.g. texture, background image
• CPC Class Codes: H04N 21/8549 : Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739
• CPC Class Codes: H04N 5/7605 : on discs or drums H04N5/781, H04N5/805, H04N5/83, H04N5/85 take precedence
• CPC Class Codes: H04N 5/77 : between a recording apparatus and a television camera

Abstract

In one embodiment of the present invention, a video camera selectively streams to a remote server. Still images or short video events are intermittently transmitted when there is no significant motion detected. When significant motion is detected, video is streamed to the remote server. The images and video can be higher resolution than the bandwidth used, by locally buffering the images and video, and transmitting it at a lower frame rate that extends to when there is no live streaming. This provides a time-delayed stream, but with more resolution at lower bandwidth.

Claims

1. A method of providing a video summary from a camera, comprising:

detecting motion using a processor in the camera;

determining, using the processor, whether the motion exceeds a first threshold;

during periods of inactivity wherein motion is less than the first threshold, recording in a memory of the camera only a periodic video of more than one frame but less than constant video recording;

identifying events from periods of activity having detected motion that exceeds the first threshold and creating event tags only for events with detected motion that exceeds the first threshold;

recording in the memory of the camera event video from the identified events and the event tags; and

intermittently transmitting both the periodic and event video in the memory to a remote computing device using a transmitter in the camera.

2. The method of claim 1 wherein the periodic video during periods of inactivity comprises a video of between 1-10 seconds.

3. The method of claim 1 further comprising transmitting the periodic and event video at a first resolution bandwidth and transmitting real-time live streaming at a lower resolution than the first resolution.

4. The method of claim 1 further comprising determining, by one of the processor in the camera and the remote computing device, the end of an event and the start of a new event based on the amount of time after movement stops.

5. The method of claim 1 further comprising determining, by one of the processor in the camera and the remote computing device, the end of an event and the start of a new event based on new motion in a different portion of an image view of the camera.

6. The method of claim 1 further comprising determining, by one of the processor in the camera and the remote computing device, one of the end of an event and the start of a new event based on a change in one of the number of moving objects in the video and the number of people in the video.

7. The method of claim 1 further comprising creating, with the remote computing device, a summary video from multiple video events provided by the camera, comprising:

creating a time lapse video having motion video events that exceed the first threshold and periodic video that do not exceed the first threshold over a period of time;

allocating a first amount of time, at a first time lapse speed, to the periodic video; and

allocating more time than said first amount of time, at a second time lapse speed less than the first time lapse speed, to videos with motion that exceeds the first threshold.

8. The method of claim 7 further comprising weighting the video events according to an assigned importance with different video events being assigned at least a first importance and a second importance, and providing one of a the first time lapse speed and the first amount of time to weighted video events having the first importance, and providing one of the second time lapse speed and a second amount of time to video events having the second importance.

9. The method of claim 8 wherein the video events have contextual tabs, and the weighting is based on at least one of the number of people detected, the identity of people detected, the duration of the motion and the amount of the motion.

10. The method of claim 7 further comprising providing more detailed video events, at a time lapse slower than the second time lapse speed, for portions of the summary video selected by a user.

11. The method of claim 7 further comprising weighting the video events based on:

an amount of inactivity before the video event;

the duration of motion in the video event;

the proximity of the motion in the video event to the center of the video event;

the amount of difference between the motion in the video event and motion from previous video events; and

the number of objects moving in the video event.

12. A method of providing a video summary from a camera, comprising:

detecting motion using a processor in the camera;

determining, using the processor, whether the motion exceeds a first threshold;

during periods of inactivity wherein motion is less than the first threshold, recording in a memory of the camera only a periodic video of more than one frame but less than constant video recording;

identifying events from periods of activity having detected motion that exceeds the first threshold and creating event tags only for events with detected motion that exceeds the first threshold;

recording in the memory of the camera event video from the identified events and the event tags;

intermittently transmitting the periodic and event video in the memory to a remote computing device using a transmitter in the camera;

creating, with the remote computing device, a summary video from multiple video events provided by the camera, comprising;

creating a time lapse video having motion video events that exceed the first threshold and periodic video that do not exceed the first threshold over a period of time;

allocating a first amount of time, at a first time lapse speed, to periodic video; and

allocating a second amount of time greater than the first amount of time, at a second time lapse speed less than the first time lapse speed, to videos with motion that exceeds the first threshold;

providing the video events with contextual tabs; and

weighting the video events based on at least one of the number of people detected, the identity of people detected, the duration of the motion and the amount of the motion.

13. A system for providing a video summary, comprising:

a camera having a processor configured to analyze pixels in video captured by the camera to detect motion in a video;

the processor being configured to determine whether the motion exceeds a first threshold;

a memory of the camera configured to record only a periodic video of more than one frame but less than constant video recording during periods of wherein motion is less than the first threshold;

the processor being configured to identify events from periods of activity having detected motion that exceeds the first threshold and create event tags only for events with detected motion that exceeds the first threshold;

the processor being further configured to record in the memory of the camera event video from the identified events and the event tags; and

a transmitter configured to intermittently transmit the periodic and event video in the memory to a remote computing device.

14. The system of claim 13 wherein the periodic video during periods of inactivity comprises a video of between 1-10 seconds.

15. The system of claim 13 wherein the processor is further configured to transmit the periodic and event video at a first resolution bandwidth and transmitting real-time live streaming at a lower resolution than the first resolution.

16. The system of claim 13 further comprising determining, by one of the processor in the camera and the remote computing device, the end of an event and the start of a new event based on the amount of time after movement stops.

17. The system of claim 13 further comprising one of the processor in the camera and the remote computing device being configured to determine the end of an event and the start of a new event based on new motion in a different portion of an image view of the camera in the video.

18. The system of claim 13 further one of the processor in the camera and the remote computing device being configured to determine one of the end of an event and the start of a new event based on a change in one of the number of moving objects in the video and the number of people in the video.

19. The system of claim 13 wherein the remote computing device is further configured to create a summary video from multiple video events provided by the camera, comprising:

creating a time lapse video having motion video events exceeding the first threshold and periodic video not exceeding the first threshold over a period of time;

allocating a first amount of time, at a first time lapse speed, to periodic video; and

allocating a second amount of time, greater than the first amount of time, at a second time lapse speed less than the first time lapse speed, to videos with motion exceeding the first threshold.

20. The system of claim 19 wherein the remote computing device is further configured to weight the video events according to an assigned importance with different video events being assigned at least a first importance and a second importance, and provide one of a the first time lapse speed and the first amount of time to weighted video events having the first importance, and providing one of the second time lapse speed and a second amount of time to video events having the second importance.