Method and apparatus for classifying video data

• US 10,499,005 B2
• Filed: 12/28/2017
• Issued: 12/03/2019
• Est. Priority Date: 09/16/2005
• Status: Active Grant

• Current Assignee: III Holdings 10, LLC (Intellectual Ventures LLC)
• Original Assignee: III Holdings 10, LLC (Intellectual Ventures LLC)
• Inventors: Burke, Trevor John
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): Morehead, III, John H
• Application Number:
  US15/856,738
• Publication Number:
  US 20180191995A1
• Time in Patent Office: 705 Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/9206 the additional signal being a character code signal
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A method of classifying video data representing activity within a space to be monitored. A method comprises storing video data obtained from a camera configured to monitor the space. Sensor data indicative of a condition occurring within the space is obtained, and a plurality of programme elements are defined within the video data. Each programme element has an associated classification code, and each classification code is selected using the sensor data.

Claims

1. A method comprising:

receiving, at a controller, video monitor data from a camera located in a monitored space;

receiving, at the controller, sensor data indicative of a condition occurring within the monitored space,

defining, at the controller, a plurality of elements within the video monitor data, wherein each of the plurality of elements includes an associated classification code selected by the controller using the received sensor data;

communicating, from the controller to a non-volatile storage device, instructions to store the video monitor data and associated classification codes, wherein the associated classification codes represent values on a non-binary monitored activity scale extending from a low activity value to a high activity value;

receiving, at the controller, a user-selected classification code for a first element in the stored video monitor data; and

sending instructions, from the controller, to display the first element in the stored video monitor data associated with the user-selected classification code.

2. The method of claim 1, wherein the sensor data is received from a passive infrared detector.

3. The method of claim 1, wherein the associated classification code selected comprises a class code, a number of sensor samples used to compute the class code, and a time point in the video monitor data at which the element begins.

4. The method of claim 1, wherein the instructions to store the video monitor data and associated classification codes include instructions to store only when received sensor data indicates high activity in the monitored space.

5. The method of claim 1, further comprising:

calculating an average level of activity for all elements of the plurality of elements; and

determining classification codes representing periods of activity that is at least a calculated average level of activity for elements within the monitored space.

6. The method of claim 1, further comprising receiving video monitor data from a plurality of video recording devices, wherein the classification code associated with each element identifies a specific video recording device of the plurality of video recording devices which was used to obtain the video data of the respective element.

7. The method of claim 6, further comprising generating a single stream of video data from the plurality of video recording devices using classification codes to determine which of the plurality of video recording devices to use for the single stream of video data.

8. The method of claim 1, further comprising:

receiving a user input representing a request to display elements representing periods of activity within the monitored space above a calculated average level of activity for elements within the monitored space;

determining at least one classification code associated with elements representing periods of activity within the monitored space above the calculated average level of activity for elements within the monitored space; and

displaying elements associated with the determined at least one classification code.

9. The method of claim 1, wherein the sensor data comprises motion data obtained from a motion sensor.

10. The method of claim 1, wherein the sensor data comprises audio data obtained from a sound sensor.

11. The method of claim 1, wherein the sensor data comprises pressure data obtained from a pressure sensor.

12. The method of claim 1, wherein the sensor data is obtained by the sensor device during a time period of capture of a respective element by the video monitor device.

13. The method of claim 1, further comprising ensuring, by the controller, the plurality of elements within the video monitor data have an equal duration of time.

14. The method of claim 1, further comprising:

receiving a user input representing a request to display all elements defined by more than two sensor activations within a predetermined time; and

displaying all elements defined by more than two sensor activations within the predetermined time.

15. The method of claim 1, further comprising deleting video monitor data based on determining at least one classification code associated with elements representing periods of activity within the monitored space below a calculated average level of activity for elements within the monitored space.

16. A system comprising:

a video data buffer;

a program memory that stores readable instructions; and

a controller that reads and executes instructions stored in the program memory, wherein the readable instructions comprise instructions to cause the controller to;

receive, at the video data buffer, video monitor data from a camera located in a monitored space;

receive sensor data indicative of a condition occurring within the monitored space;

define a plurality of elements within the received video monitor data, wherein each of the defined plurality of elements includes an associated classification code selected using the received sensor data;

communicate instructions to store the video monitor data and associated classification codes, wherein the associated classification codes represent values on a non-binary monitored activity scale extending from a low activity value to a high activity value;

receive a user-selected classification code for a first element in the stored video monitor data; and

send instructions to display the first element in the stored video monitor data associated with the user-selected classification code.

17. The system of claim 16, further comprising:

the sensor device;

a video recording device that captures the video data; and

a storage device that stores the video data.

18. The system of claim 17, wherein the sensor device comprises at least one of a motion sensor, a pressure sensor, and a sound sensor.

19. A non-transitory computer-readable memory storing computer program code to cause a computer to:

receive video monitor data from a camera located in a monitored space;

receive sensor data indicative of a condition occurring within the monitored space;

define a plurality of elements within the received video monitor data, wherein each of the defined plurality of elements includes an associated classification code selected using the received sensor data;

communicate instructions to store the video monitor data and associated classification codes, wherein the associated classification codes represent values on a non-binary monitored activity scale extending from a low activity value to a high activity value;

receive a user-selected classification code for a first element in the stored video monitor data; and

send instructions to display the first element in the stored video monitor data associated with the user-selected classification code.