Video processing for human occupancy detection

• US 10,372,977 B2
• Filed: 07/09/2015
• Issued: 08/06/2019
• Est. Priority Date: 07/09/2015
• Status: Active Grant

• Current Assignee: Analog Devices International Unlimited Company (Analog Devices, Inc.)
• Original Assignee: Analog Devices, Inc.
• Inventors: Singh, Raka
• Primary Examiner(s): Bella, Matthew C
• Assistant Examiner(s): Shin, Brian D
• Application Number:
  US14/794,991
• Publication Number:
  US 20170011261A1
• Time in Patent Office: 1,489 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Many conventional video processing algorithms attempting to detect human presence in a video stream often generate false positives on non-human movements such as plants moving in the wind, rotating fan, etc. To reduce false positives, a technique exploiting temporal correlation of non-human movements can accurately detect human occupancy while reject non-human movements. Specifically, the technique involves performing temporal analysis on a time-series signal generated based on an accumulation of foreground maps and an accumulation of motion map and analyzing the running mean and the running variance of the time-series signal. By determining whether the time-series signal is correlated in time, the technique is able to distinguish human movements and non-human movements. Besides having superior accuracy, the technique lends itself to an efficient algorithm which can be implemented on low cost, low power digital signal processor or other suitable hardware.

Claims

1. A method for determining occupancy in a video stream, comprising:

processing a plurality of frames of the video stream, wherein for a given frame, processing comprises localizing one or more foreground areas in a foreground map and generating a motion map, wherein localizing the one or more foreground areas comprises assigning a given pixel to a foreground area if an integrated value of foreground pixel values for the given pixel over a moving window of foreground maps is greater than a threshold;

generating a time-series signal based on the motion map and the localized one or more foreground areas; and

determining whether there is occupancy in the video stream based on temporal analysis of the time-series signal, wherein the temporal determines whether peaks of the time-series signal are uncorrelated that indicates occupancy.

2. The method of claim 1, wherein processing the plurality of frames further comprises generating the foreground map having foreground pixel values identifying pixels of the given frame that are different from a background reference frame.

3. The method of claim 2, wherein generating the foreground map comprises:

if a difference between a pixel value of a given pixel of the given frame and a corresponding pixel value of the background reference frame is greater than a second threshold, assigning a first foreground pixel value to the given pixel; and

otherwise, assigning a second foreground pixel value to the given pixel.

4. The method of claim 1, wherein localizing the one or more foreground areas comprises rejecting pixels having episodic or periodic change in foreground pixel values over the moving window of foreground maps as not being part of the foreground area.

5. The method of claim 1, wherein:

localizing the one or more foreground areas in the foreground map comprises integrating on a per pixel basis, foreground pixel values in the foreground map over a moving window of foreground maps; and

the method further comprises integrating the foreground pixel values computing a weighted integration of the foreground pixel values for the given pixel over the moving window of foreground maps.

6. The method of claim 1, wherein processing the plurality of frames further comprises generating a difference map having difference pixel values identifying pixels of the given frame with pixel values different from corresponding pixel values of the pixels of a frame previous to the given frame.

7. The method of claim 6, wherein generating the difference map comprises:

if a difference between a pixel value of a given pixel of the given frame and a corresponding pixel value of the frame previous to the given frame is greater than a second threshold, assigning a first difference pixel value to the given pixel; and

otherwise, assigning a second difference pixel value to the given pixel.

8. The method of claim 1, wherein the temporal analysis comprises performing a running mean or a running variance of the time-series signal.

9. The method of claim 1, wherein performing temporal analysis comprises:

computing a running mean of the time-series signal; and

computing a finite difference of the running mean.

10. The method of claim 9, wherein performing temporal analysis comprises:

computing a plurality of average values, wherein each average value is computed based on a window of values of the finite difference.

11. The method of claim 10, wherein performing temporal analysis comprises:

if one or more ones of the plurality of average values in series is greater than a second threshold, determining occupancy; and

otherwise, determining no occupancy.

12. The method of claim 1, wherein performing temporal analysis comprises:

computing a running variance of the time-series signal; and

computing a finite difference of the running variance.

13. The method of claim 12, wherein performing temporal analysis comprises:

computing a plurality of average values, wherein each average value is computed based on a window of values of the finite difference.

14. The method of claim 13, wherein performing temporal analysis comprises:

if one or more ones of the plurality of average values in series is greater than a second threshold, determining occupancy; and

otherwise, determining no occupancy.

15. The method of claim 1, further comprising:

in response to determining occupancy, triggering one or more classifiers for determining human presence in the one or more foreground areas.

16. A computer-readable non-transitory medium comprising one or more instructions for determining occupancy in a video stream, that when executed on a processor, configure the processor to:

process a plurality of frames of the video stream, wherein for a given frame, processing comprises localizing one or more foreground areas in a foreground map and generating a motion map;

wherein;

localizing the one or more foreground areas in the foreground map comprises integrating, on a per pixel basis, foreground pixel values in the foreground map over a moving window of foreground maps, and

integrating the foreground pixel values comprises computing, using a first order infinite impulse response filter, an integrated value for a given pixel based on a weighted sum of a previous integrated value and a present foreground pixel value for the given pixel;

generate a time-series signal based on the localized one or more foreground areas and the motion map; and

determine whether there is occupancy in the video stream based on temporal analysis of the time-series signal, wherein the temporal analysis determines whether peaks of the time-series signal are uncorrelated that indicates occupancy.

17. The computer-readable non-transitory medium of claim 16, wherein the temporal analysis comprises performing a running mean or a running variance of the time-series signal.

18. A method for determining occupancy in a video stream, comprising:

processing a plurality of frames of the video stream, wherein for a given frame, processing comprises localizing one or more foreground areas in a foreground map and generating a motion map, wherein generating the motion map comprises accumulating, on a per pixel basis, difference pixel values over a moving window of difference maps, and summing motion pixel values of the motion map in the one or more foreground areas to generate a sample of a time-series signal;

generating the time-series signal based on a plurality of samples of the time-series signal generated by the processing of the plurality of the frames of the video stream; and

determining whether there is occupancy in the video stream based on temporal analysis of the time-series signal, wherein the temporal analysis determines whether peaks of the time-series signal are uncorrelated that indicates occupancy.

19. The method of claim 18, wherein generating the motion map further comprises:

if an accumulated value of difference pixel values for a given pixel over the moving window of difference maps is greater than a threshold, assigning a first motion pixel value to the given pixel; and

otherwise, assigning a second motion pixel value to the given pixel.

20. The method of claim 18, wherein accumulating difference pixel values further comprises computing a sum of difference pixel values for a given pixel over the moving window of difference maps.

21. The method of claim 18, wherein localizing the one or more foreground areas comprises assigning a given pixel to a foreground area if an integrated value of foreground pixel values for the given pixel over a moving window of foreground maps is greater than a threshold.

22. The method of claim 18, wherein:

localizing the one or more foreground areas in the foreground map comprises integrating, on a per pixel basis, foreground pixel values in the foreground map over a moving window of foreground maps; and

integrating the foreground pixel values comprises computing a weighted integration of the foreground pixel values for a given pixel over the moving window of foreground maps.

23. A computer-readable non-transitory medium comprising one or more instructions for determining occupancy in a video stream, that when executed on a processor, configure the processor to:

process a plurality of frames of the video stream, wherein for a given frame, processing comprises localizing one or more foreground areas in a foreground map and generating a motion map;

generate a time-series signal based on the localized one or more foreground areas and the motion map; and

determine whether there is occupancy in the video stream based on temporal analysis of the time-series signal, wherein the temporal analysis determines whether peaks of the time-series signal are uncorrelated that indicates occupancy.

24. The computer-readable non-transitory medium of claim 23, wherein generating the motion map comprises accumulating, on a per pixel basis, difference pixel values over a moving window of difference maps.

25. The computer-readable non-transitory medium of claim 23, wherein the temporal analysis comprises performing a running mean or a running variance of the time-series signal.