Movement monitoring system

• US 10,482,613 B2
• Filed: 10/06/2017
• Issued: 11/19/2019
• Est. Priority Date: 07/06/2017
• Status: Active Grant

• Current Assignee: Wisconsin Alumni Research Foundation (University of Wisconsin System)
• Original Assignee: Wisconsin Alumni Research Foundation (University of Wisconsin System)
• Inventors: Radwin, Robert, Wang, Xuan, Hu, Yu Hen, DiFranco, Nicholas
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US15/727,283
• Publication Number:
  US 20190012794A1
• Time in Patent Office: 774 Days
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition

Abstract

A monitoring system may include an input port, an output port, and a controller in communication with the input port and the output port. The input port may receive video from an image capturing device. The image capturing device is optionally part of the monitoring system and in some cases includes at least part of the controller. The controller may be configured to receive video via the input port and identify a subject within frames of the video relative to a background within the frames. Further, the controller may be configured to identify dimensions and/or other parameters of the identified subject in frames of the video and determine when the subject is performing a predetermined task. Based on the dimensions and/or other parameters identified or extracted from the video during the predetermined task, the controller may output via the output port assessment information.

Claims

1. A monitoring system comprising:

an input port for receiving video;

an output port;

a controller in communication with the input port and the output port, the controller configured to;

identify a subject within a frame of video relative to a background within the frame;

determine when the subject in the video is performing a task;

identify a height dimension and a width dimension of the subject in one or more frames of the video during the task; and

output via the output port risk assessment information relative to the subject during the task based on the height dimension and the width dimension for the subject in one or more frames of the video during the task.

2. The monitoring system of claim 1, further comprising:

an image capturing device adapted to capture video of the subject and provide the video to the controller via the input port.

3. The monitoring system of claim 1, wherein the controller is configured to:

determine extreme-most pixels in two dimensions of the subject to identify the height dimension and the width dimension based on the identified subject; and

identify a posture of the subject based on the determined extreme-most pixels in two dimensions of the subject.

4. The monitoring system of claim 1, wherein the controller is configured to:

apply a bounding box around the subject; and

identify a posture of the subject based on a dimension of the bounding box.

5. The monitoring system of claim 1, wherein the controller is configured to:

identify when the task begins based on determining a frame of video in which a first ghost effect first appears; and

identify when the task ends based on determining a frame of video in which a second ghost effect first appears.

6. The monitoring system of claim 5, wherein the controller is configured to:

identify a location of hands of the subject at the beginning of the task based on a location in the frame of the first ghost effect; and

identify a location of hands of the subject at the ending of the task based on a location in the frame of the second ghost effect.

7. The monitoring system of claim 1, wherein the controller is adapted to:

identify a location of feet of the subject in a frame of the video;

identify a location of hands of the subject in the frame of the video; and

develop the risk assessment information by determining a recommended weight limit for an object moved during the task based on a vertical distance and a horizontal distance between the location of the feet and the location of the hands.

8. A computer readable medium having stored thereon in a non-transitory state a program code for use by a computing device, the program code causing the computing device to execute a method for analyzing movement comprising:

identifying a subject within a frame of video relative to a background within the frame;

determining when the identified subject performs a task in the video

identifying dimensions based on the identified subject in one or more frames of the video during the task; and

outputting risk assessment information relative to the identified subject based on the identified dimensions for the identified subject in the one or more frames of the video during the task.

9. The computer readable medium of claim 8, wherein identifying dimensions based on the identified subject includes:

identifying a difference between an extreme-most pixel of the identified subject in a positive y-dimension and an extreme-most pixel of the identified subject in a negative y-dimension; and

includes identifying a difference between an extreme-most pixel of the identified subject in a positive x-dimension and an extreme-most pixel of the identified subject in a negative x-dimension.

10. The computer readable medium of claim 8, wherein the method the program code causes the computing device to execute further comprises:

applying a bounding box to the identified subject; and

wherein identifying dimensions based on the identified subject includes identifying a height of the bounding box and a width of the bounding box.

11. The computer readable medium of claim 8, wherein the method the program code causes the computing device to execute further comprises:

determining a posture of the identified subject based on the identified dimensions of the identified subject.

12. The computer readable medium of claim 8, wherein determining when the identified subject performs the task in the video includes determining a beginning of the task by identifying a frame in which a ghost effect first appears.

13. The computer readable medium of claim 8, wherein determining when the identified subject performs the task in the video includes determining an end of the task by identifying a frame in which a ghost effect first appears.

14. The computer readable medium of claim 8, wherein the method the program code causes the computing device to execute further comprises:

identifying a ghost effect in a frame of the video, the ghost effect having a location in the frame; and

identifying a location of a hand of the identified subject based on the location of the ghost effect.

15. The computer readable medium of claim 14, wherein the method the program code causes the computing device to execute further comprises:

identifying a location of feet of the identified subject in the frame of the video; and

wherein a vertical distance and a horizontal distance between the location of the hand of the identified subject and the feet of the identified subject are used in determining the risk assessment information.

16. The computer readable medium of claim 8, wherein:

the identifying the subject includes comparing pixels in successive frames in the video; and

the method the program code causes the computing device to execute further comprises determining a pixel is a pixel of the identified subject by comparing a value assigned to the pixel to a mixture of pixel values from a Gaussian distribution background model for the pixel.

17. A monitoring system comprising:

an input port for receiving video;

an output port; and

a controller in communication with the input port and the output port, the controller configured to identify a subject in frames of video received via the input port, identify at least one ghost effect in the frames of the video, and determine a parameter of the subject based on a location in a frame of video of a ghost effect of the identified at least one ghost effect.

18. The monitoring system of claim 17, wherein the controller is configured to identify one of a beginning of a task involving the subject and an ending of the task involving the subject based on an initial frame of a set of frames in which a ghost effect appears.

19. The monitoring system of claim 17, wherein the ghost effect is identified by locating a plurality of foreground pixels relative to background pixels in a frame of the video, where the plurality of foreground pixels were not present in a reference frame of the video.

20. The monitoring system of claim 17, wherein the controller is configured to determine a location of a hand of the subject in the frame of the video based on the location of the ghost effect in the frame of the video.

21. A monitoring system comprising:

an input port for receiving video;

an output port;

a controller in communication with the input port and the output port, the controller configured to;

identify a subject within a frame of video relative to a background within the frame;

determine when the subject in the video is performing a task;

determine extreme-most pixels in two dimensions of the subject to identify a height dimension and a width dimension of the subject in one or more frames of the video during the task; and

output via the output port position assessment information relative to the subject during the task based on the height dimension and the width dimension for the subject in one or more frames of the video during the task.

22. A monitoring system comprising:

an input port for receiving video;

an output port;

a controller in communication with the input port and the output port, the controller configured to;

identify a subject within a frame of video relative to a background within the frame;

determine when the subject in the video is performing a task;

identify when the task begins based on determining a frame of video in which a first ghost effect first appears;

identify when the task ends based on determining a frame of video in which a second ghost effect first appears;

identify a height dimension and a width dimension of the subject in one or more frames of the video during the task; and

output via the output port position assessment information relative to the subject during the task based on the height dimension and the width dimension for the subject in one or more frames of the video during the task.

23. A monitoring system comprising:

an input port for receiving video;

an output port;

a controller in communication with the input port and the output port, the controller configured to;

identify a subject within a frame of video relative to a background within the frame;

determine when the subject in the video is performing a task;

identify a height dimension and a width dimension of the subject in one or more frames of the video during the task;

identify a location of feet of the subject in a frame of the video

develop position assessment information relative to the subject during the task based, at least in part, on

     1) the height dimension and the width dimension for the subject in one or more frames of the video during the task, and

     2) determining a recommended weight limit for an object moved during the task based on a vertical distance and a horizontal distance between a location of feet of the subject in one or more frames of the video during the task and a location of hands of the subject in the one or more frames of the video during the task; and

output via the output port the position assessment information.

24. A computer readable medium having stored thereon in a non-transitory state a program code for use by a computing device, the program code causing the computing device to execute a method for analyzing movement comprising:

identifying a subject within a frame of video relative to a background within the frame;

determining when the identified subject performs a task in the video identifying dimensions based on the identified subject in one or more frames of the video during the task;

identifying a ghost effect in a frame of the video, the ghost effect having a location in the frame;

identifying a location of a hand of the identified subject based on the location of the ghost effect;

identifying a location of feet of the identified subject in the frame of the video; and

determining position assessment information based, at least in part, on

     1) the identified dimensions for the identified subject in the one or more frames of the video during the task, and

     2) on a vertical distance and a horizontal distance between the identified location of the hand of the identified subject and the identified location of the feet of the identified subject;

outputting the position assessment information.

25. A monitoring system comprising:

an input port for receiving video;

an output port; and

a controller in communication with the input port and the output port, the controller configured to identify a subject in frames of video received via the input port, identify at least one ghost effect in the frames of the video, determine a parameter of the subject based on the identified at least one ghost effect, and identify one of a beginning of a task involving the subject and an ending of the task involving the subject based on an initial frame of a set of frames in which a ghost effect of the at least one ghost effect appears.