VISION BASED MONITORING SYSTEM FOR ACTIVITY SEQUENCY VALIDATION

• US 20150294143A1
• Filed: 04/10/2014
• Published: 10/15/2015
• Est. Priority Date: 04/10/2014
• Status: Abandoned Application

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Wells, James W., Medasani, Swarup, Owechko, Yuri, Kim, Kyungnam, Wampler, Charles W. II, Scheuerman, Robert J.
• Application Number:
  US14/249,384
• Publication Number:
  US 20150294143A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A human monitoring system includes a plurality of cameras and a visual processor. The plurality of cameras are disposed about a workspace area, where each camera is configured to capture a video feed that includes a plurality of image frames, and the plurality of image frames are time-synchronized between the respective cameras. The visual processor is configured to identify the presence of a human within the workspace area from the plurality of image frames, generate a motion track of the human within the workspace area, generate an activity log of one or more activities performed by the human throughout the motion track, and compare the motion track and activity log to an activity template that defines a plurality of required actions. The processor then provides an alert if one or more actions within the activity template are not performed within the workspace area.

Claims

1. A human monitoring system for monitoring a workspace area, the system comprising:

a plurality of cameras disposed about the workspace area, each camera configured to capture a video feed that includes a plurality of image frames;

a visual processor configured to;

identify the presence of a human within the workspace area from the plurality of image frames;

generate a motion track of the human within the workspace area, the motion tract representing the position of the human across a period of time;

generate an activity log of one or more activities performed by the human throughout the motion track;

compare the motion track and activity log to an activity template that defines a plurality of required actions; and

provide an alert if one or more actions within the activity template are not performed within the workspace area.

2. The system of claim 1, wherein the visual processor is configured to generate an activity log of the one or more activities performed by the human throughout the motion track by pattern matching a detected representation of the human to a trained database of poses.

3. The system of claim 2, wherein the pattern matching includes using at least one of a support vector machine and a neural network.

4. The system of claim 2, wherein the pattern matching can further detect the pose of the human, and wherein the pose includes at least one or standing, walking, reaching, and crouching.

5. The system of claim 1, wherein the alert includes a video synopsis of the generated motion track.

6. The system of claim 1, wherein the visual processor is configured to identify the presence of a human within the workspace area by:

detecting the human within each of the plurality of image frames, wherein each of the plurality of image frames are obtained from a different camera selected from the plurality of cameras;

mapping a representation of the detected human from the plurality of views into a common coordinate system; and

determining a point of intersection of the mapped representations.

7. The system of claim 1, wherein the visual processor is further configured to receive an actuation signal from a tool that is indicative of the tool being used;

and

wherein the visual processor is configured to use the actuation signal to confirm the performance of an activity of the one or more activities performed by the human.

8. The system of claim 1, wherein each of the plurality of required actions is specified at a location within the workspace area;

and

wherein the visual processor is configured to provide an alert if the one or more actions within the activity template are not performed within the workspace area at its respectively specified location.

10. The system of claim 1, wherein the pattern matching includes using at least one of a support vector machine and a neural network.

11. The system of claim 1, wherein the pattern matching can further detect the pose of the human, and wherein the pose includes at least one or standing, walking, reaching, and crouching.

9. A human monitoring system for monitoring a workspace area, the system comprising:

a plurality of cameras disposed about the workspace area, each camera configured to capture a video feed that includes a plurality of image frames;

a visual processor configured to;

identify the presence of a human within the workspace area from the plurality of image frames;

generate a motion track of the human within the workspace area, the motion tract representing the position of the human across a period of time;

generate an activity log of one or more activities performed by the human throughout the motion track by pattern matching a detected representation of the human to a database of poses;

compare the motion track and activity log to an activity template that defines a plurality of required actions, wherein each of the plurality of required actions is specified at a location within the workspace area; and

provide an alert if one or more actions within the activity template are not performed within the workspace area at its respectively specified area.

12. The system of claim 9, wherein the alert includes a video synopsis of the generated motion track.

13. The system of claim 9, wherein the visual processor is configured to identify the presence of a human within the workspace area by:

detecting the human within each of the plurality of image frames, wherein each of the plurality of image frames are obtained from a different camera selected from the plurality of cameras;

mapping a representation of the detected human from the plurality of views into a common coordinate system; and

determining a point of intersection of the mapped representations.

14. The system of claim 9, wherein the visual processor is further configured to receive an actuation signal from a tool that is indicative of the tool being used;

and

wherein the visual processor is configured to use the actuation signal to confirm the performance of an activity of the one or more activities performed by the human.