Automatically generating labor standards from video data

• US 9,734,464 B2
• Filed: 09/11/2007
• Issued: 08/15/2017
• Est. Priority Date: 09/11/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Angell, Robert Lee, Kraemer, James R.
• Primary Examiner(s): Nguyen, Kira
• Application Number:
  US11/853,491
• Publication Number:
  US 20090070163A1
• Time in Patent Office: 3,626 Days
• Field of Search: 705 711
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work

Abstract

A computer implemented method, apparatus, and computer usable program product for creating a labor standard for a task. The process automatically detects event data derived from a continuous video stream, wherein the event data comprises metadata describing a sequence of motions for performing the task, and parses the event data to identify appropriate event data describing a discrete set of motions from the sequence of motions. The process then analyzes, using an analysis server, the discrete set of motions to form a labor standard, wherein the labor standard specifies an optimal manner of performing the discrete set of motions. Thereafter, the process generates a set of recommendations for performing the task efficiently according to the labor standard.

Claims

1. A computer implemented method for managing a task, the computer implemented method comprising:

generating, by multiple audio and video capture devices, continuous audio and video stream data representative of a particular person performing a sequence of motions to complete the task;

automatically detecting event data by a computer of a smart detection system from the multiple audio and video capture devices including the generated continuous audio and video stream data, wherein the event data comprises metadata describing the sequence of motions to complete the task by the particular person;

parsing the event data, including one or more operations of formatting the event data for utilization and analysis in one or more data models, comparing the event data to a data model in the one or more data models and filtering the event data for relevant data elements, to identify appropriate event data describing a discrete set of motions from the sequence of motions;

cross indexing the appropriate event data into a common repository of a multi-mode event database for correlation across the multiple audio and video capture devices and event types to create cross indexed appropriate event data, wherein each event has a reference to an original media resource that includes a link to a video file enabling the particular person to view a video associated with a retrieved event;

analyzing, using an analysis server, the cross indexed appropriate event data describing the discrete set of motions to form a labor standard that is specific to the particular person and specifies an optimal manner of performing the discrete set of motions for the particular person, using profile data including height, weight, age, physical limitations, experience, a work schedule, known physical limitations that prevent performance of certain tasks, list of certifications obtained and a customized labor standard associated with the particular person that accounts for strengths and weaknesses of the particular person; and

generating a set of recommendations for performing the task efficiently according to the labor standard.

2. The computer implemented method of claim 1, wherein the set of recommendations specifies a time for performing the discrete set of motions and further comprising:

monitoring to determine when the particular person is not in compliance with the labor standard for performing the discrete set of motions; and

responsive to determining that the particular person is not in compliance with the labor standard, notifying the particular person that the discrete set of motions fails to comply with the labor standard.

3. The computer implemented method of claim 1, wherein the set of recommendations identifies the particular person for performing the discrete set of motions and further comprising:

customizing the labor standard for the particular person according to the height, weight, age, physical limitations, and experience of the particular person.

4. The computer implemented method of claim 1, wherein the optimal manner is an optimal sequence for performing the discrete set of motions;

and wherein the discrete set of motions include two or more of a placement of an identified material into a first identified hand, a placement and orientation of the identified material by the particular person, a placement of an identified tool into a second identified hand, an action performed by the particular person using the identified tool, and a placement and orientation of feet of the particular person; and

wherein the first identified hand and the second identified hand are each identified as one of a dominant hand or a non-dominant hand.

5. The computer implemented method of claim 1, wherein the set of recommendations identifies a set of tools for performing the discrete set of motions.

6. The computer implemented method of claim 1, wherein the set of recommendations are generated by the analysis server.

7. The computer implemented method of claim 1, further comprising:

receiving sensor data including detection data as video data from a set of sensors associated with a facility; and

analyzing the video data that to identify the event data, wherein analyzing the video data comprises generating the metadata describing the sequence of motions for performing the task.

8. The computer implemented method of claim 7, wherein the set of sensors comprises a set of digital video cameras, a set of environmental condition sensors, and a set of physical presence and motion sensors that identify a location and motion of people and items.

9. The computer implemented method of claim 1, wherein parsing the event data further comprises:

processing the event data using one or more of a statistical method, a data mining method, a causal model, a mathematical model, and a simulation model.

10. A computer program product comprising:

a non-transitory computer usable medium having computer usable program code for managing a task stored thereon for execution by a computer of a smart detection system, the computer usable program code comprising;

computer usable program code for receiving by the computer of the smart detection system continuous audio and video stream data simultaneously from multiple audio and video capture devices representative of a particular person performing a sequence of motions to complete task;

computer usable program code for automatically detecting event data by the computer of the smart detection system, from the continuous audio and video stream data generated by the multiple audio and video capture devices, wherein the event data comprises metadata describing the sequence of motions to complete the task by the particular person;

computer usable program code for parsing the event data including one or more operations of formatting the event data for utilization and analysis in one or more data models, comparing the event data to a data model in the one or more data models and filtering the event data for relevant data elements to identify appropriate event data describing a discrete set of motions from the sequence of motions;

computer usable program code for cross indexing the appropriate event data into a common repository of a multi-mode event database for correlation across the multiple audio and video capture devices and event types to create cross indexed appropriate event data, wherein each event has a reference to an original media resource that includes a link to a video file enabling the particular person to view a video associated with a retrieved event;

computer usable program code for analyzing, using an analysis server, the cross indexed appropriate event data describing the discrete set of motions to form a labor standard that is specific to the particular person and specifies an optimal manner of performing the discrete set of motions for the particular person, using profile data including height, weight, age, physical limitations, experience, a work schedule, known physical limitations that prevent performance of certain tasks, list of certifications obtained and a customized labor standard associated with the particular person that accounts for strengths and weaknesses of the particular person; and

computer usable program code for generating a set of recommendations for performing the task efficiently according to the labor standard.

11. The computer program product of claim 10 wherein the set of recommendations specifies a time for performing the discrete set of motions and further comprising:

computer usable program code for monitoring to determine when the particular person is not in compliance with the labor standard for performing the discrete set of motions; and

computer usable program code, responsive to determining that the particular person is not in compliance with the labor standard, for notifying the particular person that the discrete set of motions fails to comply with the labor standard.

12. The computer program product of claim 10, wherein the set of recommendations identifies the particular person for performing the discrete set of motions and further comprising:

computer usable program code for customizing the labor standard for the particular person according to the height, weight, age, physical limitations, and experience of the particular person.

13. The computer program product of claim 10, wherein the optimal manner is an optimal sequence for performing the discrete set of motions;

and wherein the discrete set of motions include two or more of a placement of an identified material into a first identified hand, a placement and orientation of the identified material by the particular person, a placement of an identified tool into a second identified hand, an action performed by the particular person using the identified tool, and a placement and orientation of feet of the particular person; and

wherein the first identified hand and the second identified hand are each identified as one of a dominant hand or a non-dominant hand.

14. The computer program product of claim 10, wherein the set of recommendations identifies a set of tools for performing the discrete set of motions.

15. The computer program product of claim 10, wherein the set of recommendations are generated by the analysis server.

16. The computer program product of claim 10, further comprising:

computer usable program code for receiving sensor data including detection data as video data from a set of sensors associated with a facility; and

computer usable program code for analyzing the video data that to identify the event data, wherein analyzing the video data comprises generating the metadata describing the sequence of motions for performing the task.

17. The computer program product of claim 16, wherein the set of sensors comprises a set of digital video cameras, a set of environmental condition sensors, and a set of physical presence and motion sensors that identify a location and motion of people and items.

18. The computer program product of claim 10, wherein the computer usable program code for parsing the event data further comprises:

computer usable program code for processing the event data using one of more of a statistical method, a data mining method, a causal model, a mathematical model, and a simulation model.

19. A system for managing a task, the system comprising:

a set of sensors comprises multiple audio and video capture devices for generating continuous audio and video stream data representative of a particular person performing a sequence of motions to complete the task;

a database, wherein the database stores the continuous audio and video stream data generated by the multiple audio and video capture devices;

an analysis server, wherein the analysis server;

automatically detects event data by a computer of a smart detection system from the multiple audio and video capture devices including the generated continuous audio and video stream data, wherein the event data comprises metadata describing the sequence of motions to complete the task by the particular person;

parses the event data, including one or more operations of formatting the event data for utilization and analysis in one or more data models, comparing the event data to a data model in the one or more data models and filtering the event data for relevant data elements, to identify appropriate event data describing a discrete set of motions from the sequence of motions;

cross indexes the appropriate event data into a common repository of a multi-mode event database for correlation across the multiple audio and video capture devices and event types to create cross indexed appropriate event data, wherein each event has a reference to an original media resource that includes a link to a video file enabling the particular person to view a video associated with a retrieved event;

analyzes the cross indexed appropriate event data describing the discrete set of motions to form a labor standard that is specific to the particular person and, specifies an optimal manner of performing the discrete set of motions for the particular person, using profile data including height, weight, age, physical limitations, experience, a work schedule, known physical limitations that prevent performance of certain tasks, list of certifications obtained and a customized labor standard associated with the particular person that accounts for strengths and weaknesses of the particular person; and

generates a set of recommendations for performing the task efficiently according to the labor standard.

20. The system of claim 19, wherein the set of sensors comprises a set of digital video cameras, a set of environmental condition sensors, and a set of physical presence and motion sensors that identify a location and motion of people and items.