METHODS FOR DETERMINING MANUFACTURING WASTE TO OPTIMIZE PRODUCTIVITY AND DEVICES THEREOF

• US 20160239769A1
• Filed: 03/27/2015
• Published: 08/18/2016
• Est. Priority Date: 02/12/2015
• Status: Active Grant

• Current Assignee: Wipro Limited
• Original Assignee: Wipro Limited
• Inventors: Gupta, Akash, Modi, Krupal Chandresh, Vatnani, Rinku, Ladak, Akbar Abdulmalik
• Granted Patent: US 10,037,504 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/0633 Workflow analysis
• CPC Class Codes: G06Q 10/06395 Quality analysis or management
• CPC Class Codes: G06Q 50/04 Manufacturing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: Y02P 90/30 Computing systems specially adapted for manufacturing

Abstract

A method, non-transitory computer readable medium, and productivity assessment computing device that identifies entities present in frames of a video. Entity movement across the frames is plotted to obtain a trajectory of the entities. Interactions of one or more of the entities in each of the frames are identified. A unique sequence encoding is generated for subtasks performed by each of the entities. One of the subtasks is classified as a waste subtask based on one or more of the interactions corresponding to the one of the subtasks and the trajectory and a type of each of the entities associated with the interactions. The sequence encodings of the one of the subtasks are correlated with a number of frames per second of the video to determine waste duration value(s) for a task and the waste duration value(s) are output.

Claims

1. A method for analyzing video data to determine manufacturing waste to optimize productivity, the method comprising:

identifying, by a productivity assessment computing device, a plurality of entities present in each of a plurality of obtained frames of a video and a type of each of the plurality of entities;

plotting, by the productivity assessment computing device, movement of each of the plurality of entities across at least a subset of the frames to obtain a trajectory of each of the plurality of entities;

identifying, by the productivity assessment computing device, a plurality of interactions of one or more of the plurality of entities in each of the plurality of frames;

generating, by the productivity assessment computing device, a unique sequence encoding for each of a plurality of subtasks performed by each of the plurality of entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;

classifying, by the productivity assessment computing device, at least one of the subset of subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of subtasks and the trajectory and the type of each of the plurality of entities associated with the one or more of the interactions;

correlating, by the productivity assessment computing device, the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video to determine at least one waste duration value for the task; and

outputting, by the productivity assessment computing device, the waste duration value.

2. The method as set forth in claim 1, wherein the identifying further comprises identifying one or more features of each of the plurality of entities in each of the plurality of frames and the determining further comprises correlating a mapping of each of the plurality of entities to a position on a floor plan with the features of each of the plurality of entities in adjacent ones of the plurality of frames.

3. The method as set forth in claim 1, wherein the plurality of entities are identified based on a plurality of classifiers, the subset of the subtasks is associated with the task based on a correlation with the classifiers, and the classifiers comprise training images of the plurality of entities and training videos of the tasks.

4. The method as set forth in claim 1, wherein the waste duration value is for transportation waste and comprises a duration that a material, one of the plurality of entities is interacting with a machine, one of the plurality of entities or a human, one of the plurality of entities and is moving from one location to another location.

5. The method as set forth in claim 1, wherein the waste duration value is for a waiting waste and comprises a duration that a human, one of the entities is idle and not interacting with any other of the plurality of entities.

6. The method as set forth in claim 1, wherein the waste duration value is for a motion waste and comprises a duration that a human, one of the entities is not interacting with a material, one of the entities or a machine, one of the entities and is moving from one location to another location.

7. The method as set forth in claim 6, further comprising determining, by the productivity assessment computing device, one or more frequently followed paths based on the trajectories and outputting an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.

8. A productivity assessment computing device comprising at least one processor and a memory coupled to the processor which is configured to be capable of executing programmed instructions comprising and stored in the memory to:

identify a plurality of entities present in each of a plurality of obtained frames of a video and a type of each of the plurality of entities;

plot movement of each of the plurality of entities across at least a subset of the plurality of frames to obtain a trajectory of each of the plurality of entities;

identify a plurality of interactions of one or more of the plurality of entities in each of the plurality of frames;

generate a unique sequence encoding for each of a plurality of subtasks performed by each of the plurality of entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;

classify at least one of the subset of subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of subtasks and the trajectory and the type of each of the entities associated with the one or more of the interactions;

correlate the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video to determine at least one waste duration value for the task; and

output the waste duration value.

9. The productivity assessment computing device as set forth in claim 8, wherein the processor coupled to the memory is further configured to be capable of executing at least one additional programmed instruction to:

identify one or more features of each of the plurality of entities in each of the plurality of frames; and

correlate a mapping of each of the plurality of entities to a position on a floor plan with the features of each of the plurality of entities in adjacent ones of the plurality of frames.

10. The productivity assessment computing device as set forth in claim 8, wherein the plurality of entities are identified based on a plurality of classifiers, the subset of the subtasks is associated with the task based on a correlation with the classifiers, and the classifiers comprise training images of the plurality of entities and training videos of the tasks.

11. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a transportation waste and comprises a duration that a material, one of the plurality of entities is interacting with a machine one of the entities or a human, one of the entities and is moving from one location to another location.

12. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a waiting waste and comprises a duration that a human, one of the entities is idle and not interacting with any other of the plurality of entities.

13. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a motion waste and comprises a duration that a human, one of the entities is not interacting with a material, one of the entities or a machine, one of the entities and is moving from one location to another location.

14. The productivity assessment computing device as set forth in claim 13, wherein the processor coupled to the memory is further configured to be capable of executing at least one additional programmed instruction to determine one or more frequently followed paths based on the trajectories and outputting an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.

15. A non-transitory computer readable medium having stored thereon instructions for determining and analyzing manufacturing waste to optimize productivity comprising executable code which when executed, by a processor, causes the processor to perform steps comprising:

identifying a plurality of entities present in each of a plurality of obtained frames of a video and a type of each of the plurality of entities;

plotting movement of each of the plurality of entities across at least a subset of the plurality of frames to obtain a trajectory of each of the plurality of entities;

identifying a plurality of interactions of one or more of the plurality of entities in each of the plurality of frames;

generating a unique sequence encoding for each of a plurality of subtasks performed by each of the plurality of entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;

classifying at least one of the subset of subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of subtasks and the trajectory and the type of each of the plurality of entities associated with the one or more of the interactions;

correlating the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video to determine at least one waste duration value for the task; and

outputting the waste duration value.

16. The non-transitory computer readable medium as set forth in claim 15, wherein the identifying further comprises identifying one or more features of each of the entities in each of the frames and the determining further comprises correlating a mapping of each of the plurality of entities to a position on a floor plan with the features of each of the plurality of entities in adjacent ones of the plurality of frames.

17. The non-transitory computer readable medium as set forth in claim 15, wherein the plurality of entities are identified based on a plurality of classifiers, the subset of the subtasks is associated with the task based on a correlation with the classifiers, and the classifiers comprise training images of the plurality of entities and training videos of the tasks.

18. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a transportation waste and comprises a duration that a material, one of the entities is interacting with a machine one of the entities or a human one of the entities and is moving from one location to another location.

19. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a waiting waste and comprises a duration that a human, one of the entities is idle and not interacting with any other of the plurality of entities.

20. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a motion waste and comprises a duration that a human, one of the plurality of entities is not interacting with a material, one of the plurality of entities or a machine, one of the entities and is moving from one location to another location.

21. The non-transitory computer readable medium as set forth in claim 20, further having stored thereon instructions which when executed, by the processor further cause the processor to perform at least one additional step comprising determining one or more frequently followed paths based on the trajectories and outputting an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.