Methods for determining manufacturing waste to optimize productivity and devices thereof

• US 10,037,504 B2
• Filed: 03/27/2015
• Issued: 07/31/2018
• Est. Priority Date: 02/12/2015
• Status: Active Grant

• Current Assignee: Wipro Limited
• Original Assignee: Wipro Limited
• Inventors: Modi, Krupal Chandresh, Vatnani, Rinku, Gupta, Akash, Ladak, Akbar Abdulmalik
• Primary Examiner(s): Miller, Alan S
• Application Number:
  US14/670,528
• Publication Number:
  US 20160239769A1
• Time in Patent Office: 1,222 Days
• Field of Search: 705 711- 742
• CPC Class Codes: G06Q 10/0633 Workflow analysis
• CPC Class Codes: G06Q 10/06395 Quality analysis or management
• CPC Class Codes: G06Q 50/04 Manufacturing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: Y02P 90/30 Computing systems specially adapted for manufacturing

Abstract

A method, non-transitory computer readable medium, and productivity assessment computing device that identifies entities present in frames of a video. Entity movement across the frames is plotted to obtain a trajectory of the entities. Interactions of one or more of the entities in each of the frames are identified. A unique sequence encoding is generated for subtasks performed by each of the entities. One of the subtasks is classified as a waste subtask based on one or more of the interactions corresponding to the one of the subtasks and the trajectory and a type of each of the entities associated with the interactions. The sequence encodings of the one of the subtasks are correlated with a number of frames per second of the video to determine waste duration value(s) for a task and the waste duration value(s) are output.

Claims

1. A method for automatically processing and analyzing video data to facilitate more effective and efficient identification of waste subtasks included therein, the method implemented by one or more productivity assessment computing devices and comprising:

obtaining video data captured by one or more video capture devices, the video data comprising a plurality of frames of a video, and automatically analyzing the video data to identify a plurality of entities present in each of the frames and a type of each of the entities based on a correlation of the video data with a plurality of stored classifiers;

plotting movement of each of the entities across at least a subset of the frames to determine a trajectory of each of the entities based on a stored digital floor plan, wherein the subset of the frames comprises consecutive ones of the frames;

generating entity adjacency data for each of the entities based on a position of the entities in each of the frames and identifying a plurality of interactions of one or more of the entities in each of the frames based on the entity adjacency data;

generating a unique sequence encoding for each of a plurality of subtasks performed by each of the entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;

classifying at least one of the subset of the subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of the subtasks and the determined trajectory and the type of each of the entities associated with the one or more of the interactions;

correlating the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video data to determine at least one waste duration value for the task; and

outputting an indication of the one of the subset of the subtasks classified as a waste subtask and the waste duration value.

2. The method as set forth in claim 1, further comprising identifying one or more features of each of the entities in each of the frames.

3. The method as set forth in claim 1, wherein the subset of the subtasks is associated with the task based on a correlation with the classifiers and the classifiers comprise training images of the entities and training videos of the tasks.

4. The method as set forth in claim 1, wherein the waste duration value is for transportation waste and comprises a duration that a material one of the entities is interacting with a machine one of the entities or a human one of the entities and is moving from one location to another location.

5. The method as set forth in claim 1, wherein the waste duration value is for a waiting waste and comprises a duration that a human one of the entities is idle and not interacting with any other of the entities.

6. The method as set forth in claim 1, wherein the waste duration value is for a motion waste and comprises a duration that a human one of the entities is not interacting with a material one of the entities or a machine one of the entities and is moving from one location to another location.

7. The method as set forth in claim 6, further comprising determining one or more frequently followed paths based on the trajectories and outputting an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.

8. A productivity assessment computing device comprising memory comprising programmed instructions stored thereon and at least one processor coupled to the memory, which is configured to be capable of executing the stored programmed instructions to:

obtain video data captured by one or more video capture devices, the video data comprising a plurality of frames, and automatically analyze the video data to identify a plurality of entities present in each of the frames and a type of each of the entities based on a correlation of the video data with a plurality of stored classifiers;

plot movement of each of the entities across at least a subset of the frames to determine a trajectory of each of the entities based on a stored digital floor plan, wherein the subset of the frames comprises consecutive ones of the frames;

generate entity adjacency data for each of the entities based on a position of the entities in each of the frames and identify a plurality of interactions of one or more of the entities in each of the frames based on the entity adjacency data;

generate a unique sequence encoding for each of a plurality of subtasks performed by each of the entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;

classify at least one of the subset of the subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of the subtasks and the determined trajectory and the type of each of the entities associated with the one or more of the interactions;

correlate the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video data to determine at least one waste duration value for the task; and

output an indication of the one of the subset of the subtasks classified as a waste subtask and the waste duration value.

9. The productivity assessment computing device as set forth in claim 8, wherein the processor is further configured to be capable of executing the stored programmed instruction to identify one or more features of each of the entities in each of the frames.

10. The productivity assessment computing device as set forth in claim 8, wherein the subset of the subtasks is associated with the task based on a correlation with the classifiers and the classifiers comprise training images of the entities and training videos of the tasks.

11. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a transportation waste and comprises a duration that a material one of the entities is interacting with a machine one of the entities or a human one of the entities and is moving from one location to another location.

12. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a waiting waste and comprises a duration that a human one of the entities is idle and not interacting with any other of the entities.

13. The productivity assessment computing device as set forth in claim 8, wherein the waste duration value is for a motion waste and comprises a duration that a human one of the entities is not interacting with a material one of the entities or a machine one of the entities and is moving from one location to another location.

14. The productivity assessment computing device as set forth in claim 13, wherein the processor is further configured to be capable of executing the stored programmed instruction to determine one or more frequently followed paths based on the trajectories and output an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.

15. A non-transitory computer readable medium having stored thereon instructions for processing and analyzing video data to facilitate more effective and efficient identification of waste subtasks included therein comprising executable code, which when executed by at least one processor, causes the processor to:

obtain video data captured by one or more video capture devices, the video data comprising a plurality of frames, and automatically analyze the video data to identify a plurality of entities present in each of the frames and a type of each of the entities based on a correlation of the video data with a plurality of stored classifiers;

plot movement of each of the entities across at least a subset of the frames to determine a trajectory of each of the entities based on a stored digital floor plan, wherein the subset of the frames comprises consecutive ones of the frames;

generate entity adjacency data for each of the entities based on a position of the entities in each of the frames and identify a plurality of interactions of one or more of the entities in each of the frames based on the entity adjacency data;

generate a unique sequence encoding for each of a plurality of subtasks performed by each of the entities, wherein the subtasks are identified based on the interactions and a sequence of at least a subset of the subtasks is associated with at least one task;

classify at least one of the subset of the subtasks as a waste subtask of the task based on one or more of the interactions corresponding to the one of the subset of the subtasks and the determined trajectory and the type of each of the entities associated with the one or more of the interactions;

correlate the sequence encodings of the one of the subset of subtasks with a number of frames per second of the video data to determine at least one waste duration value for the task; and

output an indication of the one of the subset of the subtasks classified as a waste subtask and the waste duration value.

16. The non-transitory computer readable medium as set forth in claim 15, wherein the executable code, when executed by the processor, further causes the processor to identify one or more features of each of the entities in each of the frames.

17. The non-transitory computer readable medium as set forth in claim 15, wherein the subset of the subtasks is associated with the task based on a correlation with the classifiers and the classifiers comprise training images of the entities and training videos of the tasks.

18. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a transportation waste and comprises a duration that a material one of the entities is interacting with a machine one of the entities or a human one of the entities and is moving from one location to another location.

19. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a waiting waste and comprises a duration that a human one of the entities is idle and not interacting with any other of the entities.

20. The non-transitory computer readable medium as set forth in claim 15, wherein the waste duration value is for a motion waste and comprises a duration that a human one of the entities is not interacting with a material one of the entities or a machine one of the entities and is moving from one location to another location.

21. The non-transitory computer readable medium as set forth in claim 20, wherein the executable code, when executed by the processor, further causes the processor to determine one or more frequently followed paths based on the trajectories and output an indication of the frequently followed paths, wherein the frequently followed paths correspond with the one of the subset of subtasks classified as a waste subtask.