Machine vision system for enterprise management

• US 7,617,167 B2
• Filed: 04/08/2004
• Issued: 11/10/2009
• Est. Priority Date: 04/09/2003
• Status: Active Grant

• Current Assignee: Diversified Innovations Fund, LLP
• Original Assignee: Avisere, Inc.
• Inventors: Undhagen, Roger Karl Mikael, Acharya, Tinku, Griffis, Andrew J.
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): BHARADWAJ, KALPANA
• Application Number:
  US10/820,906
• Publication Number:
  US 20070090180A1
• Time in Patent Office: 2,042 Days
• Field of Search: 709/203, 701/214, 706/45
• US Class Current: 706/45
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A system for use in managing activity of interest within an enterprise is provided. The system comprises a computer configured to (i) receive sensor data that is related to key activity to the enterprise, such key activity comprising a type of object and the object'"'"'s activity at a predetermined location associated with the enterprise, the sensor providing information from which an object'"'"'s type and activity at the predetermined location can be derived, (ii) process the sensor data to produce output that is related to key activity to the enterprise, and (ii) store the information extracted from the processed data in a suitable manner for knowledge extraction and future analysis. According to a preferred embodiment, the object is human, machine or vehicular, and the computer is further configured to correlate sensor data to key activity to the enterprise and the output includes feedback data based on the correlation.

Claims

1. A system for use in managing activity of interest within an enterprise, comprising a computer having a model for tracking one or more human objects in a scene related to the enterprise and configured to

(i) access key activity data comprising object data related at least to the human features, locations, movement patterns, and predetermined other human activity of a human object relative to other physically and visually distinct objects in a scene related to the enterprise,

(ii) receive sensor data that is taken in by a sensor of a predetermined scene related to the enterprise,

(iii) extract solely from the sensor data changes that enables separation of foreground and background objects, including human objects, localization of human object features, and connection of object features that should be connected,

(iv) classify the extracted objects, including extracted human objects, in plurality of classes of objects of different features, shapes, movement and behavioral patterns,

(v) determine whether a human object is new to the human object data in the computer and initiating tracking of the new human object and if the human object exists as object data in the computer updating a track of the existing human object, and update the computer model in accordance with the foregoing, and

(vi) compare human object data to key activity data and produce a key activity message, wherein the key activity message contains any or all of the following elements:

what is moving or changing, a person, animal, machine, or other inanimate object;

what behavior the moving or changing object is engaged in;

where the object is located;

when the motion or change is occurring and measurable interpretation of the change in phenomena or activity in order to make a decision by a machine or a human for a purpose.

2. A system as defined in claim 1, wherein the computer is configured to make a preliminary determination from the sensor data as whether to extract from the sensor data one or more human objects and the state of activity for each human object with respect to the state of activity of other physically distinct objects in the sensor data, irrespective of objective compliance.

3. A system as defined in claim 2, wherein in extracting from the sensor data the human object and the state of activity for the human object with respect to other physically distinct objects in the sensor data, irrespective of object compliance, the computer is configured to determine if the human object has been previously extracted from sensor data or is being initially extracted from the sensor data.

4. A system as defined in claim 3, wherein the computer is configured such that after the computer has processed the key activity data and the extracted human object data and determined whether to produce output that is related to the key activity, the computer is configured to receive additional sensor data taken in by the sensor of the predetermined scene related to the enterprise, extract from the additional sensor data a human object and the state of activity for the human object with respect to the state of activity for other physically distinct objects in the sensor data, irrespective of object compliance, and process the key activity and the extracted data from the additional sensor data and determine whether to produce output that is related to the key activity.

5. A method for use in managing human activity of interest within an enterprise, comprising:

a. providing a computer configured to;

(i) access key activity data comprising data related to activity of a human object relative to other physically and visually distinct objects in a predetermined scene related to the enterprise,

(ii) receive sensor data that is taken in by a sensor of a scene related to the enterprise,

(iii) extract solely from the sensor data a human object and the state of activity for the human object with respect to the state of activity for other physically and visually distinct objects in the sensor data, Irrespective of object compliance, including separation of foreground and background objects, including human objects, localization of human object features, and connection of human object features that should be connected, and

(iv) process the key activity data and the extracted data to produce output that is related to the key activity;

b. inputting to the computer sensor data that is taken in by a sensor of a scene related to the enterprise;

c. extracting solely from the sensor data, via the computer, a human object and the state of activity for the human object with respect to the state of activity for other physically and visually distinct object in the sensor data, including human objects, localization of human object features, and connection of human object features that should be connected;

d. processing the key activity data and the extracted data to produce output that is related to the key activity, wherein the output comprises of any or all of the following elements;

what is moving or changing, a person, animal, machine, or other inanimate object;

what behavior the moving or changing object is engaged in;

where the object is located;

when the motion or change is occurring and measurable interpretation of the change in phenomena or activity in order to make a decision by a machine or a human for a purpose; and

e. storing the detected activities in a database for extraction and use in a decision support system.

6. A method as defined in claim 5, wherein a preliminary determination is made, via the computer, from the sensor data as to whether to extract from the sensor data the human object and the state of activity for the human object with respect to the state of activity of other physically distinct objects in the sensor data, irrespective of objective compliance.

7. A method as defined in claim 6, wherein in extracting from the sensor data the human object and the state of activity for the human object in the scene with respect to other physically distinct objects in the sensor data, irrespective of object compliance, the computer determines if the human object has been previously extracted from sensor data or is being initially extracted from the sensor data.

8. A method as defined in claim 7, wherein after the computer has processed the key activity data and the extracted data and determined whether to produce output that is related to the key activity, the computer (i) receives additional sensor data taken in by the sensor of the predetermined scene related to the enterprise, (ii) extracts from the additional sensor data a human object and the state of activity for the human-object with respect to the state of activity for other physically distinct objects in the sensor data, irrespective of object compliance, (iii) processes the key activity and the extracted data from the additional sensor data and determines whether to produce output that is related to the key activity and (iv) update the already stored information of the key activity of the object.