Apparatus and a method for sensing events from a remote location

• US 5,097,328 A
• Filed: 10/16/1990
• Issued: 03/17/1992
• Est. Priority Date: 10/16/1990
• Status: Expired due to Fees

• Current Assignee: Robert B. Boyette
• Original Assignee: Robert B. Boyette
• Inventors: Boyette, Robert B.
• Primary Examiner(s): Kostak, Victor R.
• Application Number:
  US07/598,376
• Time in Patent Office: 518 Days
• Field of Search: 358/108, 358/105, 358/93, 358/125, 358/126
• US Class Current: 348/150
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G07C 11/00 Arrangements, systems or apparatus for checking, e.g. the occurrence of a condition, not provided for elsewhere
• CPC Class Codes: G07C 2011/04 related to queuing systems
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 13/19693 using multiple video sources viewed on a single or compound screen

Abstract

A remote sensing system suitable for use in businesses which include queues and/or servers has a camera which captures successive images of the area to be monitored. The sensing system includes one or more video cameras which generate an array of pixel samples representing an image of the area. Sub-areas of interest in the image are defined by selectively illuminating each of the sub-areas during an initialization phase. During operation, the system analyzes pixel data from only the defined sub-areas. A combination of blob analysis and motion analysis is used to determine if a server is on-duty and present at his station, and if he is serving a customer. The system also maintains a count of the number of people waiting in the queue to be served. This information is used to generate displays directing waiting customers to the next available server and to inform newly arriving customers of their expected waiting time. The remote sensing system may also be used in an office environment to detect changes initiated by a worker or inherent in his work to gather performance statistics on the worker.

Claims

1. A remote sensing system for detecting the occurrence of at least first and second contemporaneous events, comprising:

imaging means for capturing electronic signals representing an image of an area including at least first and second predetermined sub-areas in which the first and second events may occur;

address memory means for holding data values defining the first and second predetermined sub-areas of the image;

means for extracting a number of data values from the electronic signals the extracted data values representing only the first and second predetermined sub-areas of the image, the number of extracted data values being a less than a number of data values representing the entire image of the area; and

analyzing means for comparing the extracting data values to reference image data values, representing the first and second predetermined sub-areas before the occurrence of the first and second events, to determine if the first and second events have occurred in the respective first and second sub-areas.

2. The remote sensing system of claim 1 wherein:

the first and second events to be detected are the presence of first and second objects in the first and second sub-areas and the first and second objects have specific characteristics;

the remote sensing system further includes data memory means for holding the reference image data values; and

said analyzing means further includes;

means for generating difference data representing the difference between the extracted data values and the reference data values; and

means for analyzing the difference data values for the specific characteristics to determine if the first and second objects are present in the respective first and second sub-areas.

3. The remote sensing system of claim 2 wherein the specific characteristics include characteristics concerning motion which the object may exhibit and said means for analyzing includes:

means for storing difference data representing the difference between the reference data and extracted data captured during a prior time interval; and

means for comparing the difference data to the stored difference data to determine if motion is indicated n at least one of the first and second sub-areas and if the indicated motion is consistent with the specific characteristics of the object.

4. The remote sensing system of claim 1 further comprising:

light source means, responsive to a control signal, for emitting varying levels of illumination; and

means for encoding a data message and for modulating said control signal with the encoded data message;

wherein;

one of said first and second predetermined sub-areas includes an image of said light source means, the reference image data values corresponding to said one predetermined sub-area represent said light source means providing a predetermined level of illumination and the event to be detected is a change in the level of illumination emitted by said light source means; and

the analyzing means includes means for comparing the extracted data taken during respectively different time intervals to the reference data values for said one predetermined sub-area to regenerate said encoded data message.

5. The remote sensing system of claim 4 wherein:

said means for encoding said data message includes means for converting said message into a bit-serial binary signal;

said light source means is responsive to said control signal for emitting light of first and second levels of illumination; and

said analyzing means records said first level of illumination as a binary one and said second level of illumination as a binary zero to regenerate the bit-serial binary signal.

6. The remote sensing system of claim 4 wherein:

the other one of said first and second predetermined sub-areas includes an individual and the encoded data message contains data relating to the individual.

7. The remote sensing system of claim 1, further including means for specifying the data values defining at least one of the first and second predetermined sub-areas of the image, comprising:

means for capturing electronic signals representing successive images from said imaging means;

means for detecting at least one of said electronic signals representing an element of said image having a predetermined level of illumination;

means for assigning first and second coordinates in said image to said detected image element; and

means for evaluating said first and second coordinates for selection as one of the data values defining said one predetermined sub-area.

8. The remote sensing system of claim 7, wherein the first and second coordinates are selected as center coordinates of said one predetermined sub-area;

and said remote sensing system further includes means for specifying an area about said center coordinates which is to be encompassed by the data values defining said one predetermined sub-area.

9. The remote sensing system of claim 7 wherein said means for evaluating includes:

means for comparing said first coordinate with previously determined first coordinates to define maximum and minimum values for said first coordinate; and

means for comparing said second coordinate with previously determined second coordinates to define maximum and minimum values for said second coordinate;

wherein the respective maximum and minimum values for said first and second coordinates constitute the data values defining said one predetermined sub-area.

10. The remote sensing system of claim 1 further including:

means for associating said first sub-area with a first class of individuals and for associating said second sub-area with a second class of individuals; and

signalling means for indicating a predetermined relationship between said first class of individual sand said second class of individuals responsive to the detected occurrence of said first and second contemporaneous events.

11. The remote sensing system of claim 10 wherein the first class of individuals includes bank tellers and the second class of individuals includes bank customers and the signalling means indicates that a bank teller is serving a bank customer responsive to the detected occurrence of said first and second contemporaneous events.

12. The remote sensing system of claim 1, further including:

means, coupled to said analyzing means for recording the occurrence of said first and second events to maintain a historical record of events occurring in said first and second sub-areas.

13. A method for detecting the occurrence of first and second contemporaneous events in respective predetermined first and second sub-areas of a remote area, comprising the steps of:

capturing electronic signals representing an image of the remote area including the predetermined first and second sub-areas;

extracting a number of data values, representing only the predetermined first and second sub-areas of the image, from the electronic signals, wherein the number of extracted data values is a less than a number of data values representing the entire image of the remote area; and

automatically comparing the extracted data values to reference image data values representing the first and second sub-areas before the respective first and second events have occurred to determine if the respective first and second events have occurred.

14. The method of claim 13 where the events to be detected are the presence of first and second objects in the respective first and second sub-areas, said first and second objects having specific characteristics, and the step of automatically comparing the extracted data values includes the steps of:

storing the reference image data values;

generating difference data values representing the difference between the extracted data values and the reference data values;

analyzing the difference data values for the specific characteristics to determine if the first and second objects are present in the respective first and second sub-areas.

15. The method of claim 13 wherein one of the predetermined first and second sub-areas includes a light source and the event to be detected is a coded message transmitted by modulation of the intensity of the light emitted by said light source and said reference image data values corresponding to said one sub-area represent a predetermined level of illumination of said light source, said method further including the steps of:

comparing the extracted data to said corresponding reference data values to provide a signal indicating whether said extracted data value is greater than or less than said reference intensity value; and

recording successive signal values to regenerate said coded message.

16. A method of monitoring a queue of people comprising the steps of:

generating electronic signals representing successive images of the queue as seen from a remote location;

extracting, from said electronic signals, first and second sets of successive data values representing a time sequence of images of respective areas surrounding an entrance into the queue and an exit from the queue, respectively;

analyzing the data values representing each of the time sequences of images to detect motion into the queue and motion out of the queue;

increasing a count of people in the queue when motion into the queue is detected; and

decreasing the count of people in the queue when motion out of the queue is detected.

17. The method of claim 16 further comprising the steps of:

monitoring the count of people in the queue to determine when the count value is less than a predetermined value;

capturing electronic signals representing at least a portion of the queue when said count value is less than the predetermined value;

analyzing said captured electronic signals to count the number of people in the queue; and

setting said count value to the number of people counted in the analyzing step.

18. The method of claim 17 where the predetermined value is unity and the setting step sets the count value to zero when the analyzing step indicates that the queue is empty.

19. In a system for processing a group of items, a method of automatically determining an expected waiting time for an item entering a queue which feeds a set of servers, each of which process one item at a time, the method comprising the steps of:

generating electronic signals representing successive images of the queue and the servers as seen from a remote location;

extracting, from said electronic signals, data values representing sub-areas of the image including the queue and each of the servers;

analyzing the image data obtained from each of the sub-areas for each of the servers to determine an expected service time for each server;

averaging the expected service time for all servers to obtain an average expected service time;

analyzing the image data obtained from the sub-area for the queue to determine the approximate number of people in the queue; and

multiplying the determined approximate number of people in the queue by the average expected service time to obtain the expected waiting time in the queue for the item entering the queue.