Monitoring device, method thereof

• US 9,019,373 B2
• Filed: 03/23/2012
• Issued: 04/28/2015
• Est. Priority Date: 09/08/2011
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Yamamoto, Daisuke, Tasaki, Tsuyoshi
• Primary Examiner(s): ELAHI, SHAN E
• Application Number:
  US13/428,410
• Publication Number:
  US 20130063593A1
• Time in Patent Office: 1,131 Days
• Field of Search: 348/143
• US Class Current: 348/143
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A monitoring device includes an indicating unit configured to indicate an object to be monitored; a camera to acquire time-series images; a predicting unit configured to predict the position of destination of the characteristic points belonging to a first aggregation including characteristic points having an amount of movement larger than a reference amount of movement set in advance; a detecting unit configured to detect the position of center of gravity of the changed area; a determining unit configured to determine the positions of destination as an indicating position when representative amount of movement is larger than a first threshold value and determine the position of center of gravity as the indicating position when the representative amount of movement is smaller than the first threshold value; and a control unit configured to control the indicating unit to cause the indicating unit to indicate the indicating position are provided.

Claims

1. A monitoring device comprising:

an indicator, comprising a facial feature, for indicating an object to be monitored;

a motor configured to move the indicator;

a camera configured to take an image of the object to be monitored and acquire time-series images;

a computer processor configured to

extract a plurality of characteristic points from the time-series images, calculate amounts of movement of the respective characteristic points, predict a position of destination of the characteristic points belonging to a first aggregation including characteristic points having an amount of movement larger than a reference amount of movement set in advance, and obtain a representative amount of movement that is a representative value of amounts of movement of the respective characteristic points belonging to the first aggregation;

extract a changed area on the basis of a difference between time-series images, and detect a position of a center of gravity of the changed area;

determine the position of destination as an indicating position when the representative amount of movement is larger than a first threshold value and determine the position of the center of gravity as the indicating position when the representative amount of movement is smaller than the first threshold value; and

a control unit configured to control the motor to cause the indicator to indicate the indicating position.

2. The device according to claim 1, further comprising:

storage configured to store a plurality of monitoring areas in the images acquired by the camera, wherein

the computer processor is further configured to

set a position in one of the monitoring areas as the indicating position when the representative amount of movement is relatively larger than the first threshold value and the distances between respective positions in the plurality of monitoring areas and the position of destination are relatively smaller than a preset second threshold value.

3. The device according to claim 2, wherein

the computer processor is further configured to

output the position of destination at a time point when the determination to set the position of destination as the indicating position is performed continuously by a reference number of times set in advance and the determination not to set the position of destination as the indicating position is performed as a position of termination of movement to the storage, and

the storage stores the position of termination of movement or a peripheral area thereof as a new monitoring area.

4. The device according to claim 1, wherein the computer processor is further configured to

calculate the representative amount of movement by averaging the amounts of movement of the respective characteristic points belonging to the first aggregation and a representative direction of movement by averaging the directions of movement of the respective characteristic points, and

set a representative position of the respective characteristic points having the representative amount of movement and the representative direction of movement and belonging to the first aggregation to the position of destination.

5. The device according to claim 1, wherein

the computer processor is further configured to

calculate the representative amount of movement by averaging the amounts of movement of the respective characteristic points belonging to the first aggregation and a representative direction of movement by averaging the directions of movement of the respective characteristic points,

set a representative position of the respective characteristic points having the representative amount of movement and the representative direction of movement and belonging to the first aggregation,

obtain a second aggregation including the characteristic points having an angle formed between the direction of movement and the representative direction of movement of the characteristic points belonging to the first aggregation smaller than a reference angle set in advance, and

set the position of the characteristic point located at the farthest position along the representative direction of movement from the representative position among the characteristic points belonging to the second aggregation as the position of destination.

6. The device according to claim 1, wherein the computer processor is further configured to determine the indicating position and then retain the determination of the indicating position until the lapse of a certain period set in advance.

7. The device according to claim 1, wherein the computer processor is further configured to determine the position of the center of gravity as the indicating position when dispersion of the changed area is smaller than a predetermined value set in advance.

8. A method of monitoring with a monitoring device including an indicator, comprising a facial feature, for indicating an object to be monitored, a motor configured to move the indicator, and a camera to take images of the object to be monitored to obtain time-series images, the method comprising:

extracting a plurality of characteristic points from the time-series images, calculating amounts of movement of the respective characteristic points, predicting a position of destination of the characteristic points belonging to a first aggregation including characteristic points having an amount of movement larger than a reference amount of movement set in advance, and obtaining a representative amount of movement that is a representative value of amounts of movement of the respective characteristic points belonging to the first aggregation;

extracting a changed area on the basis of a difference between time-series images, and detecting a position of a center of gravity of the changed area;

determining the position of destination as an indicating position when the representative amount of movement is larger than a first threshold value, and determining the position of the center of gravity as the indicating position when the representative amount of movement is smaller than the first threshold value; and

controlling the motor to cause the indicator to indicate the indicating position.

9. A program stored in a non-transitory computer readable medium, the program, when executed, causing a computer to control a monitoring device including an indicator, comprising a facial feature, for indicating an object to be monitored, a motor configured to move the indicator, and a camera to take images of the object to be monitored to obtain time-series images, the computer controlling the monitoring device to at least:

extract a plurality of characteristic points from the time-series images, calculate amounts of movement of the respective characteristic points, predict a position of destination of the characteristic points belonging to a first aggregation including characteristic points having an amount of movement larger than a reference amount of movement set in advance, and obtain a representative amount of movement that is a representative value of amounts of movement of the respective characteristic points belonging to the first aggregation;

extract a changed area on the basis of a difference between time-series images, and detect a position of a center of gravity of the changed area;

determine the position of destination as an indicating position when the representative amount of movement is larger than a first threshold value, and determine the position of the center of gravity as the indicating position when the representative amount of movement is smaller than the first threshold value; and

control the motor to cause the indicator to indicate the indicating position.