MONITORING DEVICE, MONITORING METHOD, CONTROL DEVICE, CONTROL METHOD, AND PROGRAM

• US 20090022368A1
• Filed: 03/15/2007
• Published: 01/22/2009
• Est. Priority Date: 03/15/2006
• Status: Active Grant

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: Ando, Tanichi, Matsuoka, Miki
• Granted Patent: US 8,406,457 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/785 on or in relation to the windshield or windows
• CPC Class Codes: B60K 2370/149 Input by detecting viewing direction
• CPC Class Codes: B60K 2370/785 on or in relation to the windshield or windows
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions

Abstract

The present invention relates to a monitoring device, monitoring method, control device, control method, and program that use information on a face direction or gaze direction of a person to cause a device to perform processing in accordance with a movement or status of the person. A target detector 251 detects a target object. A face direction/gaze direction detector 252 detects a face direction and gaze direction of a person, determines a movement or status of the person on the basis of the relationship between the detected target object and the face direction or gaze direction of the person, and notifies face direction/gaze direction information using portions 253-1 to 253-3 of a result of the determination. The face direction/gaze direction information using portions 253-1 to 253-3 cause the device to execute predetermined processing on the basis of the result of the determination. The present invention can be applied to, for example, an on-vehicle system.

Claims

1. A monitoring device that outputs a movement or status of a person to a control device that controls a device by using the movement or status of the person, the monitoring device comprising:

face direction detection means for detecting a face direction of the person;

gaze direction detection means for detecting a gaze direction of the person; and

determination means for determining a movement or status of the person on the basis of a relationship between a target object coming into view of the person and the face direction or gaze direction of the person, and outputting a result of the determination to the control device.

2. The monitoring device according to claim 1, further comprising registration means for registering the target object in a determination target list, wherein the determination means determines the movement or status of the person on the basis of the relationship between the target object registered in the determination target list and the face direction or gaze direction of the person.

3. The monitoring device according to claim 2, wherein for the target object, there is a dynamic target object that appears in or disappears from the view of the person as time passes, and a static target object that comes into the view of the person regardless of the passage of time, and the registration means registers, in the determination target list, the static target object for which a registration request for registering [the static target object] in the determination target list is supplied from the control device.

4. The monitoring device according to claim 3, wherein the registration means registers, in the determination target list, the dynamic target object for which a registration request for registering [the dynamic target object] in the determination target list is supplied from a target object detection device that detects the dynamic target object.

5. The monitoring device according to claim 2, wherein in the determination target list, at least a period for executing the determination, the control device to which the result of the determination is output, and a condition for deleting the target object from the determination target list are registered for the each target object, and the determination means determines the movement or status of the person during the period for executing the determination, the period being registered in the determination target list.

6. The monitoring device according to claim 1, further comprising:

face position detection means for detecting a position of the face of the person; and

position converting means for converting a position of the target object into a position based on the position of the face of the person, wherein

the determination means determines the movement or status of the person on the basis of the relationship between the target object based on the position of the face of the person and the face direction or gaze direction of the person.

7. The monitoring device according to claim 1, further comprising facial feature status detection means for detecting a status of a facial feature on the face of the person, wherein the determination means determines the movement or status of the person on the basis of the status of the facial feature on the face of the person.

8. A monitoring method of a monitoring device that outputs a movement or status of a person to a control device that controls a device by using the movement or status of the person, the monitoring method comprising the steps of:

detecting a face direction of the person;

detecting a gaze direction of the person; and

determining a movement or status of the person on the basis of a relationship between a target object coming into view of the person and the face direction or gaze direction of the person, and outputting a result of the determination to the control device.

9. A program that causes a computer to execute processing for outputting a movement or status of a person to a control device that controls a device by using the movement or status of the person, the program comprising the steps of:

detecting a face direction of the person;

detecting a gaze direction of the person; and

determining a movement or status of the person on the basis of a relationship between a target object coming into view of the person and the face direction or gaze direction of the person, and outputting a result of the determination to the control device.

10. A control device that controls a device by using a movement or status of a person that is output from a monitoring device,

wherein the control device requests the monitoring device to register a part of the device as a target object coming into view of the person, in a determination target list for determining a movement or status of the person on the basis of a relationship between the target object and a face direction or gaze direction of the person, and causes the device to execute predetermined processing, by using a result of determination that is received from the monitoring device in response to the request.

11. A control method of a control device that controls a device by using a movement or status of a person that is output from a monitoring device, the control method comprising the step of:

requesting the monitoring device to register a part of the device as a target object coming into view of the person, in a determination target list for determining a movement or status of the person on the basis of a relationship between the target object and a face direction or gaze direction of the person, and causing the device to execute predetermined processing, by using a result of determination that is received from the monitoring device in response to the request.

12. A program for causing a computer to execute control of a device by using a movement or status of a person that is output from a monitoring device, the program comprising the step of:

requesting the monitoring device to register a part of the device as a target object coming into view of the person, in a determination target list for determining a movement or status of the person on the basis of a relationship between the target object and a face direction or gaze direction of the person, and causing the device to execute predetermined processing, by using a result of determination that is received from the monitoring device in response to the request.