INFORMATION PROCESSING DEVICE, GESTURE DETECTION METHOD, AND GESTURE DETECTION PROGRAM

• US 20160004321A1
• Filed: 04/25/2014
• Published: 01/07/2016
• Est. Priority Date: 09/11/2013
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: MATSUBARA, Takashi, MORI, Naoki, TAKADA, Shintaro
• Granted Patent: US 9,696,814 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/11 Instrument graphical user interfaces or menu aspects
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1464 3D-gesture
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

It is possible to operate a device while reducing erroneous recognition of a device even when a line of sight does not face the device in an accurate direction. An information processing device that detects a motion of a user includes a detecting unit that detects the motion of the user and a face direction of the user from an image photographed by an imaging unit, a detected motion determining unit that determines whether or not the motion detected by the detecting unit is an operation on the information processing device based on the face direction of the user and position information of the information processing device stored in a storage unit, and a display content control unit that causes a display unit to reflect the motion when the detected motion determining unit determines that the motion detected by the detecting unit is the operation on the information processing device.

Claims

1. An information processing device that detects a motion of a user, comprising:

a detecting unit that detects the motion of the user and a face direction of the user from an image photographed by an imaging unit;

a detected motion determining unit that determines whether or not the motion detected by the detecting unit is an operation on the information processing device based on the face direction of the user and position information of the information processing device stored in a storage unit in advance; and

a display content control unit that causes a display unit to reflect the motion when the detected motion determining unit determines that the motion detected by the detecting unit is the operation on the information processing device.

6. The information processing device according to claim 1,

wherein, when a line of sight of the user has not been detected or when the line of sight of the user has not been detected continuously, the detected motion determining unit counts the number of non-detection, and when the number of non-detection reaches a predetermined value, the determining based on the face direction is performed before the determining based on the line-of-sight direction.

7. The information processing device according to claim 1,

wherein the display content control unit decides an area including the position information based on a relative position between the user and the information processing device or characteristics of the user.

8. The information processing device according to claim 1,

wherein the detected motion determining unit chronologically accumulates and records coordinate data of the face direction in the storage unit, and even when the motion detected by the detecting unit is determined to be not the operation on the information processing device, if an immediate previous or next face direction stored in the storage unit is a direction of the information processing device, the detected motion determining unit determines that the motion detected by the detecting unit is the operation on the information processing device.

2. An information processing device that detects a motion of a user, comprising:

a detecting unit that detects the motion of the user from an image photographed by an imaging unit;

a gaze area calculating unit that calculates a gaze area of the user based on a line-of-sight direction and a distance of sight of the user;

a virtual image area calculating unit that causes a display unit to display a virtual image of the information processing device, and calculates a virtual image area serving as an area of the displayed virtual image;

a detected motion determining unit that determines whether or not the motion detected by the detecting unit is an operation on the information processing device based on the gaze area calculated by the gaze area calculating unit and the virtual image area calculated by the virtual image area calculating unit; and

a display content control unit that causes the display unit to reflect the motion when the detected motion determining unit determines that the motion detected by the detecting unit is the operation on the information processing device.

3. The information processing device according to claim 2,

wherein the detecting unit further detects a face direction of the user, and

the detected motion determining unit determines whether or not the motion detected by the detecting unit is the operation on the information processing device based on the face direction of the user and the virtual image area.

9. The information processing device according to claim 2,

wherein the detected motion determining unit chronologically accumulates and records coordinate data of the line-of-sight direction and the distance of sight in a storage unit, and even when the motion detected by the detecting unit is determined to be not the operation on the information processing device, if an immediately previous or next line-of-sight direction and an immediately previous or next distance of sight stored in the storage unit are a direction of the information processing device, the detected motion determining unit determines that the motion detected by the detecting unit is the operation on the information processing device.

4. An information processing device that detects a motion of a user, comprising:

a first detecting unit that detects the motion of the user from an image photographed by an imaging unit photographing the user;

a second detecting unit that detects a moving object from an outside image photographed by an outside imaging unit photographing an outside;

a gaze area calculating unit that calculates a gaze area of the user based on a line-of-sight direction and a distance of sight of the user;

a moving object position calculating unit that calculates a position of the moving object detected by the second detecting unit;

a detected motion determining unit that determines whether or not the motion detected by the first detecting unit is an operation on the information processing device based on the gaze area and the position of the moving object; and

a display content control unit that causes a display unit to reflect the motion when the detected motion determining unit determines that the motion detected by the detecting unit is the operation on the information processing device.

5. The information processing device according to claim 4,

wherein the first detecting unit further detects a face direction of the user;

and

the detected motion determining unit determines whether or not the motion detected by the first detecting unit is the operation on the information processing device based on the face direction of the user and the position of the moving object.

10. A gesture detection method of detecting a motion of a user, comprising:

a detecting step of detecting the motion of the user and a face direction of the user from an image photographed by an imaging unit;

a detected motion determining step of determining whether or not the motion detected in the detecting step is an operation on an information processing device based on the face direction of the user and position information of the information processing device stored in a storage unit in advance; and

a display content control step of causing a display unit to reflect the motion when the motion detected in the detecting step is determined to be the operation on the information processing device in the detected motion determining step.

11. A gesture detection program causing a computer to execute:

a detecting step of detecting the motion of the user and a face direction of the user from an image photographed by an imaging unit;

a detected motion determining step of determining whether or not the motion detected in the detecting step is an operation on an information processing device based on the face direction of the user and position information of the information processing device stored in a storage unit in advance; and

a display content control step of causing a display unit to reflect the motion when the motion detected in the detecting step is determined to be the operation on the information processing device in the detected motion determining step.