Interface for car-mounted devices

• US 20050134117A1
• Filed: 10/28/2004
• Published: 06/23/2005
• Est. Priority Date: 12/17/2003
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Kumazawa, Yuuichi, Ito, Takafumi
• Granted Patent: US 7,437,488 B2
• Time in Patent Office: Days
• US Class Current: 307/10.100
• CPC Class Codes: B60R 16/0373 Voice control in general G10L
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image

Abstract

An interface for car-mounted devices includes hand data detecting means for determining devices to be controlled among the car-mounted devices based upon hand data corresponding to an operator hand state, voice recognizing means for determining how to change the state of the devices to be controlled based upon an operator voice utterance, and control signal-forming means for forming control signals for the devices to be controlled relying upon the devices to be controlled determined by said hand data detecting means and upon the content of change determined by said voice recognizing means.

Claims

1. An interface for car-mounted devices comprising:

hand data detecting means for determining devices to be controlled among the car-mounted devices based upon hand data corresponding to an operator hand state;

voice recognizing means for determining how to change the state of the devices to be controlled based upon an operator voice utterance; and

control signal-forming means for forming control signals for the devices to be controlled relying upon the devices to be controlled determined by said hand data detecting means and upon the content of change determined by said voice recognizing means.

2. An interface for car-mounted devices according to claim 1, wherein the operator hand state expressed by the hand data includes at least any one of a shape of operator hand, a motion of operator hand or a position of operator hand.

3. An interface for car-mounted devices according to claim 1, wherein the control signal-forming means evaluates a combination of the device to be controlled determined by the hand data detecting means and the manner of changing the state determined by the voice recognizing means, and invalidates the control signal that is formed when the device to be controlled is changed into a state that cannot be executed.

4. An interface for car-mounted devices according to claim 1, wherein the voice recognizing means determines the manner of changing the state of the device to be controlled when the operator voice utterance is in match with any one of candidates that have been provided in advance for changing the state of the device to be controlled, and sends, to the hand data detecting means, a signal of determination as a trigger for determining the operator hand state of the hand data detecting means.

5. An interface for car-mounted devices according to claim 1, wherein, when a demonstrative pronoun that has been set in advance is recognized by the voice recognizing means, the control signal-forming means specifies the device to be controlled based on the hand data detected by the hand data detecting means, and forms a control signal for changing over the operating state of the device to be controlled.

6. An interface for car-mounted devices according to claim 1, wherein, when the device to be controlled is specified based on the hand data detected by the hand data detecting means and when the reserved word corresponded to the operation for the device to be controlled is recognized by the voice recognizing means, the control signal-forming means forms a control signal for having the device to be controlled execute the operation corresponding to the reserved word.

7. An interface for car-mounted devices according to claim 1, wherein the control signal-forming means specifies the device to be controlled based on the hand data detected by the hand data detecting means when a preset demonstrative pronoun is recognized by the voice recognizing means, and forms a control signal for having the device to be controlled execute the operation corresponding to the reserved word when the reserved word corresponded to the operation for the device to be controlled is recognized by the voice recognizing means during an allowable time that is set in advance after the device to be controlled is specified.

8. An interface for car-mounted devices according to claim 1, wherein the control signal-forming means specifies the device to be controlled based on the hand data detected by the hand data detecting means when a preset demonstrative pronoun is recognized by the voice recognizing means, specifies the operation for the device to be controlled based on the hand data detected by the hand data detecting means, and forms a control signal for having the device to be controlled execute the operation when the preset demonstrative pronoun is recognized again by the voice recognizing means during an allowable time that is set in advance after the device to be controlled is specified.

9. An interface for car-mounted devices according to claim 1, further comprising operation permission means for permitting the operation of the hand data detecting means or the control signal-forming means when an operator hand position is detected or when the operator hand position exists in a predetermined specified region.

10. An interface for car-mounted devices according to claim 9, wherein the operation permission means detects the operator hand position based on the input image from the camera arranged so as to image the specified region.

11. An interface for car-mounted devices according to claim 1, wherein the hand data detecting means includes:

extracting means for extracting a hand portion which is a region imaging the operator hand in an input image based on the input image from a camera imaging the operator hand; and

shape specifying means for comparing a shape of the hand portion picked up by the extracting means with shapes of templates that have been provided in advance to specify a shape of the most matched template as the shape of hand;

wherein the shape of hand specified by the shape specifying means is used as the hand data.

12. An interface for car-mounted devices according to claim 1, wherein the hand data detecting means includes:

extracting means for extracting a hand portion which is a region imaging the operator hand in an input image based on the input image from a camera imaging the operator hand; and

position specifying means for specifying a center of gravity of the hand portion picked up by the extracting means as a position of hand;

wherein the position of hand specified by the position specifying means is used as the hand data.

13. An interface for car-mounted devices according to claim 1, wherein the hand data detecting means includes:

extracting means for extracting a hand portion which is a region imaging the operator hand in a input image based on the input image from a camera imaging the operator'"'"'s hand;

position specifying means for specifying a center of gravity of the hand portion picked up by the extracting means as a position of hand; and

motion specifying means for specifying a time-series change in the position of hand specified by the position specifying means as the motion of hand;

wherein the motion of hand specified by the motion specifying means is used as the hand data.

14. An interface for car-mounted devices according to claim 11, wherein the extracting means uses at least either the brightness or the color data of pixels constituting the input image as the data for extracting, and uses, as the hand portion, a region in an allowable range in which the data for extracting have been set in advance in the input image.

15. An interface for car-mounted devices according to claim 14, wherein the hand data extracting means finds a difference between time-series frames in the input image to pick up a region where there is a motion in the input image, and sets an allowable range of data for extracting based on an average value of data for extracting in the region that is picked up.

16. An interface for car-mounted devices according to claim 1, wherein the hand data detecting means detects the hand data in a three-dimensional manner based on the input images from a plurality of cameras arranged at different positions.

17. An interface for car-mounted devices according to claim 1, wherein a control signal formed by the control signal-forming means is transmitted to the devices to be controlled through a communication network connecting the car-mounted devices to each other.

18. An interface for car-mounted devices according to claim 12, wherein the extracting means uses at least either the brightness or the color data of pixels constituting the input image as the data for extracting, and uses, as the hand portion, a region in an allowable range in which the data for extracting have been set in advance in the input image.

19. An interface for car-mounted devices according to claim 13, wherein the extracting means uses at least either the brightness or the color data of pixels constituting the input image as the data for extracting, and uses, as the hand portion, a region in an allowable range in which the data for extracting have been set in advance in the input image.