Human machine interface apparatus for vehicle and methods of controlling the same

• US 10,180,729 B2
• Filed: 12/08/2014
• Issued: 01/15/2019
• Est. Priority Date: 10/06/2014
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Park, Hyungsoon
• Primary Examiner(s): Pan, Yongjia
• Application Number:
  US14/563,811
• Publication Number:
  US 20160098088A1
• Time in Patent Office: 1,499 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/111 for controlling multiple devices
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1464 3D-gesture
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A human machine interface (HMI) minimizing the number of gestures for operation control in which user-intended operation commands are accurately recognized by dividing a vehicle interior into a plurality of regions. The HMI receives an input of a gesture according to each region, and controls any one device according to the gesture. Convenience of a user is improved because the gesture may be performed in a state in which region restriction is minimized by identifying an operation state and an operation pattern of an electronic device designated according to each region to recognize the user'"'"'s intention when the user performs the gesture in a boundary portion between two regions or even in multiple regions.

Claims

1. A vehicle having a human machine interface (HMI) apparatus configured to output an operation command of a user to at least one of a plurality of electronic devices, the HMI apparatus comprising:

a storage unit configured to store position information of a plurality of regions, operation commands of the plurality of electronic devices positioned in the plurality of regions, and gesture information of the operation commands;

a recognition unit configured to recognize a gesture of the user; and

a control unit configured to identify electronic devices installed in a region adjacent to a boundary region when the gesture is recognized in the boundary region of the plurality of regions, determine adjustment possibilities of the identified electronic devices based on at least one of operation information and pre-stored operation patterns of the identified electronic devices, traveling information of the vehicle, and environmental information of the vehicle, and control the operation command corresponding to the gesture to be output to an electronic device having a highest adjustment possibility among the determined adjustment possibilities, wherein

when an operation of the identified electronic devices is controlled by the operation command corresponding to the recognized gesture, the control unit is further configured to determine whether the operation of any of the identified electronic devices deviates from the pre-stored operation patterns and determine an electronic device which operates within the pre-stored operation patterns as the electronic device having the highest adjustment possibility,

the control unit is further configured to identify the electronic devices corresponding to the gesture according to each adjacent region, identify values of functions corresponding to the respective operation patterns of the identified electronic devices, compare the identified values of functions corresponding to the respective operation patterns of the identified electronic devices with values of functions corresponding to respective current operation information of the identified electronic devices, and based on the comparison, determine an electronic device with the largest difference as the electronic device having the highest adjustment possibility, and

the boundary region includes a region within a given range at a position at which at least two regions of the plurality of regions are adjacent to each other.

2. The vehicle according to claim 1, wherein the control unit identifies priorities of the electronic devices having the gesture according to each adjacent region and outputs the operation command corresponding to the gesture to an electronic device having highest priority.

3. The vehicle according to claim 1, wherein:

the plurality of electronic devices include at least two devices of an air conditioning device, an audio device, a navigation device, a lighting device, a Bluetooth device, a heater within a seat, a window glass opening/closing device, a sunroof opening/closing device, a door opening/closing device, and a door lock device, and

the operation command includes a power ON/OFF command and a function adjustment command.

4. The vehicle according to claim 1, wherein, when the gesture is recognized in one region, the control unit identifies an electronic device having the gesture among electronic devices installed in the one region and controls the operation command corresponding to the gesture to be output to the identified electronic device.

5. The vehicle according to claim 1, further comprising a detection unit configured to detect an object for the gesture.

6. A vehicle having a plurality of electronic devices, the vehicle comprising:

a plurality of detection units arranged in a plurality of regions; and

a human machine interface (HMI) apparatus configured to output an operation command of a user to the plurality of electronic devices arranged in the plurality of regions, wherein

the HMI apparatus includes;

a storage unit configured to store position information of the plurality of regions, operation commands of the plurality of electronic devices arranged in the plurality of regions, and gesture information of the operation commands;

a recognition unit configured to recognize a gesture of the user; and

a control unit configured to identify a region in which a detection unit recognizing the gesture is arranged when the gesture is identically recognized in the plurality of detection units, identify electronic devices installed in the identified region, determine adjustment possibilities of the identified electronic devices based on at least one of operation information and pre-stored operation patterns of the identified electronic devices, traveling information of the vehicle, and environmental information of the vehicle, and control the operation command corresponding to the gesture to be output to an electronic device having a highest adjustment possibility among the determined adjustment possibilities,

when an operation of the identified electronic devices is controlled by the operation command corresponding to the recognized gesture, the control unit is further configured to determine whether the operation of any of the identified electronic devices deviates from the pre-stored operation patterns and determine an electronic device which operates within the pre-stored operation patterns as the electronic device having the highest adjustment possibility, and

the control unit is further configured to identify the electronic devices corresponding to the gesture according to each adjacent region, identify values of functions corresponding to the respective operation patterns of the identified electronic devices, compare the identified values of functions corresponding to the respective operation patterns of the identified electronic devices with values of functions corresponding to respective current operation information of the identified electronic devices, and based on the comparison, determine an electronic device with the largest difference as the electronic device having the highest adjustment possibility.

7. A control method of a vehicle having a plurality of electronic devices and an HMI apparatus configured to output an operation command of a user to at least one of the plurality of electronic devices, the control method comprising the steps of:

detecting an object;

recognizing a detection region and a gesture of the detected object;

identifying electronic devices installed in a region adjacent to a boundary region when it is determined that the gesture is recognized in the boundary region of a plurality of regions;

determining adjustment possibilities of the identified electronic devices based on at least one of operation information and pre-stored operation patterns of the identified electronic devices, traveling information of the vehicle, and environmental information of the vehicle;

when an operation of the identified electronic devices is controlled by the operation command corresponding to the recognized gesture, determining whether the operation of any of the identified electronic devices deviates from the pre-stored operation patterns;

determining an electronic device which operates within the pre-stored operation patterns as an electronic device having a highest adjustment possibility among the determined adjustment possibilities; and

outputting the operation command corresponding to the electronic device having the highest adjustment possibility, wherein

the boundary region includes a region within a given range at a position at which at least two regions of the plurality of regions are adjacent to each other, and

the determining adjustment possibilities of the identified electronic devices comprises;

identifying the electronic devices corresponding to the gesture according to each adjacent region,

identifying values of functions corresponding to the respective operation patterns of the identified electronic devices,

comparing the identified values of functions corresponding to the respective operation patterns of the identified electronic devices with values of functions corresponding to respective current operation information of the identified electronic devices, and

based on the comparison, determining an electronic device with the largest difference as the electronic device having the highest adjustment possibility.

8. The control method of the vehicle according to claim 7, wherein the outputting of the operation command corresponding to the gesture to any one of the identified electronic devices includes:

identifying priorities of the electronic devices having the gesture according to each adjacent region; and

outputting the operation command corresponding to the gesture to an electronic device having highest priority.

9. The control method of the vehicle according to claim 7, wherein the outputting of the operation command corresponding to the gesture to any one of the identified electronic devices includes:

identifying operation information of electronic devices corresponding to the gesture according to each adjacent region;

receiving traveling information and environmental information of the vehicle;

determining adjustment possibilities of the identified electronic devices based on the operation information of the identified electronic devices, the traveling information, and the environmental information; and

outputting the operation command corresponding to the gesture to an electronic device having a highest adjustment possibility.

10. The control method of the vehicle according to claim 7, wherein the outputting of the operation command corresponding to the gesture to any one of the identified electronic devices includes:

identifying operation information and operation patterns of the identified electronic devices according to each adjacent region;

determining possibilities that operations of the identified electronic devices will be adjusted based on the operation information and the operation patterns of the identified electronic devices; and

outputting the operation command corresponding to the gesture to an electronic device having a highest adjustment possibility.

11. The control method of the vehicle according to claim 7, wherein:

the plurality of electronic devices include at least two devices of an air conditioning device, an audio device, a navigation device, a lighting device, a Bluetooth device, a heater within a seat, a window glass opening/closing device, a sunroof opening/closing device, a door opening/closing device, and a door lock device, and

the operation command includes a power ON/OFF command and a function adjustment command.

12. The control method of the vehicle according to claim 7, further comprising:

identifying, when the gesture is recognized in one region, an electronic device corresponding to the gesture among electronic devices installed in the one region; and

outputting the operation command corresponding to the gesture to the identified electronic device.

13. A human machine interface (HMI) apparatus comprising:

a storage unit configured to store position information of a plurality of regions, information of a plurality of electronic devices arranged in the plurality of regions, and gesture information of a user corresponding to operation commands of the plurality of electronic devices;

a recognition unit configured to recognize a gesture of the user;

a control unit configured to identify electronic devices corresponding to a gesture according to each recognized detection region when the number of recognized detection regions is at least two, determine adjustment possibilities of the electronic devices based on current operation information of the identified electronic devices, determine adjustment possibilities of the identified electronic devices based on at least one of operation information of the identified electronic devices, pre-stored operation patterns of the identified electronic devices, and environmental information, and output an operation command corresponding to the gesture to an electronic device having a highest adjustment possibility among the determined adjustment possibilities; and

an output unit connected to each of the plurality of electronic devices and configured to output the operation command to at least one electronic device, wherein

when an operation of the identified electronic devices is controlled by the operation command corresponding to the recognized gesture, the control unit is further configured to determine whether the operation of any of the identified electronic devices deviates from the pre-stored operation patterns and determine an electronic device which operates within the pre-stored operation patterns as the electronic device having the highest adjustment possibility, and

the control unit is further configured to identify the electronic devices corresponding to the gesture according to each adjacent region, identify values of functions corresponding to the respective operation patterns of the identified electronic devices, compare the identified values of functions corresponding to the respective operation patterns of the identified electronic devices with values of functions corresponding to respective current operation information of the identified electronic devices, and based on the comparison, determine an electronic device with the largest difference as the electronic device having the highest adjustment possibility.

14. The HMI apparatus according to claim 13, wherein, when the number of recognized detection regions is one, the control unit identifies an electronic device having the recognized gesture among electronic devices installed in the detection region and outputs the operation command corresponding to the gesture to the identified electronic device.

15. The HMI apparatus according to claim 13, further comprising a detection unit configured to detect an object for recognizing the gesture of the user,

wherein the recognition unit receives a detection signal from the detection unit configured to detect the object for recognizing the gesture of the user.