Interface method and apparatus for inputting information with air finger gesture

• US 9,292,093 B2
• Filed: 11/18/2011
• Issued: 03/22/2016
• Est. Priority Date: 11/18/2010
• Status: Active Grant

• Current Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics Incorporated (Alps Alpine Co., Ltd.)
• Inventors: Iao, Mike
• Primary Examiner(s): Lao, Lun-Yi
• Assistant Examiner(s): Khan, Ibrahim
• Application Number:
  US13/299,958
• Publication Number:
  US 20130076615A1
• Time in Patent Office: 1,586 Days
• Field of Search: 345/156, 701/400, 701/426, 701/438, 715/863
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 2360/782 on the steering wheel
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: G01C 21/3605 Destination input or retrieval
• CPC Class Codes: G01C 21/3623 using a camera or code reader, e.g. for optical or magnetic codes
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

An interface apparatus and method is disclosed for inputting information with a user'"'"'s finger gesture while the user is driving with his two hands on a steering wheel. In one aspect, the interface apparatus includes a gesture sensor, a gesture processor, and a head-up display (HUD), where the gesture sensor and the gesture processor recognizes and interpret the information input by the user'"'"'s finger gesture and such information is displayed on the HUD. In one embodiment, a plurality of point of interest (POI) icons are displayed on the HUD after the user inputs at least one letter into the system and the user can select the POI by his/her finger gesture. In another embodiment, the gesture sensor can recognize the user'"'"'s finger gesture in non-alphabet characters.

Claims

1. An interface apparatus for an electronics system, comprising:

a gesture sensor provided near a steering wheel of a vehicle to capture a user'"'"'s finger gesture;

a head-up display configured to display information on a vehicle windshield and to provide feedback in response to the user'"'"'s finger gesture captured by the gesture sensor,

a gesture processor communicably coupled to the gesture sensor and configured to recognize the user'"'"'s finger gesture that is received from the gesture sensor, to convert at least a portion of the recognized user'"'"'s finger gesture into at least one letter, and to interpret the converted one letter to produce a list of entries or options, the entries or options comprising the one letter as a first letter, in order to cause the head-up display to display the information for selection based on the interpreted gesture;

a light adjustment unit for maintaining a constant lighting level in an area defined as a finger gesture area, where the finger gesture area is a limited space close to the steering wheel within which finger gestures can be made without releasing a user'"'"'s hand performing the finger gesture from the steering wheel; and

at least one touch sensor on a steering wheel configured to detect whether a user'"'"'s hand is placed on the steering wheel and if so, the position of the hand on the steering wheel;

wherein the list of entries is a plurality of candidate points of interest based on a current vehicle position for optimum routing information, and wherein the gesture sensor and the light adjustment unit are mounted on a dashboard of the vehicle close to the steering wheel, and further wherein the recognition of the user'"'"'s finger gesture is conditioned upon the detection of the user'"'"'s hand is placed on the steering wheel detected by the touch sensor.

2. The interface apparatus as defined in claim 1, wherein the gesture sensor is a video camera or a motion sensor.

3. The interface apparatus as defined in claim 1, wherein the head-up display shows a list of entries or options in response to a character drawn by the finger gesture of the user.

4. The interface apparatus as defined in claim 3, wherein the head-up display is configured to show available options in response to the information input by the user'"'"'s finger gesture, the gesture sensor is configured to detect a direction of the user'"'"'s finger, and the finger gesture processor is configured to interpret which of the available options on the head-up display has been pointed by the direction of the user'"'"'s finger.

5. The interface apparatus as defined in claim 1, further comprising:

a steering wheel turn sensor that detects a movement of the steering wheel and sending a detection signal to the gesture processor;

wherein the gesture processor corrects the user'"'"'s finger gesture in response to the detection signal indicating the movement of the steering wheel.

6. The interface apparatus as defined in claim 1, further comprising:

a button located on or close to the steering wheel, which works in conjunction with the gesture sensor to indicate start and stop of sending the detected finger gesture to the finger gesture processor.

7. The interface apparatus as defined in claim 1, wherein the gesture processor receives a detection signal from the touch sensor and allows an input operation by the finger gesture when the user'"'"'s both hands are placed on the steering wheel.

8. The interface apparatus as defined in claim 1, wherein only the finger gestures made within the finger gesture area are validated.

9. The interface apparatus as defined in claim 1, wherein the light adjustment unit is comprised of:

a light sensor for detecting lighting intensity at the finger gesture area in which the user performs the finger gesture; and

a light source that receives a signal from the light sensor and generates light for adjusting the lighting level at the finger gesture area to be constant thereby improving detection accuracy and stability of the finger gesture.

10. The interface apparatus as defined in claim 1, further comprising:

a finger gesture database for storing data related to the finger gesture;

wherein the gesture processor interprets the finger gesture detected by the gesture sensor based on the data retrieved from the finger gesture database.

11. The interface apparatus as defined in claim 1, wherein the electronics system is a vehicle navigation system, and wherein the finger gesture by the user specifies entries related to functions of the navigation system including a selection of a destination.

12. An interface method of inputting information for an electronics system, comprising the following steps of:

capturing a user'"'"'s finger gesture by a gesture sensor provided near a steering wheel of a vehicle;

recognizing the user'"'"'s finger gesture captured by the gesture sensor by a finger gesture processor, to convert at least a portion of the recognized user'"'"'s finger gesture into at least one letter, and to interpret the converted one letter to produce a list of entries or options, the entries or options comprising the one letter as a first letter, in order;

displaying, on a head-up display, information for selection to provide feedback to the user in response to the user'"'"'s finger gesture;

maintaining a constant lighting level in an area defined as a finger gesture area by a light adjustment unit, where the finger gesture area is a limited space close to the steering wheel within which finger gestures can be made without releasing a user'"'"'s hand performing the finger gesture from the steering wheel; and

detecting, by a touch sensor, whether a user'"'"'s hand is placed on the steering wheel of the vehicle and if so, the position of the hand on the steering wheel;

wherein the list of entries is a plurality of candidate points of interest based on a current vehicle position for optimum routing information, and wherein the gesture sensor and the light adjustment unit are mounted on a dashboard of the vehicle close to the steering wheel, and further wherein the recognition of the user'"'"'s finger gesture is conditioned upon the detection of the user'"'"'s hand placed on the steering wheel detected by the touch sensor.

13. The method of inputting information as defined in claim 12, further comprising a step of:

receiving a signal generated by a button located on or close to the steering wheel;

wherein the button work in conjunction with the gesture sensor to indicate start and stop of sending the detected finger gesture to the finger gesture processor.

14. The interface method of inputting information as defined in claim 12, further comprising the following steps of:

displaying a plurality of information entries on the head-up display; and

interpreting a direction of the user'"'"'s finger to determine which information entry is selected.

15. The method of inputting information as defined in claim 12, further comprising a step of:

validating only the finger gestures made within the finger gesture area.

16. The method of inputting information as defined in claim 12, further comprising the following steps of:

detecting a movement of the steering wheel by a steering wheel turn sensor and sending a detection signal to the gesture processor; and

correcting, by the gesture processor, the user'"'"'s finger gesture in response to the detection signal indicating the movement of the steering wheel.

17. The method of inputting information as defined in claim 12, further comprising a step of:

allowing an input operation by the finger gesture when the user'"'"'s both hands are placed on the steering wheel.

18. The method of inputting information as defined in claim 12, wherein the electronics system is a vehicle navigation system, and wherein the finger gesture by the user specifies entries related to functions of the navigation system including a selection of a destination.