Touch gesture based interface for motor vehicle

• US 7,295,904 B2
• Filed: 08/31/2004
• Issued: 11/13/2007
• Est. Priority Date: 08/31/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Viswanathan, Mahesh, Kanevsky, Dimitri, Sicconi, Roberto
• Primary Examiner(s): NGUYEN, TAN QUANG
• Application Number:
  US10/930,225
• Publication Number:
  US 20060047386A1
• Time in Patent Office: 1,169 Days
• Field of Search: 701/1, 701/36, 701/49, 701/41, 345/173, 345/156, 340/426.35, 704/275, 382/103, 382/104
• US Class Current: 701/36
• CPC Class Codes: B60K 2360/143 Touch sensitive instrument input devices
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G06F 2203/04809 Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

An improved apparatus and method is provided for operating devices and systems in a motor vehicle, while at the same time reducing vehicle operator distractions. One or more touch sensitive pads are mounted on the steering wheel of the motor vehicle, and the vehicle operator touches the pads in a pre-specified synchronized pattern, to perform functions such as controlling operation of the radio or adjusting a window. At least some of the touch patterns used to generate different commands may be selected by the vehicle operator. Usefully, the system of touch pad sensors and the signals generated thereby are integrated with speech recognition and/or facial gesture recognition systems, so that commands may be generated by synchronized multi-mode inputs.

Claims

1. Apparatus for selectively controlling one or more systems included with a motor vehicle driven by a vehicle operator, said apparatus comprising:

one or more touch pad devices respectively mounted to receive a specified touch gesture pattern from said operator while said vehicle is being driven, said touch pad devices responsive to said received touch gesture pattern to generate a corresponding pattern of touch pad signals;

an interpretation module adapted to recognize that said corresponding pattern of touch pad signals is associated with a pre-specified command for selectively operating a particular one of said systems;

an execution module disposed to receive said pre-specified command from said interpretation module, and to operate said particular system in accordance with said pre-specified command; and

means for detecting occurrences of an extreme situation and for causing said execution module to disregard any commands until said apparatus is positively informed that said extreme situation has ended.

2. The apparatus of claim 1, wherein:

said extreme situation is indicated by the occurrence of one of a number of pre-specified actions of said operator, wherein said pre-specified actions include, at least, rapidly applying the brakes and rapidly turning the steering wheel of said vehicle, selectively.

3. The apparatus of claim 1, wherein:

at least one device having a mode of operation different from the mode of operation of said touch pad devices is provided for use by said operator to generate one or more multi-mode signals; and

said interpretation module is disposed to route said pre-specified command to said execution module upon receiving both said corresponding pattern of touch pad signals, and a specified one of said multi-mode signals.

4. The apparatus of claim 3, wherein:

said apparatus includes a speech recognition device disposed to encode a spoken instruction of said operator in a signal, in order to provide one of said multi-mode signals.

5. The apparatus of claim 4, wherein:

said apparatus further includes a facial gesture recognition device disposed to analyze a viewed facial gesture of said operator, and to encode the analyzed facial gesture in a signal to provide another of said multi-mode signals.

6. The apparatus of claim 5, wherein:

said speech and facial gesture recognition devices respectively provide confidence correlation scores of low, mid range or high value for each of said multi-mode signals.

7. The apparatus of claim 6, wherein:

said interpretation module stops further action to execute a command associated with a multi-mode signal of low correlation score, arranges for execution of a command associated with a multi-mode signal of high correlation score, and requests the operator to confirm a command associated with a mid range correlation score.

8. The apparatus of claim 1, wherein:

at least two of said touch pad devices are mounted on the steering wheel of said vehicle to collectively receive a pre-specified pattern of synchronized touch gestures from said operator.

9. The apparatus of claim 8, wherein:

each touch pad is provided with a touch sensitive surface.

10. The apparatus of claim 9 wherein:

said pattern of synchronized touch gestures is produced by touching each of said touch pads in a specified sequence during a specified time window.

11. The apparatus of claim 9, wherein:

said touch gesture pattern is produced at least in part by moving a selected object along a specified path in contact with the touch sensitive surface of one of said touch pads.

12. The apparatus of claim 9, wherein:

tactile reference indicia are placed at pre-specified locations on at least one of said touch sensitive surfaces.

13. The apparatus of claim 9, wherein:

said touch gesture pattern is produced at least in part by enabling said operator to simultaneously apply pressure at two spatially separated locations on one of said touch sensitive surfaces, using two different fingers.

14. The apparatus of claim 9, wherein:

each of said touch pads is mounted to receive touch gestures from a thumb of said operator while said operator continues to hold said steering wheel.

15. The apparatus of claim 1, wherein:

said apparatus includes a feedback device coupled to said interpretation module for requesting that said operator confirm an intent to generate and execute one of said commands.

16. The apparatus of claim 15, wherein:

said feedback device is disposed to electronically generate a vocal statement.

17. The apparatus in claim 1, wherein:

said apparatus includes means for acknowledging that it has received said specified touch gesture pattern.

18. The apparatus of claim 17, wherein:

said acknowledging means comprises an electronic device for producing a vocal statement indicating that said specified touch gesture pattern has been received by said apparatus.

19. The apparatus of claim 17, wherein:

said acknowledging means comprises one or more vibrators mounted on said steering wheel, and further includes means for actuating said vibrators in response to receipt of said specified touch gesture pattern.

20. The apparatus of claim 19, wherein:

one of said vibrators is placed adjacent to each of said touch pad devices on said steering wheel.

21. The apparatus of claim 20, wherein:

one of said vibrators is placed underneath each of said touch pad devices on said steering wheel.

22. The apparatus of claim 17, wherein:

said acknowledging means comprises a visual indicator, and further includes means for actuating said visual indicator in response to said receipt of said specified touch gesture pattern.

23. The apparatus of claim 22, wherein:

said visual indicator comprises a light disposed to flash in a specified pattern in response to said specified touch gesture pattern.

24. The apparatus of claim 1, wherein:

a comparator is provided to compare a command received from said interpretation module with the current status of a vehicle system associated with said command.

25. The apparatus of claim 1, wherein:

a training module is connected to said interpretation module for use by said operator to enter operator selected touch gesture patterns, and commands respectively associated therewith, into said interpretation module.

26. The apparatus of claim 1, wherein:

said apparatus includes means for determining whether or not a set of received touch gestures has been provided by a person who is allowed to operate said motor vehicle.

27. A method for selectively controlling one or more systems included with a motor vehicle driven by a vehicle operator comprising the steps of:

applying a touch gesture pattern to one or more touch pad devices, while said vehicle is being driven, to generate a corresponding pattern of touch pad signals;

analyzing said corresponding pattern of touch pad signals to identify an associated pre-specified command for selectively operating a particular one of said systems;

generating a signal to operate said particular system in accordance with said associated pre-specified command; and

detecting occurrences of an extreme situation, and in response thereto causing said particular system to disregard any commands until said extreme situation has ended.

28. The method of claim 27, further including:

operating at least one device having a mode of operation different from the mode of operation of said touch pad devices to generate one or more multi-mode signals; and

executing said pre-specified command in response to both said corresponding pattern of touch pad signals, and a specified one of said multi-mode signals.

29. The method of claim 27, wherein:

said touch gesture pattern is applied to at least two of said touch pad devices mounted on the steering wheel of said vehicle.

30. The method of claim 27, wherein:

a feedback device is operated to request that said operator confirm an intent to generate and execute one of said commands.

31. Apparatus for selectively controlling one or more systems included with a motor vehicle having a steering wheel and driven by a vehicle operator, said apparatus comprising:

at least two touch pad devices respectively mounted on the steering wheel of said vehicle, wherein each touch pad receives at least one touch gesture from said operator to generate a specified pattern comprising multiple touch pad signals;

an interpretation module adapted to recognize that said specified pattern of touch pad signals is associated with a pre-specified command for selectively operating a particular one of said systems; and

an execution module disposed to receive said pre-specified command from said interpretation module, and to operate said particular system in accordance with said pre-specified command.

32. The apparatus of claim 31, wherein:

a training module is connected to said interpretation module for use by said operator to enter operator selected touch gesture patterns, and commands respectively associated therewith, into said interpretation module.

33. The apparatus of claim 31, wherein:

at least one device having a mode of operation different from the mode of operation of said touch pad devices is provided for use by said operator to generate one or more multi-mode signals; and

said interpretation module is disposed to route said pre-specified command to said execution module upon receiving both said specified pattern of touch pad signals, and a specified one of said multi-mode signals.

34. The apparatus of claim 31, wherein:

said apparatus includes means for detecting occurrences of an extreme situation and for causing said execution module to disregard any command until said apparatus is positively informed that said extreme situation has ended.