Interactive control device and method for operating the interactive control device

• US 9,671,867 B2
• Filed: 03/22/2007
• Issued: 06/06/2017
• Est. Priority Date: 03/22/2006
• Status: Active Grant

• Current Assignee: Volkswagen AG
• Original Assignee: Volkswagen AG
• Inventors: Bachfischer, Katharina, Waeller, Christoph, Dehmann, Rainer, Wagner, Volkmar
• Primary Examiner(s): CHOI, DAVID E
• Application Number:
  US12/294,022
• Publication Number:
  US 20090327977A1
• Time in Patent Office: 3,729 Days
• Field of Search: 715863, 715205, 345 13, 345 7
• CPC Class Codes: B60K 2360/141 Activation of instrument input devices by approaching fingers or pens
• CPC Class Codes: B60K 2360/143 Touch sensitive instrument input devices
• CPC Class Codes: B60K 2360/1438 Touch screens
• CPC Class Codes: B60K 2370/141 Activation by approaching fingers or pens
• CPC Class Codes: B60K 2370/143 Touch sensitive input devices
• CPC Class Codes: B60K 2370/1438 Touch screens
• CPC Class Codes: B60K 2370/736 the user being the driver
• CPC Class Codes: B60K 2370/739 the user being the passenger
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60K 35/654 the user being the driver
• CPC Class Codes: B60K 35/656 the user being a passenger
• CPC Class Codes: B60K 37/00 Dashboards as road-vehicle superstructure sub-units B62D25/14
• CPC Class Codes: B60R 11/0235 of flat type, e.g. LCD
• CPC Class Codes: B60R 11/0264 for control means
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04812 : Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text

Abstract

An interactive control device includes a display device, and a method is for operating the interactive control device. The method includes: displaying graphical information on the display device; receiving sensor information; activating a control action if on the basis of the sensor information it is ascertained that a body part of a user is located within an activation region that is spatially defined relative to a display region of a control element on the display device with which the control action is associated; the received sensor information including user information that is evaluated prior to an activation of the control action in order to ascertain a control intention for the at least one control element; and the information represented on the display device being adapted as a function of the ascertained control intention such that the at least one control element is represented in a manner optimized for the activation of the control action associated with the control element. The control device may be arranged as a component of a motor vehicle console so as to be able to implement the control method.

Claims

1. A method for operating an interactive control device, the control device including a display device configured to represent information including control elements, comprising:

ascertaining a control probability based on an ascertained control intention for each of a plurality of control elements represented on the display device, from a movement of a user'"'"'s body part in relation to the plurality of control elements that does not activate the control elements, wherein the control intention is ascertained based on the movement of the user'"'"'s body part in relation to a respective control element;

ascertaining a control spotlight region corresponding to a portion of the display device encompassing an expected touch position of the user'"'"'s body part on the display device, wherein the location and size of the control spotlight region is ascertained based on the ascertained control probability for each control element, the size of the control spotlight region being greater than the user'"'"'s body part and smaller than the display device; and

selectively adapting display of the control elements such that (i) each control element which is at least partially within the control spotlight region is proportionally optimized for activation according to the respective ascertained control probability, and (ii) control elements outside the control spotlight region are not proportionally optimized for activation;

wherein the control probability for each control element is ascertained prior to activating a control action;

wherein each control element which is at least partially within the control spotlight region is scaled as a function of at least one of (i) distance from the center of the control spotlight region and (ii) amount of planar overlap with the control spotlight region.

2. The method according to claim 1, wherein the ascertaining of the control spotlight region includes ascertaining whether the user'"'"'s body part is located within an activation region that is spatially defined relative to a display region of at least one of the control elements on the display device.

3. The method according to claim 1, wherein the ascertaining of the control intention and the adapting of the represented information are performed at least one of (a) iteratively and (b) continuously.

4. The method according to claim 2, wherein the ascertaining of the control spotlight region is based on information relating to at least one of (a) a bodily action, (b) a direction of movement of a body part, and (c) a viewing direction of a user.

5. The method according to claim 1, further comprising ascertaining user information at least partly by at least one of (a) a camera system and (b) an ultrasonic sensor system.

6. The method according to claim 1, wherein the ascertaining of the control intention is based on information from surroundings about a driving situation, which is taken into account when adapting the at least one of the control elements.

7. The method according to claim 1, wherein the adapting includes changing at least one of (a) a transparency, (b) the size of the control element, (c) a distance from adjacent additional control elements and (d) an animation of at least one of (i) the control element and (ii) several control elements.

8. The method according to claim 1, wherein the ascertaining of the control intention includes detecting and evaluating gestures executed by the body part, the adapting of the represented information being performed in accordance with an adaptation function assigned to the detected gesture.

9. The method according to claim 8, wherein the gestures include at least one static gesture, which is detected on the basis of a predefined body part attitude.

10. The method according to claim 8, wherein the gestures include at least one dynamic gesture, which is detected on the basis of a predefined path line traversed by the body part.

11. The method according to claim 8, wherein the gestures include at least one complex gesture, which is detected on the basis of at least one of (a) a transition between predefined static gestures and (b) a static gesture, which traverses a predefined path line.

12. The method according to claim 1, wherein the information represented on the display device is adapted for a representation that is optimized for a visual communication of information if no control intention is ascertained.

13. An interactive control device, comprising:

a display device adapted to represent information that includes control elements; and

a control device having at least one processor adapted to ascertain a control probability based on an ascertained control intention for each of a plurality of control elements represented on the display device, from a movement of a user'"'"'s body part in relation to the plurality of control elements that does not activate the control elements, the control intention ascertained based on the movement of the user'"'"'s body part, ascertain a control spotlight region corresponding to a portion of the display device encompassing an expected touch position of the user'"'"'s body part on the display device, wherein the location and size of the control spotlight region is ascertained based on the ascertained control probability for each control element, the size of the control spotlight region being greater than the user'"'"'s body part and smaller than the display device, and to selectively adapt display of the control elements such that (i) each control element which is at least partially within the control spotlight region is proportionally optimized for activation according to the respective ascertained control probability, and (ii) control elements outside the control spotlight region are not proportionally optimized for activation;

wherein the control probability for each control element is ascertained prior to activating a control action;

wherein each control element which is at least partially within the control spotlight region is scaled as a function of at least one of (i) distance from the center of the control spotlight region and (ii) amount of planar overlap with the control spotlight region.

14. The device according to claim 13, further comprising at least one sensor device adapted to detect the user'"'"'s body part within an activation region that is spatially defined relative to a display region of at least one of the control elements on the display device in order to ascertain a control intention for at least one of the control elements represented on the display device.

15. The device according to claim 13, wherein the control device is adapted to ascertain the control intention and to adapt the information at least one of (a) iteratively and (b) continuously.

16. The device according to claim 14, wherein sensor information includes information about a bodily action of the user.

17. The device according to claim 13, further comprising at least one of (a) a camera system and (b) an ultrasonic sensor system adapted to ascertain user information.

18. The device according to claim 14, wherein sensor information includes information relating to at least one of (a) a viewing direction of the user and (b) a direction of movement of the user'"'"'s body part.

19. The device according to claim 14, wherein the control device is adapted to ascertain at least one of (a) a direction of movement of the body part and (b) a viewing direction of the user in accordance with sensor information.

20. The device according to claim 13, wherein the control device is configured to adapt the at least one of the control elements in accordance with information from surroundings about a driving situation.

21. The device according to claim 13, wherein the control device is configured to adapt a transparency by modification at least one of (a) a distance from adjacent additional control elements and (b) an animation of at least one control element.

22. The device according to claim 14, wherein the at least one sensor device is adapted to ascertain sensor information on the basis of at least one of (a) high-frequency signals transmitted via the body of the user and (b) a position of the body part.

23. The device according to claim 14, wherein the at least one sensor device includes a position-resolving touch-sensitive device.

24. The device according to claim 14, wherein at least one sensor device is adapted to detect and evaluate gestures performed by the body part, the control device configured to adapt the represented information in accordance with an adaptation function assigned to the detected gesture.

25. The device according to claim 24, wherein the gestures include at least one static gesture that is detectable by the at least one sensor device as a predefined body part attitude.

26. The device according to claim 24, wherein the gestures include at least one dynamic gesture, which is detectable by the at least one sensor device on the basis of a predefined path line traversed by the body part.

27. The device according to claim 24, wherein the gestures include at least one complex gesture, which is detectable by the at least one sensor device on the basis of at least one of (a) a transition between predefined static gestures and/(b) a static gesture that traverses a predefined path line.