User interface controlled by environmental cues

• US 8,370,755 B2
• Filed: 12/27/2007
• Issued: 02/05/2013
• Est. Priority Date: 12/27/2007
• Status: Active Grant

• Current Assignee: Conversant Wireless Licensing S.à.r.l. (f/k/a Core Wireless Licensing S.a.r.l.) (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Original Assignee: Conversant Wireless Licensing S.à.r.l. (f/k/a Core Wireless Licensing S.a.r.l.) (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Inventors: Buecker, Martin, Lobko, Maxim, Hoffmann, Marc, Coersmeier, Edmund
• Primary Examiner(s): PITARO, RYAN F
• Application Number:
  US11/965,136
• Publication Number:
  US 20090172527A1
• Time in Patent Office: 1,867 Days
• Field of Search: 715751-757, 701/24, 701/68, 701/93
• US Class Current: 715/751
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G08G 1/096716 where the received information does not generate an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096725 where the received information generates an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096758 where no selection takes place on the transmitted or the received information
• CPC Class Codes: G08G 1/096783 where the origin of the information is a roadside individual element

Abstract

A method comprises receiving a signal containing an environmental cue; correlating the environmental cue to a user interface control element; and generating a user interface control command based on the environmental cue. A device may be controlled by receiving a visual image, processing the image to identify a visual cue, correlating the visual cue with a user interface control, and controlling the device using the user interface control. The visual image may be received from a camera that is part of an automobile, and the user interface control may include a control for a radio to tune to a radio station; a speed control for controlling a speed of the automobile corresponding to a speed sign visual cue; or a navigation system control for controlling a navigation system based on traffic signs. Other variations include an apparatus and computer-readable media including instructions for performing the methods.

Claims

1. A method comprising:

receiving a signal including an environmental cue;

correlating the environmental cue to an interface control element from a plurality of possible interface control elements, wherein the plurality of possible interface control elements includes a control for automatically operating at least one of a radio device, a telephone device, and a navigation device; and

generating an interface control command based on the environmental cue,

wherein when the received signal comprises a visual image and the environmental cue comprises an advertising element in a user-selected portion of the visual image, generating an interface control command comprises generating a command for the radio device.

2. The method of claim 1, wherein receiving a signal including an environmental cue comprises receiving a visual image from a camera.

3. The method of claim 2, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating a traffic sign within the visual image to a vehicle speed control.

4. The method of claim 2, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating a traffic sign within the visual image to a navigation system control.

5. The method of claim 2, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating an advertising sign within the visual image to a radio station control.

6. The method of claim 1, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises receiving user input regarding the user-selected portion of the visual image corresponding to the received signal.

7. The method of claim 6, further comprising processing the user-selected portion of the visual image to identify the environmental cue.

8. The method of claim 7, further comprising identifying information in the visual image indicative of a speed limit, and wherein generating the interface control command comprises generating a speed control command for a vehicle.

9. The method of claim 1, further comprising:

displaying a visual image comprising one or more environmental cues;

receiving user input selecting a portion of the displayed visual image;

displaying a border having a first dimension around the selected portion of the displayed image;

in response to additional user input, displaying a border smaller than the first dimension around the selected portion of the displayed image; and

upon further user input or after expiration of a predetermined time period, using a portion of the image within the smaller border as the basis for comparison with a database of potentially matching environmental cues.

10. The method of claim 7, further comprising identifying a navigation element within the visual image, and wherein generating the interface control command comprises automatically generating a detour command for the navigation device when the environmental cue comprises a traffic alert.

11. The method of claim 7, further comprising identifying the advertising element within the visual image.

12. The method of claim 1, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises matching a visual cue within a received image to one of a plurality of potentially matching visual cues arranged by region in a database.

13. The method of claim 12, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises matching the visual cue to only those visual cues corresponding to a region in which the method is performed.

14. The method of claim 7, further comprising identifying the advertising element within the visual image, and wherein generating the interface control command comprises causing a telephone number to be dialed corresponding to the advertising element.

15. The method of claim 1, wherein generating the interface control command comprises generating a website address.

16. The method of claim 15, further comprising navigating to the website address.

17. An apparatus comprising a processor and memory storing executable instructions configured to, when executed by the processor, cause the apparatus to:

receive a signal including an environmental cue;

correlate the environmental cue to an interface control element from a plurality of possible interface control elements, wherein the plurality of possible interface control elements includes a control for automatically operating at least one of a radio device, a telephone device, and a navigation device; and

generate an interface control command based on the environmental cue,

wherein when the signal comprises a visual image and the environmental cue comprises an advertising element in a user-selected portion of the visual image, generating an interface control command comprises generating a command for the radio device.

18. The apparatus of claim 17, wherein receiving a signal containing an environmental cue comprises receiving a visual image from a camera.

19. The apparatus of claim 18, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating a traffic sign within the visual image to a vehicle speed control.

20. The apparatus of claim 18, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating a traffic sign within the visual image to a navigation system control.

21. The apparatus of claim 18, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises correlating an advertising sign within the visual image to a radio station control.

22. The apparatus of claim 17, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises receiving user input regarding the user-selected portion of the visual image corresponding to the received signal.

23. The apparatus of claim 22, wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

process the user-selected portion of the visual image to identify the environmental cue.

24. The apparatus of claim 23, wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

identify from the visual image a speed limit, and wherein generating the interface control command comprises generating a speed control command for the vehicle.

25. The apparatus of claim 17, wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

display a visual image comprising one or more environmental cues;

receive user input selecting a portion of the displayed visual image;

display a border having a first dimension around the selected portion of the displayed image;

in response to additional user input, display a border smaller than the first dimension around the selected portion of the displayed image; and

upon further user input or after expiration of a predetermined time period, use a portion of the image within the smaller border as the basis for comparison with a database of potentially matching environmental cues.

26. The apparatus of claim 23, wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

identify a navigation element within the visual image, and wherein generating the interface control command comprises automatically generating a detour command for the navigation device when the environmental cue comprises a traffic alert.

27. The apparatus of claim 23, wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

identify the advertising element within the visual image.

28. The apparatus of claim 17, wherein correlating the environmental cue to an interface control element from a plurality of possible interface control elements comprises matching a visual cue within a received image to one of a plurality of potentially matching visual cues arranged by region in a database.

29. The apparatus of claim 28, wherein correlating the environmental cue from a plurality of possible interface control elements comprises matching the visual cue to only those visual cues corresponding to a region in which the method is performed.

30. The apparatus of claim 23, wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

identify the advertising element within the visual image, and wherein generating the interface control command comprises causing a telephone number to be dialed corresponding to the advertising element.

31. The apparatus of claim 17, wherein generating the interface control command comprises generating a website address.

32. The apparatus of claim 31, wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

navigate to the website address.

33. A non-transitory computer-readable medium storing executable instructions configured to, when executed, cause a processor to:

receive a signal including an environmental cue;

correlate the environmental cue to an interface control element from a plurality of possible interface control elements, wherein the plurality of possible interface control elements includes a control for automatically operating at least one of a radio device, a telephone device, and a navigation device; and

generate an interface control command based on the environmental cue,

wherein when the signal comprises a visual image and the environmental cue comprises an advertising element in a user-selected portion of the visual image, generating an interface control command comprises generating a command for the radio device.

34. The computer-readable medium of claim 33, further storing executable instructions configured to, when executed, cause the processor to:

receive user input selecting a portion of a visual image corresponding to the received signal.

35. An apparatus comprising a processor and memory storing executable instructions configured to, when executed by the processor, cause the apparatus to:

receive an image from a camera configured to capture traffic signs along a travel path of a vehicle;

display on a touch-screen display the received image;

receive user input identifying a region of the displayed image;

identify a traffic sign at least in part by processing the identified region of the displayed image and comparing it to a plurality of patterns each of which corresponds to a visual cue;

in response to determining that the identified region matches one of the plurality of patterns, generate a control signal configured to automatically control an element of the vehicle, the element being one of a plurality of possible elements of the vehicle, wherein the plurality of possible elements includes at least one of a radio device, a telephone device, and a navigation device,

wherein when the signal comprises a visual image and the environmental cue comprises an advertising element in a user-selected portion of the visual image, generating a control signal comprises generating a command for the radio device.

36. The apparatus of claim 35, wherein the memory stores executable instructions configured to, when executed by the processor, cause the apparatus to:

display a border having a first dimension around the identified region of the displayed image;

in response to additional user input, display a border smaller than the first dimension around the identified region of the displayed image; and

upon further user input or after expiration of a predetermined time period, use a portion of the image within the smaller border as the basis for the comparison with the plurality of patterns.