USER INTERFACE CONTROLLED BY ENVIRONMENTAL CUES

• US 20090172527A1
• Filed: 12/27/2007
• Published: 07/02/2009
• Est. Priority Date: 12/27/2007
• Status: Active Grant

• Current Assignee: Conversant Wireless Licensing S.à.r.l. (f/k/a Core Wireless Licensing S.a.r.l.) (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Original Assignee: Nokia Corporation
• Inventors: Hoffmann, Marc, Coersmeier, Edmund, Lobko, Maxim, Buecker, Martin
• Granted Patent: US 8,370,755 B2
• Time in Patent Office: Days
• US Class Current: 715/700
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G08G 1/096716 where the received information does not generate an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096725 where the received information generates an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096758 where no selection takes place on the transmitted or the received information
• CPC Class Codes: G08G 1/096783 where the origin of the information is a roadside individual element

Abstract

A method comprises receiving a signal containing an environmental cue; correlating the environmental cue to a user interface control element; and generating a user interface control command based on the environmental cue. A device may be controlled by receiving a visual image, processing the image to identify a visual cue, correlating the visual cue with a user interface control, and controlling the device using the user interface control. The visual image may be received from a camera that is part of an automobile, and the user interface control may include a control for a radio to tune to a radio station; a speed control for controlling a speed of the automobile corresponding to a speed sign visual cue; or a navigation system control for controlling a navigation system based on traffic signs. Other variations include an apparatus and computer-readable media including instructions for performing the methods.

Claims

1. A method comprising:

receiving a signal containing an environmental cue;

correlating the environmental cue to a user interface control element; and

generating a user interface control command based on the environmental cue.

2. The method of claim 1, wherein receiving a signal containing an environmental cue comprises receiving a visual image from a camera.

3. The method of claim 2, wherein correlating the environmental cue to a user interface control element comprises correlating a traffic sign contained in the visual image to a vehicle speed control.

4. The method of claim 2, wherein correlating the environmental cue to a user interface control element comprises correlating a traffic sign contained in the visual image to a navigation system control.

5. The method of claim 2, wherein correlating the environmental cue to a user interface control element comprises correlating an advertising sign contained in the visual image to a radio station control.

6. The method of claim 1, wherein correlating the environmental cue to a user interface element comprises receiving user input selecting a portion of a visual image corresponding to the received signal.

7. The method of claim 6, further comprising processing the selected portion of the visual image to identify the environmental cue.

8. The method of claim 7, further comprising identifying information in the visual image indicative of a speed limit, and wherein generating the user interface control command comprises generating a speed control command for a vehicle.

9. The method of claim 1, further comprising:

displaying a visual image comprising one or more environmental cues;

receiving user input selecting a portion of the displayed visual image;

displaying a border having a first dimension around the selected portion of the displayed image;

in response to additional user input, displaying a border smaller than the first dimension around the selected portion of the displayed image; and

upon further user input or after expiration of a predetermined time period, using a portion of the image within the smaller border as the basis for comparison with a database of potentially matching environmental cues.

10. The method of claim 7, further comprising identifying a navigation element within the visual image, and wherein generating the user interface control command comprises generating a command for a navigation device.

11. The method of claim 7, further comprising identifying an advertising element within the visual image, and wherein generating the user interface control command comprises a command for a radio device.

12. The method of claim 1, wherein correlating the environmental cue to a user interface control element comprises matching a visual cue contained in a received image to one of a plurality of potentially matching visual cues arranged by region in a database.

13. The method of claim 12, wherein correlating the environmental cue comprises matching the visual cue to only those visual cues corresponding to a region in which the method is performed.

14. The method of claim 7, further comprising identifying an advertising element within the visual image, and wherein generating the user interface control command comprises causing a telephone number to be dialed corresponding to the advertising element.

15. The method of claim 1, wherein generating a user interface control command comprises generating a website address.

16. The method of claim 15, further comprising navigating to the website address.

17. An apparatus comprising a processor and memory storing executable instructions which, when executed by the processor, perform:

receiving a signal containing an environmental cue;

correlating the environmental cue to a user interface control element; and

generating a user interface control command based on the environmental cue.

18. The apparatus of claim 17, wherein receiving a signal containing an environmental cue comprises receiving a visual image from a camera.

19. The apparatus of claim 18, wherein correlating the environmental cue to a user interface control element comprises correlating a traffic sign contained in the visual image to a vehicle speed control.

20. The apparatus of claim 18, wherein correlating the environmental cue to a user interface control element comprises correlating a traffic sign contained in the visual image to a navigation system control.

21. The apparatus of claim 18, wherein correlating the environmental cue to a user interface control element comprises correlating an advertising sign contained in the visual image to a radio station control.

22. The apparatus of claim 17, wherein correlating the environmental cue to a user interface control element comprises receiving user input selecting a portion of a visual image corresponding to the received signal.

23. The apparatus of claim 22, further comprising processing the selected portion of the visual image to identify the environmental cue.

24. The apparatus of claim 23, further comprising identifying from the visual image a speed limit, and wherein generating the user interface control command comprises generating a speed control command for a vehicle.

25. The apparatus of claim 17, wherein the instructions when executed further perform:

displaying a visual image comprising one or more environmental cues;

receiving user input selecting a portion of the displayed visual image;

displaying a border having a first dimension around the selected portion of the displayed image;

in response to additional user input, displaying a border smaller than the first dimension around the selected portion of the displayed image; and

upon further user input or after expiration of a predetermined time period, using a portion of the image within the smaller border as the basis for comparison with a database of potentially matching environmental cues.

26. The apparatus of claim 23, wherein the instructions further perform:

identifying a navigation element within the visual image, and wherein generating the user interface control command comprises generating a command for a navigation device.

27. The apparatus of claim 23, wherein the instructions when executed further perform

identifying an advertising element within the visual image, and wherein generating the user interface control command comprises a command for a radio device.

28. The apparatus of claim 17, wherein correlating the environmental cue to a user interface control element comprises matching a visual cue contained in a received image to one of a plurality of potentially matching visual cues arranged by region in a database.

29. The apparatus of claim 28, wherein correlating the environmental cue comprises matching the visual cue to only those visual cues corresponding to a region in which the method is performed.

30. The apparatus of claim 23, wherein the instructions when executed further perform:

identifying an advertising element within the visual image, and wherein generating the user interface control command comprises causing a telephone number to be dialed corresponding to the advertising element.

31. The apparatus of claim 17, wherein generating the user interface control command comprises generating a website address.

32. The apparatus of claim 31, wherein the instructions when executed further perform:

navigating to the website address.

33. A computer-readable medium having encoded thereon executable instruction which, when executed, perform:

receiving a signal containing an environmental cue;

correlating the environmental cue to a user interface control element; and

generating a user interface control command based on the environmental cue.

34. The computer-readable medium of claim 33, wherein the instructions further perform:

receiving user input selecting a portion of a visual image corresponding to the received signal.

35. An apparatus comprising a processor and memory storing executable instructions which, when executed by the processor, perform the following:

receive an image from a camera configured to capture traffic signs along a travel path of a vehicle;

display on a touch-screen display the received image;

receive user input identifying a region of the displayed image;

process the identified region of the displayed image by comparing it to a plurality of patterns each of which corresponds to a potential traffic sign; and

in response to determining that the identified region matches one of the plurality of patterns, generating a vehicle control signal configured to control a speed of the vehicle.

36. The apparatus of claim 35, wherein the executable instructions when executed by the processor further perform:

displaying a border having a first dimension around the identified region of the displayed image;

in response to additional user input, displaying a border smaller than the first dimension around the identified region of the displayed image; and

upon further user input or after expiration of a predetermined time period, using a portion of the image within the smaller border as the basis for the comparison with the plurality of patterns.