Audio User Interface

• US 20030018477A1
• Filed: 01/29/2002
• Published: 01/23/2003
• Est. Priority Date: 01/29/2001
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Hinde, Stephen John
• Granted Patent: US 7,190,794 B2
• Time in Patent Office: Days
• US Class Current: 704/273
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G11B 19/025 'Virtual' control panels, e.g. Graphical User Interface [GUI]
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04S 1/005 For headphones
• CPC Class Codes: H04S 7/304 For headphones

Abstract

An audio “desktop” user-interface is provided in which services are represented by audio labels presented in an audio field through respective synthesized sound sources. A desired service is selected by identifying it through its sound source or audio label. A user can modify the layout of service-representing sound sources and preferably add and remove services.

Claims

1. An audio user-interfacing method in which services are represented by audio labels presented in an audio field through respective synthesized sound sources, the method comprising the steps of:

(a) storing, for each service to be represented, service access data and data associating the service with at least one said sound source and specifying at least one audio label;

(b) generating an audio field in which said sound sources are synthesized at respective rendering positions to sound their associated service-representing audio labels;

(c) selecting a service by identifying it through at least one of its sound source and audio label;

the method further involving enabling a user to modify the audio-field layout of the service-representing sound sources and/or what services are represented in the audio field.

2. A method according to claim 1, wherein step (c) involves rotating and/or displacing the audio field to bring the sound source of a target service to lie in a predetermined selection direction.

3. A method according to claim 1, wherein step (c) involves moving an audio-cursor sound source in the audio field to align it with the sound source of a target service.

4. A method according to claim 1, wherein step (c) involves the user speaking the audio label of a target service, and using a speech recogniser to match, where possible, this spoken label to the stored audio labels.

5. A method according to claim 1, wherein the selection of a service in step (c) results in the service-representing sound sources being replaced by an audio interface to the service.

6. A method according to claim 1, wherein the access data of a said service is the path name on a local machine of a service executable file.

7. A method according to claim 1, wherein the access data of a said service is the address of a service resource on a remote machine to be accessed over a communications connection.

8. A method according to claim 1, wherein said services comprise one or more of the following service types:

application software;

a communication service;

an entertainment service;

a database-based information resource;

a file information resource;

a transactional service;

an augmented-reality service.

9. A method according to claim 1, wherein the rendering positions of the sound sources are specified in the audio field with at least two degrees of freedom.

10. A method according to claim 1, wherein at least one said audio label is a verbal service name or descriptor.

11. A method according to claim 1, wherein at least one said audio label is an audio feed from the service concerned.

12. A method according to claim 1, wherein at least one said audio label is a distinctive sound or sound sequence.

13. A method according to claim 1, wherein at least one said audio label is user specified.

14. A method according to claim 1, wherein at least one said audio label is provided by the corresponding service.

15. A method according to claim 1, wherein at least one service has on-going activity and provides notification of significant service-related events, these notifications being passed to the user in audible form through a said sound source associated with the service.

16. A method according to claim 1, wherein the rendering positions of said sound sources are specified relative to an audio-field reference, a user being enabled to modify the layout of the service-representing sound sources through the modification of the rendering positions of individual sound sources.

17. A method according to claim 1, including varying the rendering positions of said sound source by varying an offset between an audio-field reference relative to which the sounds sources are positioned in the audio field, and a presentation reference determined by a mounting configuration of audio output devices through which the sound sources are synthesised.

18. A method according to claim 17, wherein said offset is varied to stabilise the audio field relative to one of:

a user'"'"'s head;

a user'"'"'s body;

a vehicle in which the user is travelling;

the world;

this stabilisation taking account of whether the audio output devices are world, vehicle, body or head mounted, and, as appropriate, rotation of the user'"'"'s head or body, or turning of the vehicle.

19. A method according to claim 17, wherein said offset is varied in response to user input via an input device.

20. A method according to claim 1, further involving enabling a user to modify which services are represented in the audio field.

21. Apparatus for providing an audio user interface in which services are represented by audio labels presented in an audio field through respective synthesized sound sources, the apparatus comprising:

a memory for storing, for each service to be represented, service access data and data associating the service with at least one said sound source and specifying at least one audio label;

rendering means for generating, through audio output devices, an audio field in which said sound sources are synthesized at respective rendering positions to provide sounds their associated service-representing audio labels;

selection means for selecting a represented service by identifying it through at least one of its sound source and audio label; and

user input means for enabling a user to modify the audio-field layout of the service-representing sound sources and/or what services are represented in the audio field..

22. Apparatus according to claim 21, wherein the selection means comprises means for rotating and/or displacing the audio field to bring the sound source of a target service to lie in a predetermined selection direction, and means for inputting a select command to select a service whose sound source is aligned with said selection direction.

23. Apparatus according to claim 21, wherein the selection means comprises means for moving an audio-cursor sound source in the audio field to align it with the sound source of a target service, and means for inputting a select command to select a service with which the sound source is aligned.

24. Apparatus according to claim 21, wherein the selection means comprises speech input means for recognising an audio label spoken by the user and selecting the corresponding service.

25. Apparatus according to claim 21, further comprising means for responsive to the selection of a service by the selection means to replace the audio field of service-representing sound sources with an audio interface to the selected service.

26. Apparatus according to claim 21, wherein the access data of a said service is the path name on a local machine of a service executable file.

27. Apparatus according to claim 21, wherein the access data of a said service is the address of a service resource on a remote machine to be accessed over a communications connection.

28. Apparatus according to claim 21, wherein said services comprise one or more of the following service types:

application software;

a communication service;

an entertainment service;

a database-based information resource;

a file information resource;

a transactional service;

an augmented-reality service.

29. Apparatus according to claim 21, wherein the generation subsystem is such that the rendering positions of the sound sources are specified in the audio field with at least two degrees of freedom.

30. Apparatus according to claim 21, wherein at least one said audio label is a verbal service name or descriptor.

31. Apparatus according to claim 21, wherein at least one said audio label is an audio feed from the service concerned.

32. Apparatus according to claim 21, wherein at least one said audio label is a distinctive sound or sound sequence.

33. A method according to claim 21, further comprising means for enabling a user to specify said audio labels.

34. A method according to claim 21, further comprising means for receiving audio-label specifying data from the corresponding service and storing it in said memory.

35. Apparatus according to claim 21, wherein at least one service has on-going activity and the apparatus further comprises means for receiving, from that service, notification of significant service-related events, and for causing these notifications to be output to the user in audible form through a said sound source associated with the service.

36. Apparatus according to claim 21, wherein the rendering positions of said sound sources are specified relative to an audio-field reference, the said user input means is operative to modify the layout of the service-representing sound sources through the modification of the rendering positions of individual sound sources.

37. Apparatus according to claim 21, wherein the generation sub-system comprises rendering-position determining means for determining the rendering position of each sound source, and rendering means, including said audio output devices, for rendering the sound sources at their respective rendering locations in the audio field;

the rendering-position determining means comprising;

means for setting the location of each said sound source relative to an audio-field reference;

offset means for controlling an offset between the audio-field reference and a presentation reference determined by a mounting configuration of the audio output devices.

38. Apparatus according to claim 37, wherein the offset means comprises stabilisation means for varying the said offset such as to stabilise the audio field reference relative to one of:

a user'"'"'s head;

a user'"'"'s body;

a vehicle mounting the apparatus;

the world;

this stabilisation taking account of whether the audio output devices are world, vehicle, body or head mounted, and, as appropriate, rotation of the user'"'"'s head or body, or turning of the vehicle.

39. Apparatus according to claim 37, wherein said offset means comprises means for enabling a user to vary said offset.

40. Apparatus according to claim 21, further comprising means for enabling a user to modify which services are represented in the audio field.

41. Apparatus for providing an audio user interface in which services are represented by audio labels presented in an audio field through respective synthesized sound sources, the apparatus comprising:

a memory for storing, for each service to be represented, service access data and data associating the service with at least one said sound source and specifying at least one audio label;

a rendering subsystem arranged to generate, through audio output devices, an audio field in which said sound sources are synthesized at respective rendering positions to provide sounds their associated service-representing audio labels;

a selection arrangement operative to select a represented service by identifying it through at least one of its sound source and audio label; and

user input functionality for enabling a user to modify the audio-field layout of the service-representing sound sources and/or what services are represented in the audio field..

42. Apparatus according to claim 41, wherein the selection arrangement comprises an arrangement for rotating and/or displacing the audio field to bring the sound source of a target service to lie in a predetermined selection direction, and an arrangement for inputting a select command to select a service whose sound source is aligned with said selection direction.

43. Apparatus according to claim 41, wherein the selection arrangement comprises an arrangement for moving an audio-cursor sound source in the audio field to align it with the sound source of a target service, and an arrangement for inputting a select command to select a service with which the sound source is aligned.

44. Apparatus according to claim 41, wherein the selection means comprises a speech input subsystem operative to recognise an audio label spoken by the user and select the corresponding service.

45. Apparatus according to claim 41, further comprising functionality responsive to the selection of a service by the selection arrangement to replace the audio field of service-representing sound sources with an audio interface to the selected service.