Audio user interface with selective audio field expansion

• US 7,266,207 B2
• Filed: 01/29/2002
• Issued: 09/04/2007
• Est. Priority Date: 01/29/2001
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Wilcock, Lawrence, Coles, Alistair Neil
• Primary Examiner(s): Mei; Xu
• Application Number:
  US10/059,102
• Publication Number:
  US 20020150254A1
• Time in Patent Office: 2,044 Days
• Field of Search: 381/310, 381/17, 381/18, 381/19, 381/20, 381/304, 381/305, 381/61, 381/945, 381/106
• US Class Current: 381/310
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G11B 19/025 'Virtual' control panels, e.g. Graphical User Interface [GUI]
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04S 1/005 For headphones
• CPC Class Codes: H04S 7/304 For headphones

Abstract

An audio user interface is provided in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate. The interface is generated by apparatus that includes functionality for determining for each sound source, a rendering position at which the sound source is to be synthesized to sound in the audio field. To facilitate discrimination between the synthesized sound sources, the sound-source rendering positions are adjusted such that the audio field is dilated in a region of the audio field about a focus reference; this focus reference is, for example, the current direction of facing of the user or the position of an audio cursor in the audio field.

Claims

1. An audio user-interfacing method in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the method comprising the steps of:

(a) determining, for each said sound source, an associated rendering position at which the sound source is normally to be synthesized to sound in the audio field;

(b) modifying said rendering positions of the sound sources to angularly dilating dilate a region of the audio field set relative to a focus reference by modifying and to adjust said rendering positions of some of the sound sources outside said region to accommodate that region; and

(c) generating, using audio output devices, an audio field in which said sound sources, including those outside said region as well as inside, are synthesized at their associated modified rendering positions after any modification effected by step (b).

2. A method according to claim 1, wherein the focus reference is moved relative to the audio field thereby changing which sound sources are present in the region of the audio field subject to angular dilation.

3. A method according to claim 2, wherein an audio cursor is movable under user control in the audio field, the audio cursor serving as said focus reference.

4. A method according to claim 2, wherein the focus reference is the current direction of facing of the user as applied to a reference position where the user is at least notionally located.

5. A method according to claim 1, wherein said region is angularly dilated at least in one of azimuth and elevation.

6. A method according to claim 1, wherein at least a portion of the audio field outside of said region is subject to angular compression in step (b) to compensate for the angular dilation of said region.

7. A method according to claim 1, wherein the sound sources in the audio field outside of said region have their rendering positions adjusted to accommodate that region without thereby being moved closer to one another.

8. A method according to claim 1, wherein step (b) can be selectively by-passed under user control.

9. A method according to claim 1, wherein the sound sources are organised into groups and step (b) is applied to sounds sources in less than all of said groups.

10. A method according to claim 1, wherein the angular dilation applied to said region is uniform across the region.

11. A method according to claim 1, wherein the angular dilation applied to said region is non-uniform across the region, being greatest at its centre.

12. A method according to claim 1, including the step of adjusting the extent of said region under user control.

13. A method according to claim 1, wherein the audio field is stabilised relative to one of:

a user'"'"'s head;

a user'"'"'s body;

a vehicle in which the user is travelling; and

the world, this stabilisation taking account of whether the audio output devices are world, vehicle, body or head mounted and, as appropriate, rotation of the user'"'"'s head or body, or turning of the vehicle.

14. A method according to claim 1, wherein at least some of said items represented by the sound sources are audio labels for services, the method further involving selecting a service by selecting the corresponding audio-label sound source.

15. An audio user-interfacing method in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the method comprising the steps of:

(a) determining for each sound source a rendering position at which the sound source is to be synthesized to sound in the audio field, these positions being such that the audio field is angularly dilated in a region of the audio field set relative to a focus reference;

(b) generating an audio field in which said sound sources, including those outside said region as well as inside, are synthesized at their associated rendering positions;

(c) moving the focus reference relative to the audio field thereby changing which sound sources are present in the region of the audio field subject to angular dilation.

16. A method according to claim 15, wherein an audio cursor is movable under user control in the audio field, the audio cursor serving as said focus reference.

17. A method according to claim 15, wherein the focus reference is the current direction of facing of the user as applied to a reference position where the user is at least notionally located.

18. A method according to claim 15, wherein said region is angularly dilated at least in azimuth relative to the user.

19. A method according to claim 15, wherein at least a portion of the audio field outside of said region is subject to angular compression in step (b) to compensate for the angular dilation of said region.

20. A method according to claim 15, wherein the sound sources in the audio field outside of said region have their rendering positions adjusted to accommodate that region without thereby being moved closer to one another.

21. Apparatus for providing an audio user interface in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the apparatus comprising:

rendering-position determining means for determining, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound in the audio field;

position-modifying means for modifying said rendering positions of at least some of the sound sources to angularly dilate a region of the audio field set relative to a focus reference and to adjust the rendering positions of sound sources outside said region to accommodate that region; and

rendering means, including audio output devices, for generating an audio field in which said sound sources, including those outside said region as well as inside, are synthesized at their associated rendering positions, as modified, after any modification by the position-modifying means, to provide sounds related to the items concerned.

22. Apparatus according to claim 21, wherein the rendering-position determining means comprises:

means for setting the location of each said sound source relative to an audio-field reference;

means for controlling an offset between the audio field reference and a presentation reference, the presentation reference being determined by a mounting configuration of the audio output devices; and

means for deriving the rendering position of each sound source based on the location of the sound source in the audio field and said offset.

23. Apparatus according to claim 22, wherein the position-modifying means comprises:

means for determining the extent of said region, prior to angular dilation, relative to said presentation reference;

means for comparing the rendering position of each sound source with the determined extent of said region whereby to determine which sound sources lie within the region;

means for modifying rendering positions of the sound sources lying within said region to angularly dilate the audio field in that region; and

means for modifying the rendering positions of at least some of the sound sources outside of said region to accommodate the region in the audio field.

24. Apparatus according to claim 21, further comprising means for generating an audio cursor movable under user control in the audio field, and means for providing the position-modifying means with the position of the audio cursor, the position-modifying means being arranged to use the position of the audio cursor as said focus reference.

25. Apparatus according to claim 21, further comprising means for determining the direction of facing of the user, and for passing this direction to the position-modifying means, the position-modifying means being arranged to use the direction of facing of the user as said focus reference.

26. Apparatus according to claim 21, wherein the position-modifying means is arranged to modify the rendering positions of at least some of the sound sources outside of said region such that at least a portion of the audio field outside of said region is subject to angular compression.

27. Apparatus according to claim 21, wherein the position-modifying means is arranged to modify the rendering positions of the sound sources outside of said region such as to accommodate that region without thereby moving the sources outside of the region closer to one another.

28. Apparatus according to claim 21, wherein the angular dilation applied to said region by the position-modifying means is uniform across the region.

29. Apparatus according to claim 21, wherein the angular dilation applied to said region by the position-modifying means is non-uniform across the region, being greatest at its centre.

30. Apparatus according to claim 21, including user-operable means for adjusting the extent of said region.

31. Apparatus according to claim 21, wherein at least some of said items represented by the sound sources are audio labels for services, the apparatus including a selection arrangement for selecting a service by selecting the corresponding audio-label sound source.

32. Apparatus according to claim 22, wherein said means for setting an offset between the audio field reference and a presentation reference, comprises user input means for enabling a user to change said offset, and stabilisation means for varying said offset such as to stabilise the audio field reference relative to one of:

a user'"'"'s head;

a user'"'"'s body;

a vehicle mounting the apparatus; and

the world.

33. Apparatus for providing an audio user interface in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the apparatus comprising:

a rendering-position determining arrangement arranged to determine, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound in the audio field;

a position-modifying arrangement arranged to modify said rendering positions of some of the sound sources to angularly dilate a region of the audio field set relative to a focus reference and to adjust the rendering positions of sound sources outside said region to accommodate that region; and

a rendering subsystem, including audio output devices, arranged to generate an audio field in which said sound sources, including those outside said region as well as inside, are synthesized at their associated rendering positions, after any modification as modified, by the position-modifying arrangement, to provide sounds related to the items concerned.

34. Apparatus according to claim 33, wherein the rendering-position determining arrangement comprises:

a setting arrangement for setting the location of each said sound source relative to an audio-field reference;

a control arrangement for controlling an offset between the audio field reference and a presentation reference, the presentation reference being determined by a mounting configuration of the audio output devices; and

a deriving arrangement arranged to derive the rendering position of each sound source based on the location of the sound source in the audio field and said offset.

35. Apparatus according to claim 34, wherein the position-modifying arrangement comprises:

an arrangement arranged to determine the extent of said region, prior to angular dilation, relative to said presentation reference;

a comparator for comparing the rendering position of each sound source with the determined extent of said region whereby to determine which sound sources lie within the region;

an arrangement arranged to modify rendering positions of the sound sources lying within said region to angularly dilate the audio field in that region; and

an arrangement arranged to modify the rendering positions of at least some of the sound sources outside of said region to accommodate the region in the audio field.

36. Apparatus according to claim 33, further comprising a cursor subsystem operative to generate an audio cursor movable under user control in the audio field, and an arrangement operative to provide the position-modifying arrangement with the position of the audio cursor, the position-modifying arrangement being arranged to use the position of the audio cursor as said focus reference.

37. Apparatus according to claim 33, further comprising an arrangement arranged to determine the direction of facing of the user, and to pass this direction to the position-modifying arrangement, the position-modifying arrangement being arranged to use the direction of facing of the user as said focus reference.

38. Apparatus according to claim 33, wherein the position-modifying arrangement is arranged to modify the rendering positions of at least some of the sound sources outside of said region such that at least a portion of the audio field outside of said region is subject to angular compression.

39. Apparatus according to claim 33, wherein the position-modifying arrangement is arranged to modify the rendering positions of the sound sources outside of said region such as to accommodate that region without thereby moving the sources outside of the region closer to one another.

40. Apparatus according to claim 33, wherein the angular dilation applied to said region by the position-modifying arrangement is uniform across the region.

41. Apparatus according to claim 33, wherein the angular dilation applied to said region by the position-modifying an arrangement is non-uniform across the region, being greatest at its centre.

42. Apparatus according to claim 33, including user-operable functionality for adjusting the extent of said region.

43. Apparatus according to claim 33, wherein at least some of said items represented by the sound sources are audio labels for services, the apparatus including a selection arrangement for selecting a service by selecting the corresponding audio-label sound source.

44. Apparatus according to claim 34, wherein the control arrangement comprises user input functionality for enabling a user to change said offset, and a stabilisation arrangement arranged to vary said offset such as to stabilise the audio field reference relative to one of:

a user'"'"'s head;

a user'"'"'s body;

a vehicle mounting the apparatus; and

the world.

45. Apparatus for providing an audio user interface in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the apparatus comprising:

a rendering-position determination arrangement arranged to determine for each sound source a rendering position at which the sound source is to be synthesized to sound in the audio field, these positions being such that the audio field is angularly dilated in a region of the audio field set relative to a focus reference;

a rendering subsystem, including audio output devices, arranged to generate an audio field in which said sound sources, including those outside said region as well as inside, are synthesized at their associated rendering positions; and

input functionality for moving the focus reference relative to the audio field thereby changing which sound sources are present in the region of the audio field subject to angular dilation.

46. An audio user-interfacing method in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the method comprising the steps of:

(a) determining, for each said sound source, an associated rendering position at which the sound source is normally to be synthesized to sound in the audio field;

(b) angularly dilating a region of the audio field set relative to a focus reference by modifying said rendering positions of at least some of the sound sources;

(c) generating, using audio output devices, an audio field in which said sound sources are synthesized at their associated rendering positions after any modification effected by step(b); and

(d) compensating for the angular dilation of said region by subjecting, during step (b), at least a portion of the audio field outside of said region to angular compression.

47. An audio user-interfacing method in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the method comprising the steps of:

(a) determining for each sound source a rendering position at which the sound source is to be synthesized to sound in the audio field, these positions being such that the audio field is angular dilated in a region of the audio field set relative to a focus reference;

(b) compensating for the angular dilation of said region by generating an audio field outside of said region in which said sound sources are synthesized at their associated rendering positions; and

(c) moving the focus reference relative to the audio field thereby changing which sound sources are present in the region of the audio field subject to angular dilation.

48. Apparatus for providing an audio user interface in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the apparatus comprising:

rendering-position determining means for determining, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound in the audio field;

position-modifying means for modifying said rendering positions of at least some of the sound sources to angularly dilate a region of the audio field set relative to a focus reference, the position-modifying means being arranged to modify the rendering positions of at least some of the sound sources outside of said region such that at least a portion of the audio field outside of said region is subject to angular compression; and

rendering means, including audio output devices, for generating an audio field in which said sound sources are synthesized at their associated rendering positions, after any modification by the position-modifying means, to provide sounds related to the items concerned.

49. Apparatus for providing an audio user interface in which items are represented in an audio field by corresponding synthesized sound sources from where sounds related to the items appear to emanate, the apparatus comprising:

a rendering-position determining arrangement arranged to determine, for each said sound source, an associated rendering position at which the sound source is to be synthesized to sound in the audio field;

a position-modifying arrangement arranged to modify said rendering positions of at least some of the sound sources to angularly dilate a region of the audio field set relative to a focus reference, the position-modifying arrangement being arranged to modify the rendering positions of at least some of the sound sources outside of said region such that at least a portion of the audio field outside of said region is subject to angular compression; and

a rendering subsystem, including audio output devices, arranged to generate an audio field in which said sound sources are synthesized at their associated rendering positions, after any modification by the position-modifying arrangement, to provide sounds related to the items concerned.