Telemetric contextually based spatial audio system integrated into a mobile terminal wireless system

• US 6,845,338 B1
• Filed: 02/25/2003
• Issued: 01/18/2005
• Est. Priority Date: 02/25/2003
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, Inc. (Zebra Technologies Corporation)
• Inventors: Katz, Joseph, Vollkommer, Richard M., Willins, Bruce A.
• Primary Examiner(s): Bui, Bryan
• Application Number:
  US10/372,159
• Time in Patent Office: 693 Days
• Field of Search: 702/150, 702/151, 702/152, 702/153, 340/988, 340/990, 340993-995, 709/227, 709/228
• US Class Current: 702/150
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01S 3/803 using amplitude comparison of signals derived from receiving transducers or transducer systems having differently-oriented directivity characteristics
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A telemetric contextually based spatial audio system is integrated into a mobile terminal including a headset and antenna. The headset includes an electronic compass which determines head orientation data for the user. The terminal'"'"'s geographical position can either be determined locally (e.g. onboard GPS) or by network triangulation. The orientation and position data are sent to a server which overlays the position and head orientation data onto a locally stored “floor plan”, describing an object(s) of interest. The floor plan relates the user'"'"'s viewpoint to a database of contextual audio clips. A regionally significant portion of the database and respective triggers are sent to the terminal. The terminal using the database information recalculates which audio clip(s) to play. The terminal then convolves the respective audio clip(s) via an impulse response function so as to be perceived by the user as emanating from the exact spatial position of the object specified by the system.

Claims

1. A telemetric contextually based audio system integrated into a mobile terminal in a wireless system, comprising:

a) A user transported mobile terminal coupled to a server via an air link;

b) a headset coupled to the terminal and including directional means indicating the user'"'"'s head orientation relative to an object viewed by the user;

c) transmitting means transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) mapping means mapping the user'"'"'s head orientation and geographical position to a physical layout of the object viewed;

e) selecting and transmitting means selecting and transmitting to the terminal clips of audio sound descriptive of the object viewed based upon the user'"'"'s mapped position and head orientation; and

f) convolving means convolving the audio sound to give the user a perception that the audio sound is actually emanating from a point in the object viewed by the user.

2. The system of claim 1

g) wherein the convolving means continuously convolves the audio sound as the user'"'"'s geographical position and head orientation move relative to the object viewed.

3. The system of claim 1

h) wherein the terminal provides the audio sound to the head set as 3D audio.

4. The system of claim 1 wherein the head orientation relative to the object viewed is described in terms of relative angle.

5. The system of claim 1 further comprising:

i) data means relating user'"'"'s head and position information to an impulse response function for the object viewed by the user.

6. The system of claim 1 further comprising:

j) database means coupled to the server and storing the physical layout of the object viewed by the user.

7. The system of claim 6 wherein the database means stores the clips of audio sound.

8. The system of claim 1 wherein an electronic compass serves as a directional means.

9. The system of claim 1 wherein the user'"'"'s geographical position is determined from radio triangulation.

10. The system of claim 1 wherein the terminal stores a series of impulse response functions used to create 3 dimensional sounds perceived by the user as being spatially located at a defined relative angle and distance position to an object viewed by the user.

11. The system of claim 1 wherein the terminal convolves an audio stream to support a dynamic positional and head orientation relationship between the user and an audio emanation point.

12. The system of claim 1 wherein the terminal geographical position is obtained from the Global Positioning System.

13. The system of claim 1 wherein the terminal plays a clip of audio sound which describes what the user is viewing or plays a spatial audio redirecting the user'"'"'s attention to an item adjacent to the object viewed.

14. The system of claim 1 wherein the server accesses a database to obtain clips of audio sound regionally related in the physical layout.

15. The system of claim 1 wherein the headset provides binaural stereo audio sound.

16. The system of claim 1 wherein the perceived relative distance between the object viewed and the user can be altered by controlling the sound level and reverberations.

17. A telemetric method to provide a user individualized audio information of an object viewed based upon the user'"'"'s head orientation and geographical position in a mobile terminal wireless system, comprising:

a) coupling a user transported mobile terminal to a server via an air link;

b) coupling a headset to the terminal, the headset including directional means indicating the user'"'"'s head orientation relative to the object viewed by the user;

c) transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) mapping the user'"'"'s head orientation and geographical position to a physical layout of the object viewed;

e) selecting and transmitting to the terminal clips of audio sound descriptive of the object viewed based upon the user'"'"'s mapped position and head orientation; and

f) convolving the audio sound to give the user a perception that the audio sound is actually emanating from a point in the object viewed by the user.

18. The method of claim 17 further comprising:

g) continuously convolving the audio sound as the user'"'"'s geographical position and head orientation move relative to the object viewed.

19. The method of claim 17 further comprising:

h) providing the audio sound to the head set as 3D audio.

20. The method of claim 17 further comprising describing the head orientation in terms of relative angle to the object viewed.

21. The method of claim 17 further comprising:

i) relating user'"'"'s head and position information for the object viewed by the user to an impulse response function.

22. The method of claim 17 further comprising:

j) coupling database means to the server and storing the physical layout of the object viewed by the user.

23. The method of claim 22 further comprising:

k) storing clips of audio sounds in the database.

24. The method of claim 17 further comprising

l) using an electronic compass as a directional means.

25. The method of claim 17 further comprising:

m) determining the user'"'"'s geographical position from radio triangulation.

26. The method of claim 17 further comprising:

n) convolving an audio stream to support a dynamic positional and head orientation relationship between the user and an audio emanation point.

27. The method of claim 17 further comprising:

o) obtaining the geographical position from the Global Positioning System.

28. The method of claim 17 further comprising:

p) playing a clip of audio sound which describes what the user is viewing or playing spatial audio redirecting the user'"'"'s attention to an item adjacent to the object viewed.

29. The method of claim 17 further comprising:

q) accessing a database to obtain clips of audio sound regionally related in the physical layout.

30. The method of claim 17 further comprising:

r) providing the user binaural stereo audio sound.

31. The method of claim 17 further comprising:

s) altering the perceived relative distance between the object viewed and the user by controlling the sound level and reverberations.

32. A telemetric method to provide a user individualized audio information of an object viewed based upon the user'"'"'s head orientation and geographical position in a mobile terminal wireless system, comprising:

a) generating and storing impulse response function in the terminal for a user;

b) activating a server and storing in a database a physical layout of an object viewed by the user;

c) activating and storing in the terminal head orientation and geographical position data of an object viewed by the user;

d) transmitting the stored head orientation and geographical position data to the server via an air link;

f) mapping the transmitted head orientation and geographical data to a physical layout of the object view from a database;

g) transmitting to the terminal and caching locally a database subset of clips of audio sound based upon the mapped head orientation and geographical data;

h) accessing a table of impulse response functions in the terminal based upon the mapped head orientation and geographical data; and

i) convolving the audio sound to give the user a perception that the audio sound is actually emanating from a point in the object viewed by the user.

33. The method of claim 32 further comprising:

(j) returning to step (h) and repeating the process when the user moves outside the perimeter of the locally cached database subset.

34. A telemetric contextually based audio system, a terminal comprising:

a) a transceiver including a compression/decompression amplifier for receiving audio signals;

b) a processor coupled to the transceiver and to an antenna via an I/O circuit;

c) a non-volatile memory coupled to the processor and storing program instructions for operating the terminal and processing generic audio clips into spatial representations;

d) a volatile memory coupled to the processor and storing data subsets descriptive of objects viewable by a user;

user head orientation obtained from a directional device and geographical positional of the user obtained from a geographical position determining source; and

e) convolving means convolving the audio signals to give the user a perception that an audio sound is actually emanating from a point in the object viewed by the user.

35. The terminal of claim 34 wherein the convolving means are Head Relative Transfer Functions (HRTF) stored in the non-volatile memory.

36. The terminal of claim 35 wherein multiple HRTFs are stored in the terminal.

37. The terminal of claim 35 wherein the HRTFs are optimized for an individual user'"'"'s localization.

38. The terminal of 35 wherein HTRFs are based on a user training sequence.

39. In a telemetric contextually based audio system, a server comprising:

a) a processor coupled to a signal source for receiving positional data of a terminal;

b) a storage device for storing a floor plan of objects viewable by a user;

c) audio related to viewable objects by a user stored in the floor plan;

e) receiving terminal transmissions providing users'"'"' head orientation of the user relative to a viewable object;

mapping the head orientation and geographical position to the floor plan; and

g) selecting means selecting a regionally significant part of the database content related to audio for transmission to the terminal.