Telemetric contextually based spatial audio system integrated into a mobile terminal wireless system

• US 7,130,759 B2
• Filed: 12/23/2004
• Issued: 10/31/2006
• Est. Priority Date: 02/25/2003
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, Inc. (Zebra Technologies Corporation)
• Inventors: Katz, Joseph, Vollkommer, Richard M., Willins, Bruce A.
• Primary Examiner(s): Bui, Bryan
• Application Number:
  US11/019,313
• Publication Number:
  US 20050108646A1
• Time in Patent Office: 677 Days
• Field of Search: 702/151, 702/152, 702/153, 702/117, 702/122, 702/182, 702/150, 340/988, 340/989, 340/990, 709/227, 709/228, 713/165, 713/185
• US Class Current: 702/150
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01S 3/803 using amplitude comparison of signals derived from receiving transducers or transducer systems having differently-oriented directivity characteristics
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A telemetric contextually based spatial audio system is integrated into a mobile terminal including a headset and antenna. The headset includes an electronic compass which determines head orientation data for the user. The terminal'"'"'s geographical position can either be determined locally (e.g. onboard GPS) or by network triangulation. The orientation and position data are sent to a server which overlays the position and head orientation data onto a locally stored “floor plan”, describing an object(s) of interest. The floor plan relates the user'"'"'s viewpoint to a database of contextual audio clips. A regionally significant portion of the database and respective triggers are sent to the terminal. The terminal using the database information recalculates which audio clip(s) to play. The terminal then convolves the respective audio clip(s) via an impulse response function so as to be perceived by the user as emanating from the exact spatial position of the object specified by the system.

Claims

1. A telemetric contextually based audio system integrated into a mobile terminal in a wireless system, comprising:

a) A user transported mobile terminal coupled to a server via an air link;

b) a headset coupled to the terminal and including a directional indicator indicating the user'"'"'s head orientation relative to an object viewed by the user;

c) the mobile terminal including a transmitter transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) the server configured to;

(i) map the user'"'"'s head orientation and geographical position to a physical layout of the object viewed;

(ii) select and transmit to the terminal clips of audio sound descriptive of the object viewed based upon the user'"'"'s mapped position and head orientation; and

e) the mobile terminal further including a processor convolving the audio sound to give the user a perception that the audio sound is actually emanating from a point in the object viewed by the user.

2. A telemetric contextually based audio system integrated into a mobile terminal in a wireless system for instructing a user in the performance of a task, comprising:

a) A user transported mobile terminal coupled to a server via an air link;

b) a headset coupled to the terminal and including directional means indicating the user'"'"'s head orientation relative to a task to be performed by the user;

c) transmitting means transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) mapping means mapping the user'"'"'s head orientation and geographical position to a physical layout for the performance of the task; and

e) selecting and transmitting means selecting and transmitting to the terminal clips of audio sound instructing the user in the performance of the task based upon the user'"'"'s mapped position and head orientation in the physical layout.

3. The system of claim 2 wherein the task to be performed may be taken from a group of tasks, not limited to, directing a user in steps (i) to a location in the physical layout;

(ii) within a retail market;

(iii) in locating other users.

4. The system of claim 2 wherein the server accesses a database to obtain clips of audio sound related to the performance of the task in the physical layout.

5. The system of claim 2 further comprising:

f) data describing Head Relative Transfer Functions stored in the mobile terminal.

6. The system of claim 2 further comprising:

g) a database coupled to the server for dynamically determining an audio emanation point based on a user'"'"'s position.

7. The system of claim 6 wherein the database means stores the clips of audio sound, according to the steps in the task.

8. The system of claim 2 wherein the terminal plays a clip of audio sound which describes the step of the task to be performed by the user.

9. A method in a telemetric contextually based audio system integrated into a mobile terminal in a wireless system for instructing a user in the performance of a task, comprising:

a) coupling a user transported mobile terminal to a server via an air link;

b) coupling a headset to the terminal and including directional means indicating the user'"'"'s head orientation relative to a task to be performed by the user;

c) transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) mapping the user'"'"'s head orientation and geographical position to a physical layout for the performance of the task; and

e) selecting and transmitting to the terminal clips of audio sound instructing the user in the performance of the task based upon the user'"'"'s mapped position and head orientation in the physical layout.

10. The method of claim 9 wherein the task to be performed may be taken from a group of, tasks, not limited to, directing a user in steps (i) to a location in the physical layout;

(ii) within a retail market, and (iii) locating other users.

11. The method of claim 9 wherein the server accesses a database to obtain clips of audio sound related to the performance of the task in the physical layout.

12. The method of claim 9 further comprising:

f) storing Head Relative Transfer Functions in the mobile terminal.

13. The method of claim 9 further comprising:

g) dynamically determining an audio emanation point based on a user'"'"'s position using a database and the server.

14. The method of claim 13 wherein the database means stores the clips of audio sound, according to the steps in the task.

15. The method of claim 9 wherein the terminal plays a clip of audio sound which describes the step of the task to be performed by the user.

16. In a telemetric contextually based audio system, a wireless terminal for instructing a user in the performance of a task, comprising:

a) a transceiver including a compression/decompression amplifier for receiving audio signals;

b) a processor coupled to the transceiver and to an antenna via an I/O circuit;

c) a non-volatile memory coupled to the processor and storing program instructions for operating the terminal and processing generic audio clips into steps of a task; and

d) a volatile memory coupled to the processor and storing data subsets descriptive of tasks performed by a user;

user head orientation obtained from a directional device and geographical positional of the user obtained from a geographical position determining source, wherein the terminal updates its ephemeral data (current location and head orientation), and using the memory information recalculates which audio clip(s) to play.

17. The terminal of claim 16 wherein as the user'"'"'s changes steps in the task, either a Previously Presented audio clip is played or the remainder of the existing audio clip is seamlessly spliced into the audio output.

18. In a telemetric contextually based audio system, a server for instructing a user in the performance of a task, comprising:

a) a processor coupled to a signal source for receiving positional data of a wireless terminal;

b) a storage device for storing a floor plan for guidance in the performance of a task by a user;

c) audio clips related to steps of the task performed by the user stored with respect to the floor plan;

d) receiving means receiving terminal transmissions providing users'"'"' head orientation and geographic position in the performance of the task;

e) mapping means mapping the head orientation and geographical position to the task in the floor plan; and

f) selecting means selecting audio clips for transmission to the terminal for the performance of the task by the user.

19. The server of claim 18 wherein the server knowing the identity and location of the task(s) to be performed, the user'"'"'s geographical position and head orientation, selects and transmits audio clips, relative to the task to be performed.

20. A telemetric contextually based audio system integrated into a mobile terminal in a wireless system for instructing a user in the performance of a task, comprising:

a) A user transported mobile terminal coupled to a server via an air link;

b) a headset coupled to the terminal and including directional means indicating the user'"'"'s head orientation relative to a task to be performed by the user;

c) transmitting means transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) mapping means mapping the user'"'"'s head orientation and geographical position to a physical layout for the performance of the task; and

e) selecting and transmitting means selecting and transmitting to the terminal clips of audio sound instructing the user in the performance of the task based upon the user'"'"'s mapped position and head orientation in the physical layout, wherein the task to be performed may be taken from a group of tasks, not limited to, directing a user in steps (i) to a location in the physical layout;

(ii) within a retail market;

(iii) in locating other users, and wherein the server accesses a database to obtain clips of audio sound related to the performance of the task in the physical layout, and wherein the terminal plays a clip of audio sound which describes the step of the task to be performed by the user.

21. A method in a telemetric contextually based audio system integrated into a mobile terminal in a wireless system for instructing a user in the performance of a task, comprising:

a) coupling a user transported mobile terminal to a server via an air link;

b) coupling a headset to the terminal and including directional means indicating the user'"'"'s head orientation relative to a task to be performed by the user;

c) transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) mapping the user'"'"'s head orientation and geographical position to a physical layout for the performance of the task; and

e) selecting and transmitting to the terminal clips of audio sound instructing the user in the performance of the task based upon the user'"'"'s mapped position and head orientation in the physical layout wherein the task to be performed may be taken from a group of tasks, not limited to, directing a user in steps (i) to a location in the physical layout;

(ii) within a retail market, and (iii) locating other users, and wherein the server accesses a database to obtain clips of audio sound related to the performance of the task in the physical layout the terminal plays a clip of audio sound which describes the step of the task to be performed by the user.

22. A telemetric contextually based audio system integrated into a mobile terminal in a wireless system for instructing a user in the performance of a task, comprising:

a) a user transported mobile terminal coupled to a server via an air link;

b) a headset coupled to the terminal and including a directional indicator indicating the user'"'"'s head orientation relative to a task to be performed by the user;

c) a transmitter transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation; and

d) The server configured to;

(i) map the user'"'"'s head orientation and geographical position to a physical layout for the performance of the task; and

ii) select and transmit to the terminal clips of audio sound instructing the user in the performance of the task based upon the user'"'"'s mapped position and head orientation in the physical layout.

23. In a telemetric contextually based audio system, a server for instructing a user in the performance of a task, comprising:

a) a processor coupled to a signal source for receiving positional data of a wireless terminal;

b) a storage device for storing a floor plan for guidance in the performance of a task by a user;

c) audio clips related to steps of the task performed by the user stored with respect to the floor plan;

d) a receiver receiving terminal transmissions providing users'"'"' head orientation and geographic position in the performance of the task; and

e) the server configured to;

(i) map the head orientation and geographical position to the task in the floor plan; and

(ii) select audio clips for transmission to the terminal for the performance of the task by the user.

24. A telemetric contextually based audio system integrated into a mobile terminal in a wireless system for instructing a user in the performance of a task, comprising:

a) A user transported mobile terminal coupled to a server via an air link;

b) a headset coupled to the terminal and including a directional indicator indicating the user'"'"'s head orientation relative to a task to be performed by the user;

c) a transmitter transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) mapping means mapping the user'"'"'s head orientation and geographical position to a physical layout for the performance of the task; and

e) the server configured to;

(i) select and transmit to the terminal clips of audio sound instructing the user in the performance of the task based upon the user'"'"'s mapped position and head orientation in the physical layout, wherein the task to be performed may be taken from a group of tasks, not limited to, directing a user in steps (i) to a location in the physical layout;

(ii) within a retail market;

(iii) in locating other users, and wherein the server accesses a database to obtain clips of audio sound related to the performance of the task in the physical layout, and wherein the terminal plays a clip of audio sound which describes the step of the task to be performed by the user.

25. A method in a telemetric contextually based audio system integrated into a mobile terminal in a wireless system for instructing a user in the performance of a task, comprising:

a) coupling a user transported mobile terminal to a server via an air link;

b) coupling a headset to the terminal and including a directional indicator indicating the user'"'"'s head orientation relative to a task to be performed by the user;

c) transmitting signals to the server indicative of the user'"'"'s geographical position and head orientation;

d) the server configured to;

(i) map the user'"'"'s head orientation and geographical position to a physical layout for the performance of the task; and

(ii) select and transmit to the terminal clips of audio sound instructing the user in the performance of the task based upon the user'"'"'s mapped position and head orientation in the physical layout wherein the task to be performed may be taken from a group of tasks, not limited to, directing a user in steps (iii) to a location in the physical layout;

(iv) within a retail market, and (v) locating other users, and wherein the server accesses a database to obtain clips of audio sound related to the performance of the task in the physical layout the terminal plays a clip of audio sound which describes the step of the task to be performed by the user.