System and method for audio augmented reality

• US 9,727,129 B2
• Filed: 06/27/2014
• Issued: 08/08/2017
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Group)
• Original Assignee: Harman International Industries Incorporated (Samsung Group)
• Inventors: Di Censo, Davide, Marti, Stefan
• Primary Examiner(s): Patel, Hemant
• Application Number:
  US14/318,038
• Publication Number:
  US 20150025662A1
• Time in Patent Office: 1,138 Days
• Field of Search: 700 94, 4554561-466, 348 1401- 1416, 370351-357, 370259-271, 379 37- 52, 379 671- 8828, 37920101, 704270-278, 709201-207, 709217-248
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A system, controller, and computer readable medium for providing an audio augmented reality to a user. The system can detect an object, event, or the like in the environment of a user and play a digitized sound and/or a synthesized sound in response to the detected object, event, or the like. The user can specify preferences for sounds played in response to the detected object or event.

Claims

1. A system, comprising:

a computer processor;

a memory that stores at least one user preference;

at least one acoustic transducer;

at least one context sensor that acquires sensor data; and

at least one database storing a plurality of data structures, wherein each data structure included in the plurality of data structures stores auditory data associated with a different sound,

wherein the computer processor;

determines, based on the sensor data, a type of environment;

selects, based on the type of environment and the at least one user preference, at least one of the plurality of data structures that stores a first audio recording of a sound of at least one of an object and an event included in the type of environment; and

transmits an audio signal that corresponds to the first audio recording to the at least one acoustic transducer.

2. The system of claim 1, wherein the plurality of data structures comprise a plurality of audio recordings, wherein the at least one user preference includes an association of a particular audio recording with a particular location or environmental context, and wherein the computer processor processes the particular audio recording upon a determined location or environmental context of the user corresponding to the particular location or environmental context.

3. The system of claim 1, wherein the plurality of data structures comprise at least one sound synthesizer model that simulates a sound when processed by the computer processor, wherein the at least one user preference includes an association of a particular sound synthesizer model with a particular location or environmental context, and wherein the computer processor processes the particular sound synthesizer model upon a determined location or environmental context of the user corresponding to the particular location or environmental context.

4. The system of claim 3, wherein at least one of the plurality of sound synthesizer models is modified based on the user preference, and wherein the modified sound synthesizer model generates a modified simulated sound.

5. The system of claim 1, wherein at least one context sensor comprises a global positioning system (GPS) receiver.

6. The system of claim 5, wherein the computer processor identifies the type of environment by comparing the determined location of the user to a database of locations.

7. The system of claim 1, wherein the at least one database is stored on the memory.

8. The system of claim 1, further comprising a transceiver, wherein the at least one database comprises a remote database, and wherein the computer processor communicates with the remote database using the transceiver.

18. The system of claim 1, wherein the at least one context sensor comprises at least one of an acoustic sensor and an image sensor.

19. The system of claim 18, wherein the computer processor determines the type of environment by identifying at least one of an object included in image data acquired by the image sensor and a sound included in acoustic data acquired by the acoustic sensor.

9. A controller, comprising:

a first signal input that receives a first signal that indicates at least one user preference;

a second signal input that receives sensor data;

a signal output that drives at least one acoustic transducer; and

computer logic that;

determines, based on the sensor data, a type of environment;

processes a data structure, selected from a plurality of data structures based on the type of environment and the at least one user preference, wherein the data structure stores a first audio recording of a sound of at least one of an object and an event included in the type of environment, and

generates an audio signal that corresponds to the first audio recording.

10. The controller of claim 9, further comprising a computer memory, wherein the plurality of data structures are stored in the computer memory.

11. The controller of claim 9, further comprising a transceiver, wherein the plurality of data structures are stored in a remote computer memory, and wherein the computer logic retrieves the selected data structure from the remote computer memory.

12. The controller of claim 9, wherein the first signal is provided by a user interface that enables a user to indicate at least one preference.

13. The controller of claim 9, wherein the second signal is provided by a context sensor, wherein the context sensor comprises a camera that captures an image of at least one of a location and an environmental context, and wherein the second signal includes the captured image.

14. The controller of claim 9, further comprising a wearable housing, wherein the first signal input, the second signal input, and the signal output are arranged on the housing, and wherein the computer logic is arranged within the housing.

15. A non-transitory computer readable medium containing instructions that, when executed by a processor, cause the processor to perform the steps of:

receiving at least one user preference;

determining a type of environment of the user based on sensor data received from a context sensor;

selecting a data structure from a plurality of data structures, based on the type of environment of the user and the at least one user preference, wherein the data structure stores a first audio recording of a sound of at least one of an object and an event included in the type of environment; and

processing the first audio recording to generate an audio signal for output to an acoustic transducer.

16. The non-transitory computer readable medium of claim 15, wherein the at least one user preference includes a first preference, wherein the first preference is adjustable on a continuum between a first limit and a second limit.

17. The non-transitory computer readable medium of claim 15, wherein the type of environment is determined based on a location of the user.