Audiovisual Surround Augmented Reality (ASAR)

• US 20170153866A1
• Filed: 07/03/2014
• Published: 06/01/2017
• Est. Priority Date: 07/03/2014
• Status: Active Application

• Current Assignee: Reality Plus Ltd.
• Original Assignee: Imagine Mobile Augmented Reality, Ltd.
• Inventors: Grinberg, Daniel, Kahane, Anat, PORAT, Ori, Cohen, Moran
• Application Number:
  US15/323,417
• Publication Number:
  US 20170153866A1
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04R 1/04 Structural association of microphone with electric circuitry therefor in deaf-aid sets H04R25/00
• CPC Class Codes: H04R 2499/15 Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence

Abstract

A system and method to enable realistic augmented reality 2D or 3D audiovisual imagery, integrating virtual object(s) and audio source data in the vicinity of a user/wearer of a head-mounted device (HMD), the HMD integrated with a mobile communication device. The system includes a HMD to facilitate enhancement of the user/wearer'"'"'s audiovisual capabilities, a mobile communication device integrated with the HMD. The system also includes a dedicated system mounted on the HMD and comprising at least an embedded software solution, at least four (4) miniature speakers mounted on the HMD and configured to optimize the audio provided to the user/wearer and an inertial measurement unit (IMU) mounted on the HMD for processing the data on and through the speakers according to the data imputed to the software instructions, thereby providing realistic sound in the HMD and anchoring of sounds deriving from virtual objects in the real world.

Claims

1. A system for providing to a user audio source data associated with an object, the system comprising:

a head-mounted device (HMD) including;

a software module;

one or more speakers configured to provide sound associated with the object; and

an inertial measurement unit (IMU) for providing kinetic data based on motion of the HMD,

wherein the software module processes the audio source data and kinetic data to provide sound to the user as if the sound were anchored to said objects, the object being fixed to a specific position independent of the movement of the HMD.

2. The system of claim 1, comprising a mobile communication device, said mobile device configured to be in communication with said HMD.

3. The system of claim 1 wherein said HMD is configured to provide a true augmented reality to the user.

4. The system claim 1, wherein said objects are virtual objects. The system of claim 4, wherein the virtual objects are selected from the group comprising:

a virtual image of a person;

a virtual ball;

a virtual ping-pong table;

a virtual screen; and

a virtual keyboard.

6. The system of claim 3, wherein the HMD is configured to provide the virtual object synchronously in sight and sound.

7. A computerized method for enabling realistic augmented reality, the method comprising:

distributing one or more speakers along a frame of a head mounted device (HMD);

using an inertial measurement unit (IMU) to sense movement of the HMD;

providing sound to the speakers; and

;

using data from the IMU to adjust the volume of the sound from each speaker according to a distance and angle of a user of the HMD to a virtual object;

whereby the sounds of the speakers appear to originate from the virtual object.

8. The method of claim 7, further comprising adjusting the volume each speaker so that the sounds provided to the user are anchored to one or more specific objects/images fixed to (a) specific position(s) independent of the movement of the HMD.

9. The method of claim 8, further comprising:

creating a realistic audio scene in reverse direction to and proportional to the viewer/listener'"'"'s head movements and positioning of a virtual object(s) in the real world relative to the user/wearer, according to the user'"'"'s angular head movement around an imaginary lengthwise axis, from head-to-toe (yaw), as measured by the IMU;

compensating for the user'"'"'s angular head movements around imaginary axes for pitch and roll, as measured by the IMU; and

establishing audio markers of the virtual objects in the real world using the IMU, and defining in real time the relative positioning of the user/wearer relative to the audio data and virtual object audio markers.

10. The method of claim 7 comprising enabling realistic augmented reality in 2D or 3D.

11. The method of claim 7 wherein the HMD is configured to facilitate enhancement of the user audiovisual capabilities.

12. The method of claim 7 comprising integrating a mobile communication device with the HMD.

13. The method of claim 7 wherein the speakers are configured to optimize the audio provided to the user/wearer.

14. The method of claim 7 comprising mounting the IMU on the HMD.

15. The method of claim 13 comprising processing the data on and through the speakers by the IMU according to data imputed to software instructions.

19. The method of claim 15, wherein the data is audio and visual data.

20. The method of claim 19, wherein the visual data is data relating to real objects.

16. The method of claim 5, wherein the data is audio data.

17. The method of claim 16, wherein the audio data is voice data.

18. The method of claim 16, wherein the audio data is musical data.