Manual operation assistance with earpiece with 3D sound cues

• US 10,398,374 B2
• Filed: 08/14/2018
• Issued: 09/03/2019
• Est. Priority Date: 11/04/2016
• Status: Active Grant

• Current Assignee: Bragi GmbH
• Original Assignee: Bragi GmbH
• Inventors: Boesen, Peter Vincent, Dragicevic, Darko
• Primary Examiner(s): Islam, Mohammad K
• Application Number:
  US16/102,963
• Publication Number:
  US 20190000373A1
• Time in Patent Office: 385 Days
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/1124 Determining motor skills
• CPC Class Codes: A61B 5/486 Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/6817 Ear canal
• CPC Class Codes: H04R 1/1016 Earpieces of the intra-aural type
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]

Abstract

A method of providing audio feedback in response to a user performance using an earpiece includes steps of identifying a manual work operation to be performed by the user, wherein the identifying the manual work operation is performed by the earpiece, monitoring performance of the manual work operation by the user, wherein the monitoring the performance of the work operation is performed by the earpiece, generating 3D sound cues at the earpiece to assist in the performance of the manual work operation by the user, and outputting the 3D sound cues to the user at one or more speakers of the earpiece during the performance of the manual work operation by the user.

Claims

1. A method of providing audio feedback in response to a user performance using an earpiece comprising steps of:

identifying a manual operation to be performed by the user, wherein the identifying the manual operation is performed by the earpiece;

monitoring performance of the manual operation by the user by mapping movement of the user during the manual work operation as determined using one or more inertial sensors to acceptable movement for performing the manual work operation, wherein the monitoring the performance of the manual operation is performed by the earpiece;

generating 3D sound cues at the earpiece to assist in the performance of the manual operation by the user;

outputting the 3D sound cues to the user at one or more speakers of the earpiece during the performance of the manual operation by the user.

2. The method of claim 1 wherein the 3D sound cues are generated to be perceived as coming from a spatial location having a contextual relationship with the manual operation by the user.

3. The method of claim 1 wherein the monitoring the performance of the manual operation by the user is performed by monitoring audio of the user sensed by the earpiece during the manual operation using one or more microphones of the earpiece.

4. The method of claim 1 wherein the monitoring the performance of the manual operation by the user is performed by monitoring movement of the user during the manual operation using one or more inertial sensors of the earpiece and monitoring audio of the user sensed by the earpiece during the manual operation using one or more microphones of the earpiece.

5. The method of claim 1 wherein the 3D sound cues include voice.

6. The method of claim 1 wherein the manual operation is associated with therapy.

7. A method of providing audio feedback in response to a user performance using a set of earpieces including a left ear piece and a right earpiece, the method comprising steps of:

identifying a manual operation to be performed by the user, wherein the identifying the manual operation is performed by at least one of the left earpiece and the right earpiece;

monitoring performance of the manual operation by the user by mapping movement of the user during the manual work operation as determined using one or more inertial sensors to acceptable movement for performing the manual work operation, wherein the monitoring the performance of the manual operation is performed by the left earpiece and the right earpiece;

generating 3D sound cues to assist in the performance of the manual operation by the user, wherein the generating 3D sound cues is performed by at least one of the left earpiece and the right earpiece;

outputting the 3D sound cues to the user at one or more speakers of the left earpiece and one or more speakers of the right earpiece during the performance of the manual operation by the user.

8. The method of claim 7 wherein the 3D sound cues are generated to be perceived as coming from a spatial location having a contextual relationship with the manual operation by the user.

9. The method of claim 7 wherein the monitoring the performance of the manual operation by the user is performed by monitoring movement of the user during the manual operation using one or more inertial sensors of the left wireless earpiece and one or more inertial sensors of the right wireless earpiece.

10. The method of claim 7 wherein the monitoring the performance of the manual operation by the user is performed by monitoring audio of the user sensed during the manual operation using one or more microphones of the left wireless earpiece and one or more microphones of the right wireless earpiece.

11. The method of claim 7 wherein the monitoring the performance of the manual operation by the user is performed by monitoring movement of the user during the manual operation using one or more inertial sensors of the left wireless earpiece and one or more inertial sensors of the right wireless earpiece and monitoring audio of the user sensed by the earpiece during the manual operation using one or more microphones of the left wireless earpiece and one or more microphones of the right wireless earpiece.

12. The method of claim 7 wherein the 3D sound cues include voice audio.

13. The method of claim 7 wherein the manual operation is associated with therapy.