Audio assisted enrollment

• US 11,100,349 B2
• Filed: 06/28/2019
• Issued: 08/24/2021
• Est. Priority Date: 09/28/2018
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cohen, Sawyer I., Crosby, Justin D., Dave, Ruchir M., Johnson, Martin E., Pope, Benjamin J., Satongar, Darius A.
• Primary Examiner(s): Werner, Brian
• Application Number:
  US16/456,839
• Publication Number:
  US 20200104620A1
• Time in Patent Office: 788 Days
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/50 Maintenance of biometric data or enrolment thereof
• CPC Class Codes: G06V 40/67 by interactive indications to the user

Abstract

The present disclosure generally relates techniques for audio-assisted enrollment of biometric features. In some embodiments, methods and devices for assisting users with enrollment of biometric features, using spatial audio cues, are described.

Claims

1. An electronic device, comprising:

one or more biometric sensors;

one or more processors operably connected to a first speaker and second speaker; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

while detecting that a biometric feature is in a first orientation, relative to the one or more biometric sensors, that does not meet a first enrollment criteria, producing audio that is generated so as to simulate audio being produced from a first location that is not a location of the one or more biometric sensors and is not a location of the first speaker or the second speaker;

detecting a change in orientation of the biometric feature relative to the one or more biometric sensors from the first orientation to a second orientation;

in accordance with a determination that the second orientation of the biometric feature meets the first enrollment criteria;

storing first image data corresponding to the biometric feature from the one or biometric sensors captured while the biometric feature is in the second orientation; and

producing audio that is generated so as to simulate audio being produced from a second location that is not the location of the one or more biometric sensors and is not the location of the first speaker or the second speaker, wherein the first location is different from the second location.

2. The electronic device of claim 1, the one or more programs further including instructions for:

in accordance with a determination that the second orientation of the biometric feature does not meet the first enrollment criteria;

continuing to produce audio that is generated so as to simulate audio being produced from the first location.

3. The electronic device of claim 1, the one or more programs further including instructions for:

detecting a change in orientation of the biometric feature relative to the one or more biometric sensors from the second orientation to a third orientation; and

in accordance with a determination that the third orientation of the biometric feature meets a second enrollment criteria;

storing second image data corresponding to the biometric feature from the one or biometric sensors captured while the biometric feature is in the third orientation.

4. The electronic device of claim 1, wherein the first speaker and the second speaker are not in a fixed position relative to the one or more biometric sensors.

5. The electronic device of claim 4, wherein producing audio that is generated so as to simulate audio being produced from the first location includes modulating the audio of the first speaker and the second speaker to continue producing audio that is generated so as to simulate audio being produced from the first location, wherein modulating the audio is based on a change in position of the first speaker and the second speaker relative to the one or more biometric sensors.

6. The electronic device of claim 4, wherein detecting a change in orientation of the biometric feature is based on one or more motion sensors, wherein the one or more motion sensors are in a fixed position relative to the first speaker and second speaker.

7. The electronic device of claim 4, wherein the first location is in a direction perpendicular to a plane formed by the first speaker, the second speaker, and at least one of the one or more biometric sensors.

8. The electronic device of claim 4, wherein the electronic device is operably connected to a third speaker and a fourth speaker and wherein producing audio that is generated so as to simulate audio being produced from the first location includes:

determining if the first speaker and the second speaker are operable; and

in accordance with a determination that the first speaker and the second speaker are operable, foregoing producing audio using the third speaker and the fourth speaker.

9. The electronic device of claim 1, wherein the first speaker and the second speaker are in fixed positions relative to the one or more biometric sensors.

10. The electronic device of claim 9, wherein producing audio that is generated so as to simulate audio being produced from the first location includes foregoing modulating the audio of the first speaker and the second speaker based on a change in position of the biometric feature relative to the one or more biometric sensors.

11. The electronic device of claim 1, wherein detecting a change in the orientation of the biometric feature is based on data from the one or more biometric sensors.

12. The electronic device of claim 1, the one or more programs further including instructions for:

in accordance with a determination that the second orientation does not meet the first enrollment criteria and in accordance with a determination that a feedback criteria is met;

producing audio that is generated so as to simulate audio being produced from a third location that is not the location of the one or more biometric sensors, wherein the third location is different from the first location.

13. The electronic device of claim 12, the one or more programs further including instructions for:

in accordance with a determination that the second orientation does not meet the first enrollment criteria and in accordance with a determination that the feedback criteria is not met;

continuing to produce audio that is generated so as to simulate audio being produced from the first location.

14. The electronic device of claim 12, wherein the first enrollment criteria includes a target orientation and wherein the third location is determined based on a difference between the target orientation and the second orientation.

15. The electronic device of claim 1, wherein producing audio that is generated so as to simulate audio being produced from the first location includes:

detecting, using the one or more biometric sensors, a distance a user is from the one or biometric sensors; and

determining the first location based on the distance the user is from the one or more biometric sensors.

16. The electronic device of claim 1, the one or more programs further including instructions for:

prior to producing audio that is generated so as to simulate audio being produced from the first location, detecting a user physical characteristic,

wherein producing audio that is generated so as to simulate audio being produced from the first location includes modulating the audio based on the user physical characteristic.

17. The electronic device of claim 1, the one or more programs further including instructions for:

receiving a request to access a restricted function of the electronic device; and

in response to the request to access the restricted function of the device;

in accordance with a determination that biometric authentication criteria has been met based on biometric data provided by the one or more biometric sensors, providing access to the restricted function of the device; and

in accordance with a determination that biometric authentication criteria has not been met based on biometric data provided by the one or more biometric sensors, forgoing providing access to the restricted function of the device.

18. The electronic device of claim 1, wherein:

the biometric feature is at least a portion of a face; and

the first image data obtained from the one or more biometric sensors includes biometric data associated with the at least a portion of the face.

19. The electronic device of claim 18, wherein the first enrollment criteria includes a criterion that is satisfied when the biometric feature is oriented relative to the one or more biometric sensors in a predetermined manner.

20. The electronic device of claim 1, wherein producing audio that is generated so as to simulate audio being produced from the first location includes applying one or more functions selected from the group consisting of:

a head-related transfer function, an interaural time difference function, and a cross cancellation function.

21. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device having one or more biometric sensors, and operably connected to a first speaker and a second speaker, the one or more programs including instructions for:

while detecting that a biometric feature is in a first orientation, relative to the one or more biometric sensors, that does not meet a first enrollment criteria, producing audio that is generated so as to simulate audio being produced from a first location that is not a location of the one or more biometric sensors and is not a location of the first speaker or the second speaker;

detecting a change in orientation of the biometric feature relative to the one or more biometric sensors from the first orientation to a second orientation;

in accordance with a determination that the second orientation of the biometric feature meets the first enrollment criteria;

storing first image data corresponding to the biometric feature from the one or biometric sensors captured while the biometric feature is in the second orientation; and

producing audio that is generated so as to simulate audio being produced from a second location that is not the location of the one or more biometric sensors and is not the location of the first speaker or the second speaker, wherein the first location is different from the second location.

23. The non-transitory computer-readable storage medium of claim 21, the one or more programs further including instructions for:

in accordance with a determination that the second orientation of the biometric feature does not meet the first enrollment criteria;

continuing to produce audio that is generated so as to simulate audio being produced from the first location.

24. The non-transitory computer-readable storage medium of claim 21, the one or more programs further including instructions for:

detecting a change in orientation of the biometric feature relative to the one or more biometric sensors from the second orientation to a third orientation; and

in accordance with a determination that the third orientation of the biometric feature meets a. second enrollment criteria;

storing second image data corresponding to the biometric feature from the one or biometric sensors captured while the biometric feature is in the third orientation.

25. The non-transitory computer-readable storage medium of claim 21, wherein the first speaker and the second speaker are not in a fixed position relative to the one or more biometric sensors.

26. The non-transitory computer-readable storage medium of claim 25, wherein producing audio that is generated so as to simulate audio being produced from the first location includes modulating the audio of the first speaker and the second speaker to continue producing audio that is generated so as to simulate audio being produced from the first location, wherein modulating the audio is based on a change in position of the first speaker and second speaker relative to the one or more biometric sensors.

27. The non-transitory computer-readable storage medium of claim 25, wherein detecting a change in orientation of the biometric feature is based on one or more motion sensors, wherein the one or more motion sensors are in a fixed position relative to the first speaker and second speaker.

28. The non-transitory computer-readable storage medium of claim 25, wherein the first location is in a direction perpendicular to a plane formed by the first speaker, the second speaker, and at least one of the one or more biometric sensors.

29. The non-transitory computer-readable storage medium of claim 25, wherein the electronic device is operably connected to a third speaker and a fourth speaker and wherein producing audio that is generated so as to simulate audio being produced from the first location includes:

determining if the first speaker and second speaker are operable; and

in accordance with a determination that the first speaker and the second speaker are operable, foregoing producing audio using the third speaker and the fourth speaker.

30. The non-transitory computer-readable storage medium of claim 21, wherein the first speaker and the second speaker are in fixed positions relative to the one or more biometric sensors.

31. The non-transitory computer-readable storage medium of claim 30, wherein producing audio that is generated so as to simulate audio being produced from the first location includes foregoing modulating the audio of the first speaker and the second speaker based on a change in position of the biometric feature relative to the one or more biometric sensors.

32. The non-transitory computer-readable storage medium of claim 21, wherein detecting a change in the orientation of the biometric feature is based on data from the one or more biometric sensors.

33. The non-transitory computer-readable storage medium of claim 21, the one or more programs further including instructions for:

in accordance with a determination that the second orientation does not meet the first enrollment criteria and in accordance with a determination that a feedback criteria is met;

producing audio that is generated so as to simulate audio being produced from a third location that is not the location of the one or more biometric sensors, wherein the third location is different from the first location.

34. The non-transitory computer-readable storage medium of claim 33, the one or more programs further including instructions for:

in accordance with a determination that the second orientation does not meet the first enrollment criteria and in accordance with a determination that the feedback criteria is not met;

continuing to produce audio that is generated so as to simulate audio being produced from the first location.

35. The non-transitory computer-readable storage medium of claim 33, wherein the first enrollment criteria includes a target orientation and wherein the third location is determined based on a difference between the target orientation and the second orientation.

36. The non-transitory computer-readable storage medium of claim 21, wherein producing audio that is generated so as to simulate audio being produced from the first location includes:

detecting, using the one or more biometric sensors, a distance a user is from the one or biometric sensors; and

determining the first location based on the distance the user is from the one or more biometric sensors.

37. The non-transitory computer-readable storage medium of claim 21, the one or more programs further including instructions for:

prior to producing audio that is generated so as to simulate audio being produced from the first location, detecting a user physical characteristic,

wherein producing audio that is generated so as to simulate audio being produced from the first location includes modulating the audio based on the user physical characteristic.

38. The non-transitory computer-readable storage medium of claim 21, the one or more programs further including instructions for:

receiving a request to access a restricted function of the electronic device; and

in response to the request to access the restricted function of the device;

in accordance with a determination that biometric authentication criteria has been met based on biometric data provided by the one or more biometric sensors, providing access to the restricted function of the device; and

in accordance with a determination that biometric authentication criteria has not been met based on biometric data provided by the one or more biometric sensors, forgoing providing access to the restricted function of the device.

39. The non-transitory computer-readable storage medium of claim 21, wherein:

the biometric feature is at least a portion of a face; and

the first image data obtained from the one or more biometric sensors includes biometric data associated with the at least a portion of the face.

40. The non-transitory computer-readable storage medium of claim 39, wherein the first enrollment criteria includes a criterion that is satisfied when the biometric feature is oriented relative to the one or more biometric sensors in a predetermined manner.

41. The non-transitory computer-readable storage medium of claim 21, wherein producing audio that is generated so as to simulate audio being produced from the first location includes applying one or more functions selected from the group consisting of:

a head-related transfer function, an interaural time difference function, and a cross cancellation function.

22. A method comprising:

at an electronic device having one or more biometric sensors, and operably connected to a first speaker and a second speaker;

while detecting that a biometric feature is in a first orientation, relative to the one or more biometric sensors, that does not meet a first enrollment criteria, producing audio that is generated so as to simulate audio being produced from a first location that is not a location of the one or more biometric sensors and is not a location of the first speaker or the second speaker;

detecting a change in orientation of the biometric feature relative to the one or more biometric sensors from the first orientation to a second orientation;

in accordance with a determination that the second orientation of the biometric feature meets the first enrollment criteria;

storing first image data corresponding to the biometric feature from the one or biometric sensors captured while the biometric feature is in the second orientation; and

producing audio that is generated so as to simulate audio being produced from a second location that is not the location of the one or more biometric sensors and is not the location of the first speaker or the second speaker, wherein the first location is different from the second location.

42. The method of claim 22, further comprising:

in accordance with a determination that the second orientation of the biometric feature does not meet the first enrollment criteria;

continuing to produce audio that is generated so as to simulate audio being produced from the first location.

43. The method of claim 22, further comprising:

detecting a change in orientation of the biometric feature relative to the one or more biometric sensors from the second orientation to a third orientation; and

in accordance with a determination that the third orientation of the biometric feature meets a second enrollment criteria;

storing second image data corresponding to the biometric feature from the one or biometric sensors captured while the biometric feature is in the third orientation.

44. The method of claim 22, wherein the first speaker and the second speaker are not in a fixed position relative to the one or more biometric sensors.

45. The method of claim 44, wherein producing audio that is generated so as to simulate audio being produced from the first location includes modulating the audio of the first speaker and the second speaker to continue producing audio that is generated so as to simulate audio being produced from the first location, wherein modulating the audio is based on a change in position of the first speaker and second speaker relative to the one or more biometric sensors.

46. The method of claim 44, wherein detecting a change in orientation of the biometric feature is based on one or more motion sensors, wherein the one or more motion sensors are in a fixed position relative to the first speaker and second speaker.

47. The method of claim 44, wherein the first location is in a direction perpendicular to a plane formed by the first speaker, the second speaker, and at least one of the one or more biometric sensors.

48. The method of claim 44, wherein the electronic device is operably connected to a third speaker and a fourth speaker and wherein producing audio that is generated so as to simulate audio being produced from the first location includes:

determining if the first speaker and second speaker are operable; and

in accordance with a determination that the first speaker and the second speaker are operable, foregoing producing audio using the third speaker and the fourth speaker.

49. The method of claim 22, wherein the first speaker and the second speaker are in fixed positions relative to the one or more biometric sensors.

50. The method of claim 44, wherein producing audio that is generated so as to simulate audio being produced from the first location includes foregoing modulating the audio of the first speaker and the second speaker based on a change in position of the biometric feature relative to the one or more biometric sensors.

51. The method of claim 22, wherein detecting a change in the orientation of the biometric feature is based on data from the one or more biometric sensors.

52. The method of claim 22, further comprising:

in accordance with a determination that the second orientation does not meet the first enrollment criteria and in accordance with a determination that a feedback criteria is met;

producing audio that is generated so as to simulate audio being produced from a third location that is not the location of the one or more biometric sensors, wherein the third location is different from the first location.

53. The method of claim 52, further comprising:

in accordance with a determination that the second orientation does not meet the first enrollment criteria and in accordance with a determination that the feedback criteria is not met;

continuing to produce audio that is generated so as to simulate audio being produced from the first location.

54. The method of claim 52, wherein the first enrollment criteria includes a target orientation and wherein the third location is determined based on a difference between the target orientation and the second orientation.

55. The method of claim 22, wherein producing audio that is generated so as to simulate audio being produced from the first location includes:

detecting, using the one or more biometric sensors, a distance a user is from the one or biometric sensors; and

determining the first location based on the distance the user is from the one or more biometric sensors.

56. The method of claim 22, further comprising:

prior to producing audio that is generated so as to simulate audio being produced from the first location, detecting a user physical characteristic,

wherein producing audio that is generated so as to simulate audio being produced from the first location includes modulating the audio based on the user physical characteristic.

57. The method of claim 22, further comprising:

receiving a request to access a restricted function of the electronic device; and

in response to the request to access the restricted function of the device;

in accordance with a determination that biometric authentication criteria has been met based on biometric data provided by the one or more biometric sensors, providing access to the restricted function of the device; and

in accordance with a determination that biometric authentication criteria has not been met based on biometric data provided by the one or more biometric sensors, forgoing providing access to the restricted function of the device.

58. The method of claim 22, wherein:

the biometric feature is at least a portion of a face; and

the first image data obtained from the one or more biometric sensors includes biometric data associated with the at least a portion of the face.

59. The method of claim 58, wherein the first enrollment criteria includes a criterion that is satisfied when the biometric feature is oriented relative to the one or more biometric sensors in a predetermined manner.

60. The method of claim 22, wherein producing audio that is generated so as to simulate audio being produced from the first location includes applying one or more functions selected from the group consisting of:

a head-related transfer function, an interaural time difference function, and a cross cancellation function.