Augmented reality proximity sensing

• US 10,437,309 B2
• Filed: 03/27/2017
• Issued: 10/08/2019
• Est. Priority Date: 07/20/2010
• Status: Active Grant

• Current Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Milford, Peter N.
• Primary Examiner(s): Boddie, William
• Assistant Examiner(s): Parker, Jeffrey A
• Application Number:
  US15/469,687
• Publication Number:
  US 20170199560A1
• Time in Patent Office: 925 Days
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G09G 2330/021 Power management, e.g. power saving
• CPC Class Codes: G09G 2380/02 Flexible displays
• CPC Class Codes: H04M 1/0245 using open/close detection
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04W 52/0251 using monitoring of local events, e.g. events related to user activity
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management
• CPC Class Codes: Y02D 30/70 in wireless communication networks

Abstract

Technologies are generally described for employing proximity sensing mechanisms in augmented reality (AR) systems. Sensors strategically placed on AR eyeglasses in locations such as right and/or left arms, bridge, or flip-up feature of the eyeglasses may be used to detect AR eyeglass usage for power management and/or to provide user interface elements like volume control, display controls, user input, and comparable ones. According to some examples, the sensors may be mechanical sensors, capacitive sensors, optical sensors, inductive sensors, magnetic sensors, and/or similar components.

Claims

1. A system configured to determine an orientation of a body part of a wearer that wears a wearable display, the system comprising:

a plurality of proximity sensors, in communication with the wearable display, configured to generate a detection signal; and

a processor communicatively coupled to the plurality of proximity sensors and configured to perform or control performance of operations that comprise;

determine a position of the body part of the wearer based on the generated detection signal, wherein;

at least one proximity sensor of the plurality of proximity sensors is configured to provide, to the processor, a plurality of capacitance measurements that are based on at least two capacitances associated with a body of the wearer, and

to determine the position of the body part of the wearer, the processor is configured to perform or control performance of one or more operations that comprise;

construct a plurality of filtered time averages for the plurality of capacitance measurements; and

generate a plurality of filtered signals, wherein the plurality of filtered signals comprise a first filtered signal and a second filtered signal, wherein the first filtered signal is provided by a first plurality of proximity sensors, of the plurality of proximity sensors, and the second filtered signal is provided by a second plurality of proximity sensors, of the plurality of proximity sensors, wherein the first plurality of proximity sensors and the second plurality of proximity sensors are different, wherein each of the first filtered signal and the second filtered signal is generated based on at least a summation of two values, wherein each of the two values corresponds to a capacitance measurement of the plurality of capacitance measurements and a filtered time average of the plurality of filtered time averages, and wherein the capacitance measurement that corresponds to each value of the two values is provided by a different proximity sensor of the plurality of proximity sensors; and

determine the orientation of the body part of the wearer based on a respective time average of each of the first filtered signal and the second filtered signal.

2. The system of claim 1, wherein the plurality of proximity sensors include one or more of:

an inductive sensor, a magnetic sensor, a sonar sensor, an optical sensor, an Eddy-current sensor, a mechanical switch, a capacitive sensor, or a thermal infrared sensor.

3. The system of claim 1, wherein the plurality of the proximity sensors are configured to receive a user input to control at least one operational parameter of the wearable display.

4. The system according to claim 3, wherein the at least one operational parameter of the wearable display includes one of an audio control or a display control.

5. The system of claim 1, wherein the capacitance measurement, C_total, is determined by formula, C_shoulder+C_head, wherein the C_shoulder includes a capacitance associated with a shoulder of the wearer, and wherein the C_head includes a capacitance associated with a head of the wearer.

6. The system of claim 1, further comprising:

a movable part attached to a rim of the wearable display through at least one hinge, wherein the movable part includes at least one additional sensor.

7. The system of claim 6, wherein the movable part includes one of a display or a cover that is configured to block a see-through component of the wearable display.

8. The system of claim 6, wherein a status of the wearable display includes the movable part being in one of a flipped up position and a flipped down position.

9. The system of claim 8, wherein one of the at least one hinge and the at least one additional sensor is configured to detect a position of the movable part.

16. The system of claim 1, wherein the wearable display includes an augmented reality display.

17. The system of claim 1, wherein the wearable display includes a virtual reality display.

10. A method to control operational aspects of a wearable display, the method comprising:

receiving a sense signal;

determining a value of the received sense signal;

obtaining a plurality of capacitance measurements, wherein the plurality of capacitance measurements are obtained based on at least two capacitances associated with a wearer that wears the wearable display;

determining one or more positions of a body part of the wearer, a user input, and a status of the wearable display based on at least one of the received sense signal and the plurality of capacitance measurements, wherein the value of the received sense signal is compared to expected values in a look-up table to determine the user input and the status of the wearable display, and wherein;

a plurality of filtered time averages are constructed for the plurality of capacitance measurements, and

the determination of the one or more positions of the body part of the wearer is based on a plurality of filtered signals, wherein the plurality of filtered signals comprise a first filtered signal and a second filtered signal, wherein each of the first filtered signal and the second filtered signal is generated based on at least a summation of two values, and wherein each value of the two values corresponds to a capacitive measurement of the plurality of capacitance measurements and a filtered time average of the plurality of filtered time averages;

determining an orientation of the body part of the wearer based on a correlation of a positive value and a negative value of a respective time average of each of the first filtered signal and the second filtered signal; and

performing an action associated with an operational aspect of the wearable display in response to the determination of the one or more positions of the body part of the wearer, the orientation of the body part of the wearer, the user input, and the status of the wearable display.

11. The method of claim 10, wherein performing the action associated with the operational aspect of the wearable display includes one or more of:

powering up the wearable display, powering down the wearable display, transitioning the wearable display into a low power mode, adjusting audio controls, and adjusting display controls.

18. The method of claim 10, wherein the wearable display includes an augmented reality display.

19. The method of claim 10, wherein the wearable display includes a virtual reality display.

12. A non-transitory computer-readable medium that stores executable instructions that, in response to execution, cause one or more processors to perform or control performance of operations that comprise:

detect a sense signal that represents one or more of;

a position of a body part of a wearer that wears a wearable display, a user input, and a status of the wearable display;

determine whether to power up or power down the wearable display based on the status of the wearable AR display, wherein the status is determined by comparison of a value of the detected sense signal to expected values in a look-up table;

determine the position of the body part of the wearer based on the detected sense signal, wherein;

a plurality of capacitance measurements that are based on at least two capacitances associated with a body of the wearer are obtained, and

to determine the position of the body part of the wearer;

a plurality of filtered time averages are constructed for the plurality of capacitance measurements, and

a plurality of filtered signals are generated, wherein the plurality of filtered signals comprises a first filtered signal and a second filtered signal, wherein each of the first filtered signal and the second filtered signal is generated based on at least a summation of two values, and wherein each value of the two values corresponds to a capacitance measurement of the plurality of capacitance measurements and a filtered time average of the plurality of filtered time averages;

correlate the first filtered signal and the second filtered signal, wherein the correlation of the first filtered signal and the second filtered signal includes a correlation of a time average of the first filtered signal and a time average of the second filtered signal;

determine, based on the correlation of the time averages, an orientation of the body part of the wearer, wherein the determination of the orientation of the body part of the wearer includes a determination of a leftward orientation of the body part of the wearer based on a negative value of the time average of the first filtered signal and a positive value of the time average of the second filtered signal, and a determination of a rightward orientation of the body part of the wearer based on a positive value of the time average of the first filtered signal and a negative value of the time average of the second filtered signal;

infer a user response based on one or more of the position of the body part of the wearer, and the orientation of the body part of the wearer; and

determine an action associated with an operational aspect of the wearable display in response to one of the inferred user response or the sensed user input.

13. The non-transitory computer-readable medium of claim 12, wherein the status of the wearable display includes one or more of:

arms of the wearable display being in one of an open position or a closed position, and a movable part of the wearable display being in one of a flipped up position or a flipped down position.

14. The non-transitory computer-readable medium of claim 12, wherein the wearable display includes an augmented reality display.

15. The non-transitory computer-readable medium of claim 12, wherein the wearable display includes a virtual reality display.