Positioning using light capturing sensors

• US 10,291,320 B2
• Filed: 03/31/2017
• Issued: 05/14/2019
• Est. Priority Date: 03/31/2017
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gummadi, Bapineedu Chowdary, Agnihotri, Hem, Babbadi, Venkata A Naidu
• Primary Examiner(s): Payne, David C
• Assistant Examiner(s): Ismail, Omar S
• Application Number:
  US15/476,177
• Publication Number:
  US 20180287700A1
• Time in Patent Office: 774 Days
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1698 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone details of antennas disposed inside a computer H01Q1/2266
• CPC Class Codes: H04B 10/116 Visible light communication
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass

Abstract

Techniques described herein, which may provide for a location determination of a mobile device, can also provide for the determination of a viewing direction of a user of the mobile device. In particular, a user can wear a head-mountable apparatus with one or more light sensors configured to detect light from one or more modulated light sources. Using this information, not only may a location of the head-mountable apparatus be determined, but also an orientation, which can enable a determination of an approximate direction the user is looking.

Claims

1. A method of view direction determination, the method comprising:

obtaining, at a mobile device configured to receive sensor data generated by one or more light sensors of a head-mountable apparatus via wireless communication, a request to determine an approximate user view direction, wherein;

the approximate user view direction comprises an approximate field of view of a user wearing the head-mountable apparatus, and the sensor data is generated by the one or more light sensors in response to detecting light from one or more modulated light sources;

based on the request to determine the approximate user view direction, sending a request for the sensor data via wireless communication from the mobile device to the head-mountable apparatus;

receiving, in response to sending the request for the sensor data, the sensor data;

obtaining, for each of the one or more modulated light sources, location information of the respective modulated light source;

determining, with a processing unit of the mobile device, the approximate user view direction based, at least in part, on the sensor data and the location information; and

generating, with a processing unit, information indicative of the approximate user view direction.

2. The method of claim 1, wherein:

the one or more light sensors comprises a camera; and

determining the approximate user view direction comprises determining, from the sensor data, an angle of arrival of light received from the one or more modulated light sources.

3. The method of claim 1, wherein determining the approximate user view direction is based on:

a known direction the one or more light sensors are facing with respect to the head-mountable apparatus, and

an approximate location of the head-mountable apparatus on the user'"'"'s head.

4. The method of claim 1, further comprising performing a calibration function to determine an approximate location of the head-mountable apparatus on the user'"'"'s head.

5. The method of claim 1, further comprising providing the generated information indicative of the determination of the approximate user view direction to an application executed by a computing device.

6. The method of claim 1, further comprising:

obtaining information indicating a location of an object; and

determining the user is looking at the object based, at least in part, on the determination of the approximate user view direction and the indicated location of the object.

7. The method of claim 1, wherein receiving the sensor data comprises receiving, at the mobile device, sensor data for at least two photodiode measurements of received modulated light, wherein obtaining the location information comprises obtaining information indicative of an identity of each of the one or more modulated light sources associated with the at least two photodiode measurements based on decoding the received modulated light.

8. The method of claim 7, wherein the at least two photodiode measurements comprise at least a first measurement from a first photodiode and a second measurement from a second, different photodiode, the at least two photodiode measurements taken during a measurement time, wherein the received modulated light includes received modulated light emitting diode (LED) light and wherein the first photodiode and the second photodiode are configured to detect the LED light.

9. The method of claim 7, wherein the sensor data received at the mobile device comprises at least one data type selected from the group consisting of decoded sensor data and modulated sensor data.

10. A mobile device enabling view direction determination, the mobile device comprising:

a wireless communication interface configured to receive sensor data generated by one or more light sensors of a head-mountable apparatus via wireless communication, wherein the sensor data is generated in response to detecting light from one or more modulated light sources;

a memory; and

a processing unit communicatively coupled with the wireless communication interface and the memory and configured to;

obtain a request to determine an approximate user view direction, wherein the approximate user view direction comprises an approximate field of view of a user wearing the head-mountable apparatus;

based on the request to determine the approximate user view direction, send a request for the sensor data via wireless communication from the mobile device to the head-mountable apparatus;

receive, via the wireless communication interface and in response to sending the request for the sensor data, sensor data generated by one or more light sensors of the head-mountable apparatus;

obtain, for each of the one or more modulated light sources, location information of the respective modulated light source;

determine the approximate user view direction based, at least in part, on the sensor data and the location information; and

generate information indicative of the approximate user view direction.

11. The mobile device of claim 10, wherein the processing unit is configured to determine the approximate user view direction by determining, from sensor data comprising data from a camera, an angle of arrival of light received from the one or more modulated light sources.

12. The mobile device of claim 10, wherein the processing unit is configured to determine the approximate user view direction further based on:

a known direction the one or more light sensors are facing with respect to the head-mountable apparatus, and

an approximate location of the head-mountable apparatus on the user'"'"'s head.

13. The mobile device of claim 10, wherein the processing unit is further configured to perform a calibration function to determine an approximate location of the head-mountable apparatus on the user'"'"'s head.

14. The mobile device of claim 10, wherein the processing unit is further configured to provide the generated information indicative of the determination of the approximate user view direction to an application executed by a computing device.

15. The mobile device of claim 10, wherein the processing unit is further configured to:

obtain information indicating a location of an object; and

determine the user is looking at the object based, at least in part, on the determination of the approximate user view direction and the indicated location of the object.

16. The mobile device of claim 10, wherein the processing unit is configured to:

receive the sensor data by receiving sensor data for at least two photodiode measurements of received modulated light; and

obtain the location information by obtaining information indicative of an identity of each of the one or more modulated light sources associated with the at least two photodiode measurements based on decoding the received modulated light.

17. The mobile device of claim 16, wherein the at least two photodiode measurements comprise at least a first measurement from a first photodiode and a second measurement from a second, different photodiode, the at least two photodiode measurements taken during a measurement time.

18. The mobile device of claim 16, wherein the processing unit is configured to receive sensor data comprising at least one data type selected from the group consisting of decoded sensor data and modulated sensor data.

19. An apparatus comprising:

means for obtaining a request to determine an approximate user view direction, wherein the approximate user view direction comprises an approximate field of view of a user wearing a head-mountable apparatus;

means for sending, via wireless communication and based on the request to determine the approximate user view direction, a request for sensor data generated by one or more light sensors of the head-mountable apparatus in response to detecting light from one or more modulated light sources;

means for receiving, via wireless communication and in response to sending the request for the sensor data, sensor data generated by one or more light sensors of the head-mountable apparatus;

means for obtaining, for each of the one or more modulated light sources, location information of the respective modulated light source;

means for determining the approximate user view direction based, at least in part, on the sensor data and the location information; and

means for generating information indicative of the approximate user view direction.

20. The apparatus of claim 19, wherein the means for determining the approximate user view direction comprise means for determining, from the sensor data, an angle of arrival of light received from the one or more modulated light sources.

21. The apparatus of claim 19, further comprising means for performing a calibration function to determine an approximate location of the head-mountable apparatus on the user'"'"'s head.

22. A non-transitory computer-readable medium having instructions embedded thereon for view direction determination, the instructions comprising computer code for:

obtaining a request to determine an approximate user view direction, wherein the approximate user view direction comprises an approximate field of view of a user wearing a head-mountable apparatus;

based on the request to determine the approximate user view direction, sending, via wireless communication from a mobile device to the head-mountable apparatus, a request for sensor data generated by one or more light sensors of the head-mountable apparatus in response to detecting light from one or more modulated light sources;

receiving, via wireless communication with the head-mountable apparatus and in response to sending the request for sensor data, sensor data generated by one or more light sensors of the head-mountable apparatus;

obtaining, for each of the one or more modulated light sources, location information of the respective modulated light source;

determining the approximate user view direction based, at least in part, on the sensor data and the location information; and

generating information indicative of the approximate user view direction.

23. The non-transitory computer-readable medium of claim 22, wherein the computer code for determining the approximate user view direction comprises computer code for determining, from the sensor data, an angle of arrival of light received from the one or more modulated light sources.