INTEGRATED PHOTOGRAMMETRIC LIGHT COMMUNICATIONS POSITIONING AND INERTIAL NAVIGATION SYSTEM POSITIONING

• US 20140093249A1
• Filed: 09/29/2012
• Published: 04/03/2014
• Est. Priority Date: 09/29/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Roberts, Richard D., Yang, Xue
• Granted Patent: US 9,590,728 B2
• Time in Patent Office: Days
• US Class Current: 398/127
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G01C 21/1654 with electromagnetic compass
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01S 5/0264 at least one of the systems being a non-radio wave positioning system
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: H04B 10/116 Visible light communication

Abstract

A mobile device includes an inertial navigation system (INS) to measure inertial quantities associated with movement of the device, and estimate a kinematic state associated with the movement based on the measured inertial quantities. The device includes a light receiver to record light beams originating from lights at respective image positions in a sequence of images. The device photogrammetrically determines its position relative to the originating lights based on predetermined real-world positions and corresponding image positions of the lights. The device corrects the estimated kinematic state based on the photogrammetrically determined position, to produce a corrected estimated kinematic state.

Claims

1. A non-transitory computer readable medium encoded with a computer program, including instructions to cause a processor to:

receive measured inertial quantities resulting from movement of a device;

estimate a kinematic state associated with the movement based on the measured inertial quantities;

record, in the device, light beams originating from lights at respective image positions in a sequence of images;

photogrammetrically determine a position of the device relative to the originating lights based on predetermined real-world positions and corresponding ones of the image positions of the lights; and

correct the estimated kinematic state based on the photogrammetrically determined position, to produce a corrected estimated kinematic state.

2. The computer readable medium of claim 1, wherein the light beams are each modulated to convey information from which the predetermined real-world position of the corresponding originating light is determinable, the instructions further including instructions to cause the processor to:

demodulate the information from each recorded light beam; and

determine the predetermined real-world position of the originating light based on the corresponding demodulated information for that light.

3. The computer readable medium of claim 2, wherein:

the light beams are each modulated to convey an originating light identifier (ID) used to index the corresponding real-world position of the originating light with that ID;

the instructions to cause the processor to demodulate include instructions to cause the processor to demodulate the ID from each recorded light beam; and

the instruction to cause the processor to determine include instructions to cause the processor to access the predetermined real-world position of each originating light based on the demodulated ID.

4. The computer readable medium of claim 2, wherein the device includes a light imager to record the light beams at the image positions on the light imager, the instructions further including instructions to cause the processor to:

while demodulating each recorded light beam from the corresponding image position in the light imager, track movement of the image position for that light beam across the light imager, caused by the movement of the device, based on the estimated kinematic state.

5. The computer readable medium of claim 4, wherein the instructions to cause the processor to track include instructions to cause the processor to:

transform the estimated kinematic state from a navigation coordinate frame to an image plane coordinate frame of the light imager; and

predict where each beam position will move across the light imager based on the transformed estimated kinematic state.

6. The computer readable medium of claim 1, wherein the instructions to cause the processor to correct include instructions to cause the processor to:

determine an error in the estimated kinematic state based on the photogrammetrically determined position;

filter the determined error over time, to produce a correction; and

adjust the kinematic state estimate based on the correction.

7. The computer readable medium of claim 1, wherein:

the measured inertial quantities include a measured acceleration and a measured orientation of the device; and

the instructions to cause the processor to estimate include instructions to cause the processor to estimate a kinematic state to include an estimated position and an estimated velocity of the device, based on the measured acceleration and the measured orientation.

8. The computer readable medium of claim 7, wherein:

the instructions to cause the processor to estimate further include instructions to cause the processor to estimate an orientation of the device; and

the instructions to cause the processor to photogrammetrically determine include instructions to cause the processor to photogrammetrically determine the position based on the estimated orientation.

9. An apparatus, comprising:

one or more processing modules configured to;

estimate a kinematic state associated with movement of the apparatus based on measured inertial quantities associated with the movement;

access a recording, made in the apparatus, of light beams originating from lights at respective image positions in a sequence of images;

photogrammetrically determine a position of the apparatus relative to the originating lights based on predetermined real-world positions and corresponding ones of the image positions of the lights; and

correct the estimated kinematic state based on the photogrammetrically determined position, to produce a corrected estimated kinematic state.

10. The apparatus of claim 9, wherein the light beams are each modulated to convey information from which the predetermined real-world position of the corresponding originating light is determinable, and wherein the one or more processing modules are further configured to:

demodulate the information from each recorded light beam; and

determine the predetermined real-world position of the originating light based on the corresponding demodulated information for that light.

11. The apparatus of claim 10, wherein the light beams are each modulated to convey an originating light identifier (ID) used to index the corresponding real-world position of the originating light associated with that ID, and wherein the one or more processing modules are further configured to:

demodulate the ID from each recorded light beam; and

access the predetermined real-world position of each originating light based on the demodulated ID.

12. The apparatus of claim 10, further comprising a light imager to record the light beams at the image positions on the light imager, and wherein the one or more processing modules are further configured to:

while each recorded light beam is being demodulated from the corresponding image position in the light imager, track movement of the image position for that light beam across the light imager, caused by the movement of the device, based on the estimated kinematic state.

13. The apparatus of claim 12, wherein the one or more processing modules configured to track movement of the image position are configured to:

transform the estimated kinematic state from a navigation coordinate frame to an image plane coordinate frame of the light imager; and

predict where each beam position will move across the light imager based on the transformed estimated kinematic state.

14. The apparatus of claim 9, wherein the one or more processing modules configured to correct are configured to:

determine an error in the estimated kinematic state based on the photogrammetrically determined position;

filter the determined error over time, to produce a correction; and

adjust the kinematic state estimate based on the correction.

15. The apparatus of claim 9, wherein:

the measured quantities include a measured acceleration and a measured orientation of the apparatus; and

the one or more processing modules are further configured to estimate the kinematic state to include an estimated position and an estimated velocity of the apparatus.

16. The apparatus of claim 15, wherein:

the one or more processing modules are further configured to estimate the kinematic state to further include an estimated orientation of the device; and

the one or more processing modules are further configured to photogrammetrically determine the position based on the estimated orientation.

17. The apparatus of claim 9, further comprising:

an inertial measurement unit (IMU) to measure the inertial quantities;

a light receiver to record the light beams;

a communication system to communicate with a network;

a processor and memory to interface between the communication system and a user interface system; and

a housing,

wherein the IMU, the light receiver, the processor and memory, and the communication system, are positioned within the housing.

18. The apparatus of claim 17, wherein:

the communication system includes a wireless communication system; and

the housing includes a mobile hand-held housing to house the communication system, the processor, the user interface system, and a battery.

26. The method of claim 15, wherein:

the estimating further includes estimating an orientation of the device; and

the photogrammetrically determining uses the estimated orientation.

19. A method, comprising:

measuring inertial quantities resulting from movement of a device;

estimating a kinematic state associated with the movement based on the measured inertial quantities;

recording, in the device, light beams originating from lights at respective image positions in a sequence of images;

photogrammetrically determining a position of the device relative to the originating lights based on predetermined real-world positions and corresponding ones of the image positions of the lights; and

correcting the estimated kinematic state based on the photogrammetrically determined position, to produce a corrected estimated kinematic state.

20. The method of claim 19, wherein the light beams are each modulated to convey information from which the predetermined real-world position of the corresponding originating light is determinable, the method further comprising:

demodulating the information from each recorded light beam; and

determining the predetermined real-world position of the originating light based on the corresponding demodulated information for that light.

21. The method of claim 20, wherein:

the light beams are each modulated to convey an originating light identifier (ID) used to index the corresponding real-world position of the originating light with that ID;

the demodulating includes demodulating the ID from each recorded light beam; and

the determining includes accessing the predetermined real-world position of each originating light based on the demodulated ID.

22. The method of claim 20, wherein the device includes a light imager to record the light beams at the image positions on the light imager, the method further comprising:

while demodulating each recorded light beam from the corresponding image position in the light imager, tracking movement of the image position for that light beam across the light imager, caused by the movement of the device, based on the estimated kinematic state.

23. The method of claim 22, wherein the tracking includes:

transforming the estimated kinematic state from a navigation coordinate frame to an image plane coordinate frame of the light imager; and

predicting where each beam position will move across the light imager based on the transformed estimated kinematic state.

24. The method of claim 19, wherein the correcting includes:

determining an error in the estimated kinematic state based on the photogrammetrically determined position;

filtering the determined error over time, to produce a correction; and

adjusting the kinematic state estimate based on the correction.

25. The method of claim 19, wherein:

the measuring includes measuring an acceleration and an orientation of the device; and

the estimating includes estimating a kinematic state to include an estimated position and an estimated velocity of the device, based on the measured acceleration and the measured orientation.