Dynamically created and updated indoor positioning map

• US 10,395,116 B2
• Filed: 10/29/2015
• Issued: 08/27/2019
• Est. Priority Date: 10/29/2015
• Status: Active Grant

• Current Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Original Assignee: Hand Held Products Incorporated (Honeywell International Inc.)
• Inventors: Todeschini, Erik, Patel, Mehul, Norton, Kent
• Primary Examiner(s): Nolan, Peter D
• Application Number:
  US14/926,442
• Publication Number:
  US 20170124396A1
• Time in Patent Office: 1,398 Days
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01S 5/013 Identifying areas in a building
• CPC Class Codes: G01S 5/02585 at least one of the measurements being a non-radio measurement
• CPC Class Codes: G01S 5/0264 at least one of the systems being a non-radio wave positioning system
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A system for creating and/or dynamically updating indoor positioning maps includes an augmented reality (AR) device and a computing device communicatively coupled to the AR device. The AR device has a display for displaying AR content to a user that overlaps the AR device'"'"'s perspective view of an environment. The AR device also has one or more depth sensors for gathering mapping data of physical objects in the environment. The computing device has a processor that is configured by software to create a three-dimensional (3D) indoor positioning map of the environment in a building based on the mapping data gathered by the AR device, and/or dynamically update the 3D indoor positioning map of the environment in the building based on the mapping data gathered by the AR device.

Claims

1. A system for creating and dynamically updating indoor positioning maps comprising:

an augmented reality (AR) device comprising;

a display for displaying AR content to a user, the AR content overlapping the AR device'"'"'s perspective view of an environment; and

one or more depth sensors for gathering mapping data of physical objects in the environment;

a computing device communicatively coupled to the AR device, the computing device comprising a processor that is configured by software to;

dynamically create a three-dimensional (3D) indoor positioning map of the environment in a building based on the mapping data gathered by the AR device;

create guidance AR content in response to the mapping data of the physical objects in the environment in the 3D indoor positioning map being older than a refresh period, wherein the guidance AR content is created corresponding to the AR device'"'"'s perspective view of the environment for guidance;

transmit the 3D indoor positioning map with the guidance AR content to the display for the guidance to gather updated mapping data; and

dynamically update the 3D indoor positioning map of the environment in the building based on the updated mapping data gathered by the AR device when at least the mapping data of the physical objects in the environment in the 3D indoor positioning map is older than the refresh period.

2. The system according to claim 1, wherein the creation of the 3D indoor positioning map and/or the dynamically updating of the 3D indoor positioning map includes utilizing simultaneous location and mapping (SLAM) techniques for positioning, wherein the SLAM techniques utilized for positioning include collecting environmental data at various points within the building, where the environmental data includes all available RF signals, magnetic fields, lighting conditions, GPS, three dimensional imagery, ambient sound, and/or combinations thereof, wherein:

the RF signal information contains a signal strength, a transmit power, and/or an identifier for an RF source; and

the light contains an intensity, a color and/or a frequency of light.

3. The system according to claim 2, wherein the AR device is configured to record an orientation of the AR device and a location of the AR device with the environmental data, wherein the AR device is further configured to send the environmental data, the recorded location, and the recorded orientation to the computing device, wherein the processor of the computing device is configured to save with the environmental data with the recorded location and the recorded orientation in the 3D indoor positioning map.

4. The system according to claim 2, wherein the computing device includes a remote program for communicating with a remote device with at least one remote sensor for sharing remote data.

5. The system according to claim 4, wherein the computing device may determine the location of the remote device on the 3D indoor positioning map by comparing the remote data shared with the environmental data, where the location of the remote device on the 3D indoor positioning map includes a probability of accuracy based on a number of degrees of freedom of the remote device.

6. The system of claim 1, wherein the AR device is configured to send the missing data and/or the data older than the refresh period collected by the one or more depth sensors, while the AR device is guided through the building, to the computing device and constantly updated in real time.

7. The system of claim 1, wherein the guidance AR content includes visible instructions to rotate around 360 degrees at each location so that an omnidirectional mapping of the environment is recorded at each location, the visible instructions includes an AR visual graphic that shows how to turn and how long to stay at each location configured for an accurate reading to be recorded.

8. The system of claim 1, wherein the processor of the computing device is further configured to:

create a two dimensional (2D) view of the indoor positioning map; and

show at least one location of a remote device on the 2D view of the indoor positioning map in real time;

wherein the computing device includes a remote program for communicating with the remote device the 2D view of the indoor positioning map with its location.

9. The system according to claim 1, wherein:

the one or more depth sensors include an optical 3D scanner; and

the display is a head mounted display (HMD) and comprises a transparent plate that is (i) positioned in front of the user'"'"'s eye or eyes, allowing the user to view the environment through the transparent plate and (ii) arranged to display AR content to the user'"'"'s eye or eyes so that the AR content appears superimposed on the user'"'"'s view of the environment.

10. A method for creating and updating indoor positioning maps, the method comprising:

gathering mapping data of physical objects in an environment from at least one depth sensor of an AR device;

dynamically creating a three-dimensional (3D) indoor positioning map based on the gathered mapping data from the AR device;

creating guidance AR content in response to the mapping data of the physical objects in the environment in the 3D indoor positioning map being older than a refresh period, wherein the guidance AR content is created corresponding to the AR device'"'"'s perspective view of the environment for guidance;

transmitting the 3D indoor positioning map with the guidance AR content to a display in the AR device for the guidance to gather updated mapping data; and

dynamically updating the 3D indoor positioning map based on the updated mapping data gathered at least when the mapping data of the physical objects in the environment in the 3D indoor positioning map is older than the refresh period.

11. The method according to claim 10, wherein creating the 3D indoor positioning map and/or dynamically updating the 3D indoor positioning map further includes using simultaneous location and mapping (SLAM) techniques for positioning, including collecting environmental data at various points within a building, where the environmental data includes all available RF signals, magnetic fields, lighting conditions, GPS, three dimensional imagery, ambient sound, and/or combinations thereof, wherein:

the RF signal information contains a signal strength, transmit power, and/or an identifier for an RF source; and

the light contains an intensity, color and/or frequency of light.

12. The method according to claim 11, wherein collecting environmental data at various points within a building includes recording an orientation and a location of the AR device with the environmental data, and saving the recorded location and the recorded orientation of the AR device with the environmental data in the 3D indoor positioning map.

13. An augmented reality (AR) device, comprising:

a display for displaying guidance AR content to a user, the guidance AR content overlapping the AR device'"'"'s perspective view of an environment; and

one or more depth sensors for gathering mapping data of physical objects in the environment; and

a processor communicatively coupled to the one or more depth sensors, the processor configured by software to;

dynamically create a three-dimensional (3D) indoor positioning map of the environment in a building based on the mapping data gathered by the one or more depth sensors; and

create guidance AR content in response to the mapping data of the physical objects in the environment in the 3D indoor positioning map being older than a refresh period, wherein the guidance AR content is created corresponding to the AR device'"'"'s perspective view of the environment for guidance;

transmit the 3D indoor positioning map with the guidance AR content to the display for the guidance to gather updated mapping data; and

dynamically update the 3D indoor positioning map of the environment in the building based on the updated mapping data gathered by the AR device when at least the mapping data of physical objects in the environment in the 3D indoor positioning map is older than the refresh period.

14. The AR device according to claim 13, wherein the creation of the 3D indoor positioning map and/or the dynamically updating of the 3D indoor positioning map includes utilizing simultaneous location and mapping (SLAM) techniques for positioning, wherein the SLAM techniques utilized for positioning include collecting environmental data at various points within a building, where the environmental data includes all available RF signals, magnetic fields, lighting conditions, GPS, three dimensional imagery, ambient sound, and/or combinations thereof, wherein:

the RF signal information contains a signal strength, a transmit power, and/or an identifier for an RF source; and

the light contains an intensity, a color and/or a frequency of light.

15. The AR device according to claim 14, wherein the processor is configured to record an orientation of the AR device and a location of the AR device with the environmental data wherein the processor is further configured to save the recorded location and the recorded orientation of the AR device with the environmental data in the 3D indoor positioning map.

16. The AR device according to claim 13, wherein:

the one or more depth sensors include an optical 3D scanner; and

the display is a head mounted display (HMD) and comprises a transparent plate that is (i) positioned in front of the user'"'"'s eye or eyes, allowing the user to view the environment through the transparent plate and (ii) arranged to display AR content to the user'"'"'s eye or eyes so that the AR content appears superimposed on the user'"'"'s view of the environment.

17. The AR device according to claim 16, wherein the display comprises a liquid crystal display (LCD).