Head-mounted sensor system

• US 10,579,138 B2
• Filed: 12/22/2016
• Issued: 03/03/2020
• Est. Priority Date: 12/22/2016
• Status: Active Grant

• Current Assignee: REscan Inc.
• Original Assignee: REscan Inc.
• Inventors: Bognar, Botond, Herman, Robert
• Primary Examiner(s): Sherman, Stephen G
• Application Number:
  US15/388,935
• Publication Number:
  US 20180181195A1
• Time in Patent Office: 1,167 Days
• Field of Search: 345 8
• CPC Class Codes: A42B 1/242 Means for mounting detecting, signalling or lighting devices
• CPC Class Codes: G01S 15/08 Systems for measuring distance only indirect measurement G01S15/46
• CPC Class Codes: G01S 15/86 Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

Systems and method of mapping spaces include a head-mounted sensor array. The array includes sensors configured to both measure distances within spaces and capture images of the spaces. This data may be used to generate continuously navigable virtual simulations of the spaces. Mapping of the spaces may include movable objects such as doors, transparent objects such as windows, and transitions between floors.

Claims

1. A head-mounted sensor system comprising:

a helmet;

a first LIDAR sensor mounted on the helmet, the first LIDAR sensor being configured to detect distances between the first LIDAR sensor and surfaces near the helmet;

a multi-camera system mounted on the helmet;

a synchronization circuit configured to synchronize outputs of at least the first LIDAR sensor and the multi-camera system;

a movement sensor mounted on the helmet and configured to detect movement the helmet; and

an electronic output configured to provide output of the first LIDAR sensor, output of the multi-camera system, and output of the movement sensor, in real-time to a data storage device;

wherein the multi-camera system is configured to generate angle dependent textures for registration to transparent surfaces.

2. The system of claim 1, wherein the multi-camera system comprises stereoscopic optics.

3. The system of claim 1, further comprising registration logic configured to generate a wireframe representation of the surfaces based on the output of at least the first LIDAR sensor.

4. The system of claim 3, wherein the registration logic is configured to map textures of the surfaces to points on the wireframe.

5. The system of claim 1, further comprising change logic configured to detect changes in an environment detected by the first LIDAR sensor, the changes including positions of surfaces.

6. The system of claim 5 wherein said surfaces include surfaces interior to a building.

7. The system of claim 1, further comprising change logic configured to detect changes in an environment, the changes including opening of a door.

8. The system of claim 1, further comprising mapping logic configured to generate a map of an environment based on outputs of at least the first LIDAR sensor, the map including indications of detected surfaces and indication of areas of the environment that have not been mapped.

9. The system of claim 1, further comprising a second LIDAR sensor.

10. The system of claim 1, wherein the multi-camera system includes a 360 degree camera system.

11. The system of claim 1, wherein the multi-camera system includes at least two cameras.

12. The system of claim 1, further comprising a second LIDAR sensor.

13. The system of claim 1, wherein the multi-camera system includes a 360 degree camera system.

14. The system of claim 1, wherein the multi-camera system includes at least two cameras.

15. A head-mounted sensor system comprising:

a helmet;

a first LIDAR sensor mounted on the helmet, the first LIDAR sensor being configured to detect distances between the first LIDAR sensor and surfaces near the helmet;

a multi-camera system mounted on the helmet;

a synchronization circuit configured to synchronize outputs of at least the first LIDAR sensor and the multi-camera system;

a movement sensor mounted on the helmet and configured to detect movement the helmet; and

an electronic output configured to provide output of the first LIDAR sensor, output of the multi-camera system, and output of the movement sensor, in real-time to a data storage device; and

difference logic configured to identify a surface as transparent by detecting the surface using an ultrasonic range finder but not detecting the surface using the LIDAR sensor.

16. The system of claim 15 wherein the multi-camera system comprises stereoscopic optics.

17. The system of claim 15 further comprising registration logic configured to generate a wireframe representation of the surfaces based on the output of at least the first LIDAR sensor.

18. The system of claim 17 wherein the registration logic is configured to map textures of the surfaces to points on the wireframe.

19. The system of claim 15 further comprising change logic configured to detect changes in an environment detected by the first LIDAR sensor, the changes including positions of surfaces.

20. The system of claim 19 wherein said surfaces include surfaces interior to a building.

21. The system of claim 15 further comprising change logic configured to detect changes in an environment, the changes including opening of a door.

22. The system of claim 15, further comprising mapping logic configured to generate a map of an environment based on outputs of at least the first LIDAR sensor, the map including indications of detected surfaces and indication of areas of the environment that have not been mapped.