SYSTEMS AND METHODS FOR MAPPING SENSOR FEEDBACK ONTO VIRTUAL REPRESENTATIONS OF DETECTION SURFACES

• US 20160217578A1
• Filed: 04/16/2014
• Published: 07/28/2016
• Est. Priority Date: 04/16/2013
• Status: Abandoned Application

• Current Assignee: Red Sky Technologies Incorporated
• Original Assignee: Red Sky Technologies Incorporated
• Inventors: Can, Matthew, Jayatilaka, Lahiru
• Application Number:
  US14/254,470
• Publication Number:
  US 20160217578A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01C 19/5698 using acoustic waves, e.g. surface acoustic wave gyros
• CPC Class Codes: G01C 3/00 Measuring distances in line of sight; Optical rangefinders tapes, chains or wheels for measuring length G01B3/00; active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves, G01S17/48
• CPC Class Codes: G01V 3/12 operating with electromagnetic waves operating with millimetre waves G01V8/005
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/10044 Radar image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G08C 23/00 Non-electrical signal transmission systems, e.g. optical systems
• CPC Class Codes: G08C 23/02 using infrasonic, sonic or ultrasonic waves
• CPC Class Codes: G08C 23/04 using light waves, e.g. infrared
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver

Abstract

Systems and methods for mapping sensor feedback onto virtual representations of detection surfaces are disclosed herein. A system configured in accordance with an embodiment of the present technology can, for example, record and process feedback from a sensing device (e.g., a metal detector), record and process user inputs from a user input device (e.g. user-determined locations of disturbances in the soil surface), determine the 3D position, orientation, and motion of the sensing device with respect to a detection surface (e.g., a region of land being surveyed for landmines) and visually integrate captured and computed information to support decision-making (e.g. overlay a feedback intensity map on an image of the ground surface). In various embodiments, the system can also determine the 3D position, orientation, and motion of the sensing device with respect to the earth'"'"'s absolute coordinate frame, and/or record and process information about the detection surface.

Claims

1. A method in a computing system of mapping onto a virtual representation of a detection surface feedback from an above-surface mobile detector of objects below the detection surface, the method comprising:

receiving data characterizing a position and motion of the mobile detector from one or more of inertial sensors, a GPS receiver, ultrasound transducers, and optical sensors associated with the mobile detector;

determining, by the computing system and based on the received data, a pose of the mobile detector;

receiving information characterizing the detection surface from one or more imaging sensors associated with the mobile detector;

generating, by the computing system, a virtual representation of the detection surface based on the determined pose of the mobile detector and the received information characterizing the detection surface;

capturing feedback from the mobile detector regarding detection of an object below the detection surface at a certain time;

identifying a detected object location based on the captured feedback from the mobile detector and the determined pose of the mobile detector at the certain time; and

displaying a visualization of the identified detected object location integrated into the virtual representation of the detection surface.

2. The method of claim 1 wherein the mobile detector is a landmine or IED detector having a detector head, and wherein determining a pose of the mobile detector includes tracking the position and motion of the detector head.

3. The method of claim 1 wherein determining a pose of the mobile detector includes determining an orientation and heading of the mobile detector.

4. The method of claim 1 wherein determining a pose of the mobile detector includes determining a position of the mobile detector based on communication with external reference point satellites or ultrasound beacons.

5. The method of claim 1 wherein receiving information characterizing the detection surface from one or more imaging sensors includes receiving information from an infrared camera or a visible light camera.

6. The method of claim 1 wherein generating a virtual representation of the detection surface includes compiling recorded images to generate a two-dimensional or three-dimensional photographic or topological representation of the detection surface.

7. The method of claim 1 wherein displaying a visualization of the identified detected object location integrated into the virtual representation of the detection surface includes displaying a heat map, a contour map, a topographical map, or a two-dimensional or three-dimensional representation including photographic or infrared images.

8. The method of claim 1 wherein displaying a visualization of the identified detected object location includes displaying detector feedback using points, shapes, lines, or an icon to indicate a detected object or an edge or contour of a detected object.

9. The method of claim 1, further comprising:

identifying a detected object type, material, size, or configuration based on the captured feedback from the mobile detector; and

displaying a visualization of the identified detected object type, material, size, or configuration integrated into the virtual representation of the detection surface.

10. The method of claim 1, further comprising:

capturing user-defined temporal or spatial points of interest; and

displaying the captured user-defined temporal or spatial points of interest integrated into the virtual representation of the detection surface.

11. A system for mapping feedback from a mobile subsurface object detector onto a virtual representation of a detection surface, the system comprising:

one or more pose sensors, including—

one or more inertial sensors configured to sense the position, orientation, heading, or motion of the mobile subsurface object detector; and

an external reference point locator;

an optical sensor configured to have a field of view of the detection surface;

an input device configured to receive feedback from the mobile subsurface object detector;

a processor configured to visually integrate the feedback from the mobile subsurface object detector onto a virtual representation of the detection surface; and

a display device configured to display the virtual representation of the detection surface including the visually integrated feedback.

12. The system of claim 11 wherein the mobile subsurface object detector includes a metal detector or a ground-penetrating radar.

13. The system of claim 11:

wherein the one or more inertial sensors include at least one gyroscope, at least one accelerometer, and at least one magnetometer;

wherein the external reference point locator includes a GPS receiver, an ultrasound transducer, a laser rangefinder, or an infrared camera; and

wherein the optical sensor includes a camera or an infrared sensor.

14. The system of claim 11 wherein the input device is a microphone configured to detect acoustic feedback from the mobile subsurface object detector or recognize voice commands from a user.

15. The system of claim 11 wherein the input device includes a push button configured to allow a user of the mobile subsurface object detector to denote spatial or temporal points of interest.

16. The system of claim 11, further comprising a remote computing device configured to display the virtual representation of the detection surface including the visually integrated feedback to a remote user.

17. The system of claim 11, further comprising an unmanned aerial or ground vehicle configured to move the detector above the detection surface.

18. A system component for mapping sensor feedback from a detector of subsurface structure onto a virtual representation of a detection surface, the system component comprising:

a detector pose component configured to record a pose of the detector;

a detection surface component configured to record information about the detection surface;

a user input component configured to record user input from a user input device;

an object detection component configured to record detector feedback;

a processing component configured to create a virtual representation of the detection surface based on the recorded pose of the detector and information about the detection surface;

an object mapping component configured to map locations based on the recorded user input and detector feedback; and

a display component configured to visually display the mapped locations integrated into the virtual representation of the detection surface.

19. The system component of claim 18, further comprising an ultrasound or radio transceiver configured to determine a position, orientation, heading, or motion of the detector in relation to one or more external reference points.

20. The system component of claim 18 wherein the processing component is a computing device remote from the detector and operatively coupled via a wired or wireless data connection to at least one component associated with the detector.

21. The system component of claim 18 wherein the object detection component is configured to capture electrical, optical, or acoustic signals from the detector.

22. The system component of claim 18, further comprising a communication component configured to transmit the mapped locations or the virtual representation of the detection surface to a remote computing system.

23. The system component of claim 18 wherein:

the processing component is configured to create a virtual representation of the detection surface based on recorded poses of multiple detectors and information about the detection surface from multiple detectors; and

the object mapping component is configured to map locations based on recorded user input and detector feedback from multiple detectors.