Method and apparatus for using thermal imaging and augmented reality

• US 7,262,747 B2
• Filed: 05/12/2006
• Issued: 08/28/2007
• Est. Priority Date: 08/09/2001
• Status: Expired due to Fees

• Current Assignee: Information Decision Technologies LLC
• Original Assignee: Information Decision Technologies LLC
• Inventors: Ebersole, John Franklin, Ebersole, John Franklin Jr., Hobgood, Andrew Wesley
• Primary Examiner(s): Shankar; Vijay
• Application Number:
  US11/382,978
• Publication Number:
  US 20060232499A1
• Time in Patent Office: 473 Days
• Field of Search: 345 4- 9, 359/13, 359/465, 359630-635, 349 11- 15, 349/59
• US Class Current: 345/7
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26

Abstract

A method for using Augmented Reality (AR), in conjunction with a real or simulated thermal imager. A primary application is to train emergency first responders. Particularly, the system uses a thermal imaging camera or standard video camera, and tracking system to provide a tracked viewpoint in an AR environment. The augmented portions of the environment can consist of fire, smoke, extinguishing agent, or other emergencies. This allows for inexpensive, flexible, and realistic on-site training of fire fighting, damage control, search and rescue, and other first responder techniques using thermal imaging.

Claims

1. A method for using thermal imaging and augmented reality, comprising

providing a thermal imaging device to acquire an image of the user'"'"'s environment;

providing a tracking system mechanically coupled to the device;

using the tracking system to collect the position and orientation of the device;

providing a computer;

using the device to capture a thermal view of the environment to be augmented;

sending the view to a video capture device connected to the computer;

using the computer to compute and render a simulated thermal image of objects and features to be augmented onto the thermal view of the environment;

combining the simulated thermal image of the objects and features with the thermal image of the real world to display a single, augmented view to the user that represents an augmented thermal view of the real world;

displaying the single, final, augmented thermal view to the user.

2. The method of claim 1 where the device used to capture the image of the real world is a portable thermal imaging camera.

3. The method of claim 2 where the portable imaging camera is an ISG Talisman K-90XL.

4. The method of claim 2 where the portable thermal imaging camera is an ISG K-1000.

5. The method of claim 2 where the thermal imaging camera is any model of portable thermal imaging camera from any other vendor.

6. The method of claim 1 where the device used to capture the image of the real world is a thermal imaging device attached to a firefighter'"'"'s SCBA.

7. The method of claim 1 where the tracking system is ultrasonic/inertial.

8. The method of claim 1 where the tracking system is optical.

9. The method of claim 1 where the tracking system is comprised of GPS-based trackers.

10. The method of claim 1 where the tracking system is comprised of markerless trackers.

11. The method of claim 1 where the tracking system is comprised of motorized camera mount trackers.

12. The method of claim 1 where the single, augmented view is used for training.

13. The method of claim 1 where the single, augmented view comprises thermal signatures of at least one of victims, hot spots, fire, smoke, steam, gases, extinguishing agents, other first responders and personnel, signals of impending structural failure, and explosion hazards.

14. The method of claim 13 where the single, augmented view is used for training firefighters and other first responders for almost any thermally detectable hazard scenario.

15. The method of claim 1 where the objects and features are selected from the group of thermal signatures consisting of thermal signatures of equipment, structures, vehicles, personnel, and indications of activity, victims, hot spots, fire, smoke, steam, gases, extinguishing agents, other first responders and personnel, signals of impending structural failure, and explosion hazards.

16. The method of claim 15 where the single, augmented view is used for training military personnel.

17. The method of claim 15 where the single, augmented view is used for training law enforcement personnel.

18. The method of claim 15 where the single, augmented view is used for training surveillance personnel.

19. The method of claim 15 where the purpose is training for the evaluation of thermal stealth designs in the field.

20. The method of claim 15 where the purpose is the operational evaluation of thermal stealth designs in the field.

21. The method of claim 1 where the tracking system is a subcomponent of a motorized camera base such that the motorized camera has knowledge of the orientation of where the camera is pointing at any given time, and it can provide the orientation information of the camera to the computer.

22. The method of claim 1 where the thermal view that is sent from the imaging device to the video capture device is sent via a wireless transmitter and received by a wireless receiver.

23. The method of claim 1 where the tracking system attached to the device is a wireless device, or minimally, does not require any cables to be run from the tracking system to any fixed location.

24. The method of claim 1 where at least the computer is carried by the user.