Method and system for computer assisted localization and navigation in industrial environments

• US 20020010694A1
• Filed: 12/20/2000
• Published: 01/24/2002
• Est. Priority Date: 12/23/1999
• Status: Abandoned Application

• Current Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Genc, Yakup, Navab, Nassir
• Application Number:
  US09/741,581
• Publication Number:
  US 20020010694A1
• Time in Patent Office: Days
• US Class Current: 707/1
• CPC Class Codes: G06F 16/29 Geographical information databases

Abstract

A method for computer assisted localization and navigation in a defined environment, comprises the steps of: obtaining a floor map of the defined environment; placing a set of identifiable markers on the floor in given locations; determining the position of a user with respect to ones of the markers; determining the global position of the user by utilizing the floor map and the given locations; and indexing a database for data associated with the global position.

Claims

1. A method for computer assisted localization and navigation in a defined environment, comprising the steps of:

obtaining a floor map of said defined environment;

placing a set of identifiable markers on said floor in given locations;

determining the position of a user with respect to ones of said markers;

determining the global position of said user by utilizing said floor map and said given locations; and

indexing a database for data associated with said global position.

2. A method for computer assisted localization and navigation in a defined environment as recited in claim 1, wherein said step of obtaining a floor map comprises forming a data base.

3. A method for computer assisted localization and navigation in a defined environment as recited in claim 1, wherein said step of placing a set of identifiable markers comprises recording actual positions of said markers on said floor map.

4. A method for computer assisted localization and navigation in a defined environment as recited in claim 1, wherein said step of indexing a database for data comprises accessing data relating to objects associated with said global position.

5. A method for computer assisted localization and navigation in a defined environment as recited in claim 1, wherein said step of indexing a database for data comprises accessing data relating to objects in a perceptual environment of said global position.

6. A method for computer assisted localization and navigation in a defined environment as recited in claim 1, wherein said step of determining the position of a user with respect to ones of said markers comprises a step of identifying a marker comprising any of the following types of markers:

visual markers such as photogrammetry markers, three-dimensional objects or barcodes, infrared beacons, magnetic markers, sonar, radar, and markers using microwave techniques detectable by a suitable detection method.

7. A method for computer assisted localization and navigation in an industrial environment, comprising:

constructing a database of augmented images, include at least one of still and animated images;

placing a set of markers on a floor of said environment, each marker being unique so as to enable identification of a position of a user in said environment;

registering positions of said markers in a corresponding floor map;

automatically localizing said user by using said markers floor and a corresponding floor map;

directing said user to a particular location in said environment; and

accessing said database for information.

8. A method for computer assisted localization and navigation as recited in claim 7, wherein said step of accessing said database for information comprises a step for obtaining from said database graphical items such as drawings or animations explaining how to perform a maintenance/inspection task.

9. A method for computer assisted localization and navigation as recited in claim 7, wherein said step of obtaining from said database graphical comprises obtaining retrieved views close to the pose of said user.

10. A method for computer assisted localization and navigation as recited in claim 8, wherein said step of obtaining from said database graphical comprises rendering of a retrieved image or animation on a monitor.

11. A method for computer assisted localization and navigation as recited in claim 10, wherein said step of obtaining from said database graphical items comprises rendering intermediate views if said retrieved views are not close enough to the pose of said user.

12. A method for computer assisted localization and navigation as recited in claim 8, wherein said step of automatically localizing said user by using said markers and a corresponding floor map includes a step of pointing a camera at said markers.

13. A method for computer assisted localization and navigation as recited in claim 7, including a step of computing the position of said camera, given a set of detected markers.

14. A method for computer assisted localization and navigation as recited in claim 7, wherein said step of accessing said database for information comprises a step for obtaining from said database a list of items around said user and properties of said items.

15. A method for computer assisted localization and navigation as recited in claim 7, wherein said step of accessing said database for information comprises a step for obtaining from said database a list of items around said user and properties of said items.

16. A method for computer assisted localization and navigation as recited in claim 7, wherein said step of placing a set of markers comprises constructing set of more than 500 unique markers.

17. A method for computer assisted localization and navigation as recited in claim 7, wherein said step of placing a set of markers comprises constructing set of more than 500 unique markers.

18. A method for computer assisted localization and navigation as recited in claim 7, wherein said step of automatically localizing said user by using said markers comprises a step of detecting and tracking said markers in real time.

19. A method for computer assisted localization and navigation as recited in claim 7, including a step computing the shortest path between selected points in a delimited area on a plane which is highlighted on said floor map by the user.

20. A method for computer assisted localization and navigation as recited in claim 19, wherein said step of computing the shortest path comprises steps for guiding said user with the help of said computer through an environment that is unknown to said user.

21. A method for computer assisted localization and navigation of a user in an industrial-type environment, comprising:

obtaining a floor map of a site;

placing a set of unique visual markers on said floor;

storing location information for set of unique visual markers relative to said floor map;

deriving an image of said floor by a user-carried camera;

processing said image by a user-carried computer for detecting a visual marker on said floor; and

calculating position and orientation of said user-carried camera with respect to said marker.

22. A method for computer assisted localization and navigation of a user in accordance with claim 21, wherein said step of calculating position and orientation of said user-carried camera comprises:

determining a local position of said user through said step of detecting said markers; and

determining a global location of said user by utilizing said stored location information.

23. A method for computer assisted localization and navigation of a user in accordance with claim 21, wherein said user-carried camera and said user-carried computer are arranged in an integral unit.

24. A method for computer assisted localization and navigation of a user in accordance with claim 21, including a step of planning a desired path for said user on said floor map.

25. A method for computer assisted localization and navigation of a user in accordance with claim 24, including a step of guiding a said user in accordance with said desired path by said computer utilizing said calculated position and orientation.

26. A method for computer assisted localization and navigation of a user in accordance with claim 25, including steps of:

indexing a database of positional information; and

displaying said information to said user.

27. A method for computer assisted localization and navigation of a user in accordance with claim 25, including steps of:

indexing a database of position-specific information; and

displaying said position-specific information to said user.

28. A method for computer assisted localization and navigation of a user in accordance with claim 27, wherein said step of displaying said position-specific information to said user comprises a step of displaying images of objects in the field of view of said user.

29. A method for computer assisted localization and navigation of a user in an industrial environment, comprising:

obtaining a floor data map of a site;

establishing a set of unique markers on said floor data map;

detecting said markers; and

determining a location of said user through said step of detecting said markers.

30. A method for computer assisted localization and navigation as recited in claim 29, wherein said markers comprise any of visual markers including photogrammetry markers;

three-dimensional objects;

three-dimensional barcodes;

optical, visible, infrared or ultraviolet beacons;

fluorescent markers;

magnetic markers;

sonar systems;

radar systems;

microwave beacons;

microwave reflectors;

physically detectable markers.

31. A method for computer assisted localization and navigation of a user in an industrial-type environment, comprising:

obtaining a floor map of a site;

placing a set of unique markers on said floor;

storing location information for set of unique markers relative to said floor map;

detecting ones of said markers by a user-carried sensor; and

calculating position and orientation of said user-carried sensor by a user-carried computer with respect to said ones of said markers.

32. A method for computer assisted localization and navigation of a user in accordance with claim 31, wherein said step of calculating position and orientation of said user-carried sensor comprises:

determining a local position of said user through said step of detecting said ones of said markers; and

determining a global location of said user by utilizing said stored location information.

33. A method for computer assisted localization and navigation of a user in accordance with claim 31, wherein said user-carried sensor and said user-carried computer are arranged in an integral unit.

34. A method for computer assisted localization and navigation of a user in accordance with claim 32, including a step of planning a desired path for said user on said floor map.

35. A method for computer assisted localization and navigation of a user in accordance with claim 34, including a step of guiding a said user in accordance with said desired path by said computer utilizing said calculated position and orientation.

36. A method for computer assisted localization and navigation of a user in accordance with claim 31, including steps of:

indexing a database of positional information; and

displaying said information to said user.

37. A method for computer assisted localization and navigation of a user in accordance with claim 36, including a step of:

displaying position-specific information to said user.

38. A method for computer assisted localization and navigation of a user in accordance with claim 37, wherein said step of displaying said position-specific information to said user comprises a step of displaying images of objects in the field of view of said user.

39. A method for computer assisted localization and navigation of a user in accordance with claim 37, wherein said step of displaying said position-specific information to said user comprises a step of displaying information on task performance relating to items at said position.

40. A method for computer assisted localization and navigation of a user in accordance with claim 31, wherein said step of calculating position and orientation of said user-carried sensor by a user-carried computer with respect to said ones of said markers includes a step of combining observations of sensors utilizing different physical characteristics for detection.

41. A method for computer assisted localization and navigation in an industrial environment, comprising:

an “

off-line”

step of constructing a database of augmented images, include at least one of still and animated images;

an off-line step of placing a set of markers on a floor of said environment, each marker being unique so as to enable identification of a position of a user in said environment;

an off-line step of registering positions of said markers in a corresponding floor map;

an “

on-line”

step of automatically localizing said user by using said markers floor and a corresponding floor map;

an “

on-line”

step of directing said user to a particular location in said environment; and

an on-line step of accessing said database for information.

42. Apparatus for computer assisted localization and navigation of a user in an industrial environment, comprising:

apparatus for obtaining a floor data map of a site;

a set of unique markers on said floor data map;

a detector system for detecting said markers; and

computer apparatus for determining a location of said user through said step of detecting said markers.

43. Apparatus for computer assisted localization and navigation of a user as recited in claim 42, wherein said floor data map comprises at least one of an industrial drawing and a computer-aided-design (CAD) database.

44. Apparatus for computer assisted localization and navigation of a user as recited in claim 43, wherein said floor data map comprises at least one of an industrial drawing in electronic form and a computer-aided-design (CAD) database.

45. Apparatus for computer assisted localization and navigation as recited in claim 42, wherein said markers comprise any of visual markers including photogrammetry markers;

three-dimensional objects;

three-dimensional barcodes;

optical, visible, infrared or ultraviolet beacons;

fluorescent markers;

magnetic markers;

sonar systems;

radar systems;

microwave beacons;

microwave reflectors; and

transponders.

46. Apparatus for computer assisted localization and navigation as recited in claim 42, including inertial sensor apparatus for enhancing accuracy of orientation data derived from said markers.