System and method for determining and maintaining object location and status

• US 9,996,551 B2
• Filed: 03/14/2014
• Issued: 06/12/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Huntington Ingalls, Inc. (Huntington Ingalls Industries Incorporated)
• Original Assignee: Huntington Ingalls, Inc. (Huntington Ingalls Industries Incorporated)
• Inventors: Bare, Brian, Martin, Jonathan, Sells, Paul, Ryan, Patrick
• Primary Examiner(s): Raab, Christopher J
• Application Number:
  US14/210,650
• Publication Number:
  US 20140280199A1
• Time in Patent Office: 1,551 Days
• Field of Search: None
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/909 using geographical or spatial information, e.g. location spatiotemporally dependent retrieval from the web G06F16/9537
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04W 12/06 Authentication
• CPC Class Codes: H04W 12/65 Environment-dependent, e.g. using captured environmental data
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings
• CPC Class Codes: H04W 64/003 locating network equipment

Abstract

An asset information management system is provided for obtaining and storing information on assets disposed within a structural environment. The system comprises at least one mobile interface device for capturing information about a target object in the structural environment. An information management operating system operates on a central processor in communication with the mobile interface device and a local positioning system. The domain coordinate management system comprises one or more computers configured for establishing spatial addresses for spaces and points within the structural environment. The asset log comprises one or more data storage modules configured for storing information associated with objects disposed within the structural environment. The spatial indexing engine is configured for assigning a spatial address to the target object and for storing information associated with the target object in the asset log.

Claims

1. An asset information management system for obtaining and storing information on assets disposed within a physical structural environment, the system comprising:

at least one mobile interface device configured for variable disposition within a physical dynamic structure in the physical structural environment, for capturing information about a target object disposed within the physical dynamic structure, and for transmitting the target object information over a network;

a local positioning system in communication with the at least one mobile interface device via the network and configured to provide information usable to determine a mobile device pose, said mobile device pose comprising a position and orientation of the mobile interface device relative to the physical structural environment;

a central processor in communication with the at least one mobile interface device and the local positioning system via the network and having an information management operating system operating thereon, the information management operating system being configured for receiving and processing the target object information;

a domain coordinate management system comprising one or more computers configured for establishing and maintaining spatial addresses for spaces and points within the physical structural environment, for receiving first spatial address information from the local positioning system and second spatial address information from a three-dimensional model of the physical dynamic structure, and for determining differences between the first and second spatial address information resulting from spatial changes in the physical dynamic structure;

an asset log comprising one or more data storage modules configured for storing information associated with objects disposed within the physical structural environment; and

a spatial indexing engine in communication with the domain coordinate management system, the asset log and the central processor, the spatial indexing engine being configured for assigning a spatial address to the target object and for storing information associated with the target object in the asset log.

2. An asset information management system according to claim 1, wherein the target object information captured by the mobile interface device includes information usable with the mobile device pose to determine the target object'"'"'s position and orientation relative to the physical structural environment.

3. An asset information management system according to claim 1, wherein the mobile interface device is configured for capturing an image of the target object, the image being included in the target object information.

4. An asset information management system according to claim 1, wherein the mobile interface device is one of the set consisting of a tablet computer, a smartphone, and a wearable heads-up display.

5. An asset information management system according to claim 1, wherein the mobile interface device is configured for capturing an image of the target object and for displaying the captured image with a superimposed computer-generated image associated with at least one of the set consisting of the target object and a portion of the physical structural environment in which the target object is disposed.

6. An asset information management system according to claim 1, wherein the target object information includes at least one of the set consisting of an image of the target object, information on target object geometry, information on target object location, and information on target object orientation.

7. An asset information management system according to claim 1, wherein the information associated with the target object stored in the asset log by the spatial indexing engine includes a position and an orientation of the target object relative to the physical dynamic structure.

8. An asset information management system according to claim 1, wherein the position and orientation of the target object relative to the physical dynamic structure are determined from the mobile device pose, an image captured by the mobile device, and spatial differences resulting from changes in the physical dynamic structure.

9. An asset information management system according to claim 1, wherein the spatial indexing engine is further configured for

receiving target object information from the central processor,

comparing the target object information to known object information stored in the asset log,

and assigning a spatial address to the target object only if the object information does not match any known object information in the asset log.

10. An asset information management system according to claim 1, wherein the local positioning system is configured to determine the mobile device pose using pre-identified visual tracking cues within the physical structural environment.

11. A method of storing object information associated with a target object disposed within a physical structural environment, the method comprising:

positioning a mobile interface device within a portion of the physical structural environment where the target object is located, the mobile interface device having a mobile device position and orientation relative to the physical structural environment;

determining a mobile device pose comprising the mobile device position and orientation relative to the physical structural environment;

capturing object information with the mobile interface device;

receiving at a central data processor the mobile device pose and the object information from the mobile interface device;

determining a location and orientation of the object relative to the physical structural environment, accounting for spatial changes in the physical structural environment;

adding the location and orientation of the object to the object information;

comparing the object information to information stored in an asset log database;

determining whether the object is recognized based on results of the comparing action;

responsive to determining that the object is recognized, updating information on the object in the asset log database; and

responsive to determining the object is not recognized,

assigning a spatial address to the object, and

storing the object information as a new entry in the asset log database.

12. A method according to claim 11 wherein the action of capturing object information includes:

capturing an image of the object.

13. A method according to claim 12 wherein the action of capturing object information further includes:

applying object recognition techniques to the object image to provide an object recognition result; and

adding the object recognition result to the object information.

14. A method according to claim 11 wherein the action of determining a location and orientation of the object relative to the physical structural environment-includes:

determining first spatial address information for the object using the object information and information from a local positioning system;

determining second spatial address information for the object using a three-dimensional model of the physical dynamic structure; and

determining differences between the first and second spatial address information resulting from changes in the physical structural environment.

15. A method according to claim 11 wherein the location and orientation of the object are determined at least in part from the mobile device pose and an image captured by the mobile interface device.

16. A method according to claim 11 wherein the object information includes at least one of the set consisting of an image of the object, a geometry of the object, and information from a tag attached to the object.

17. A method according to claim 11 further comprising:

transmitting the object information by the mobile interface device over a wireless communication network.

18. A method according to claim 11 wherein the action of updating information on the object in the asset log database includes noting changes in at least one of the set consisting of object location, object orientation, and object status.