Panoptic visualization of elements of a complex system using localization of a point on a physical instance of the complex system

• US 9,665,557 B2
• Filed: 01/28/2013
• Issued: 05/30/2017
• Est. Priority Date: 01/28/2013
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Floyd, Joseph F, Hadley, Brent L, Eames, Patrick J, Miller, Stephen P
• Primary Examiner(s): Padmanabhan, Kavita
• Assistant Examiner(s): Garmon, Brian
• Application Number:
  US13/751,994
• Publication Number:
  US 20140215318A1
• Time in Patent Office: 1,583 Days
• Field of Search: 715243
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals

Abstract

A system is provided that includes a localization module configured to receive and process range measurements and odometry data relative to a physical instance of a complex system to calculate an unknown location within a coordinate system of the complex system. A search engine configured to identify a document component that depicts an element of the complex system at or within a volume about the location. The document component is from a collection for which at least some components depict elements of the complex system and have associated metadata that includes information reflecting the 3D geometry of at least some depicted elements. A layout engine configured to generate a layout of panoptically-arranged document components including the identified document component and other document component(s) identified according to the associated metadata for the identified document component, which further includes information identifying a link between the respective document components.

Claims

1. A method comprising:

receiving and processing range measurements and odometry data relative to a physical instance of a complex system to calculate an unknown, particular location within a coordinate system of the complex system;

identifying a document component of a panoptic visualization document collection having a plurality of document components, each document component of at least some of the document components including respective media content depicting an element of the complex system, and having associated metadata providing structured information reflecting a three- dimensional (3D) geometry of the element within a coordinate system of the complex system, the respective media content of the identified document component depicting an element of the complex system at the particular location or within a volume about or at least in part defined by the particular location; and

generating a layout of panoptically-arranged visual representations of document components including the identified document component and one or more other document components identified according to the associated metadata for the identified document component, which further includes structured information identifying a link between the identified document component and one or more other document components that establishes a logical relationship between the respective media content thereof,

wherein the layout is a two-dimensional layout in which the logical relationship is expressed by a difference in size, location or depth of at least some of the visual representations relative to others of the visual representations, and wherein the visual representations of the document components in the layout are images of the document components including the respective media content thereof.

2. The method of claim 1, wherein first and second points with known locations within the coordinate system of the complex system are given, and a third point is at the unknown, particular location within the coordinate system of the complex system, and

wherein calculating the unknown, particular location includes calculating respective locations of the first, second and third points within another, second coordinate system, and registering the respective location of the third point within the coordinate system of the complex system.

3. The method of claim 1, wherein the link establishes a logical relationship between the respective media content of the identified document component and one or more other document components according to a spatial, design or functional relationship between one or more elements of the complex system including the element depicted by the identified document component.

4. The method of claim 1, wherein for each document component of the identified document component and the one or more other document components, the associated metadata further includes structured information specifying one or more of a size, location or depth of the document component in the layout, and the layout is generated according thereto.

5. The method of claim 1, wherein the respective media content of at least some of the document components of the layout include two-dimensional (2D) images depicting one or more elements of the complex system, and are produced from a digital 3D model of the complex system.

6. The method of claim 5, wherein the respective media content of one or more document components of the layout depict one or more elements of the complex system, and are not produced from the 3D model.

7. The method of claim 1, wherein the respective media content of at least one of the document components of the layout depicts an element of the complex system and has associated metadata that includes structured information reflecting the 3D geometry of the respective depicted element within the coordinate system of the complex system, and

wherein the respective media content of at least one other of the document components of the layout depicts an element of the complex system and has associated metadata absent information reflecting the 3D geometry of the respective depicted element within the coordinate system of the complex system.

9. The apparatus of claim 1, wherein the localization module is given first and second points with known locations within the coordinate system of the complex system, and a third point is at the unknown, particular location within the coordinate system of the complex system, and

wherein the localization module being configured to calculate the unknown, particular location includes being configured to calculate respective locations of the first, second and third points within another, second coordinate system, and register the respective location of the third point within the coordinate system of the complex system.

10. The apparatus of claim 1, wherein the link establishes a logical relationship between the respective media content of the identified document component and one or more other document components according to a spatial, design or functional relationship between one or more elements of the complex system including the element depicted by the identified document component.

11. The apparatus of claim 1, wherein for each document component of the identified document component and the one or more other document components, the associated metadata further includes structured information specifying one or more of a size, location or depth of the document component in the layout, and the layout engine is configured to generate the layout according thereto.

12. The apparatus of claim 1, wherein the respective media content of at least some of the document components of the layout include two-dimensional (2D) images depicting one or more elements of the complex system, and are produced from a digital 3D model of the complex system.

13. The apparatus of claim 12, wherein the respective media content of one or more document components of the layout depict one or more elements of the complex system, and are not produced from the 3D model.

14. The apparatus of claim 1, wherein the respective media content of at least one of the document components of the layout depicts an element of the complex system and has associated metadata that includes structured information reflecting the 3D geometry of the respective depicted element within the coordinate system of the complex system, and

wherein the respective media content of at least one other of the document components of the layout depicts an element of the complex system and has associated metadata absent information reflecting the 3D geometry of the respective depicted element within the coordinate system of the complex system.

8. An apparatus for implementation of a system for panoptic visualization of one or more elements of a complex system, the apparatus comprising a processor and a memory storing executable instructions that in response to execution by the processor, cause the apparatus to implement at least:

a localization module configured to receive and process range measurements and odometry data relative to a physical instance of a complex system to calculate an unknown, particular location within a coordinate system of the complex system;

a search engine coupled to the localization module and configured to identify a document component of a panoptic visualization document collection having a plurality of document components, each document component of at least some of the document components including respective media content depicting an element of the complex system, and having associated metadata providing structured information reflecting a three-dimensional (3D) geometry of the element within a coordinate system of the complex system, the respective media content of the identified document component depicting an element of the complex system at the particular location or within a volume about or at least in part defined by the particular location; and

a layout engine coupled to the search engine and configured to generate a layout of panoptically-arranged visual representations of document components including the identified document component and one or more other document components identified according to the associated metadata for the identified document component, which further includes structured information identifying a link between the identified document component and one or more other document components that establishes a logical relationship between the respective media content thereof,

wherein the layout is a two-dimensional layout in which the logical relationship is expressed by a difference in size, location or depth of at least some of the visual representations relative to others of the visual representations, and wherein the visual representations of the document components in the layout are images of the document components including the respective media content thereof.

15. A computer-readable storage medium that is non-transitory and has computer-readable program code portions stored therein that, in response to execution by a processor, cause an apparatus to at least:

receive process range measurements and odometry data relative to a physical instance of a complex system to calculate an unknown, particular location within a coordinate system of the complex system;

identify a document component of a panoptic visualization document collection having a plurality of document components, each document component of at least some of the document components including respective media content depicting an element of the complex system, and having associated metadata providing structured information reflecting a three-dimensional (3D) geometry of the element within a coordinate system of the complex system, the respective media content of the identified document component depicting an element of the complex system at the particular location or within a volume about or at least in part defined by the particular location; and

generate a layout of panoptically-arranged visual representations of document components including the identified document component and one or more other document components identified according to the associated metadata for the identified document component, which further includes structured information identifying a link between the identified document component and or more other document components that establishes a logical relationship between the respective media content thereof,

wherein the layout is a two-dimensional layout in which the logical relationship is expressed by a difference in size, location or depth of at least some of the visual representations relative to others of the visual representations, and wherein the visual representations of the document components in the layout are images of the document components including the respective media content thereof.

16. The computer-readable storage medium of claim 15, wherein first and second points with known locations within the coordinate system of the complex system are given, and a third point is at the unknown, particular location within the coordinate system of the complex system, and

wherein the apparatus being caused to calculate the unknown, particular location includes being caused to calculate respective locations of the first, second and third points within another, second coordinate system, and register the respective location of the third point within the coordinate system of the complex system.

17. The computer-readable storage medium of claim 15, wherein the link establishes a logical relationship between the respective media content of the identified document component and one or more other document components according to a spatial, design or functional relationship between one or more elements of the complex system including the element depicted by the identified document component.

18. The computer-readable storage medium of claim 15, wherein for each document component of the identified document component and the one or more other document components, the associated metadata further includes structured information specifying one or more of a size, location or depth of the document component in the layout, and the layout is generated according thereto.

19. The computer-readable storage medium of claim 15, wherein the respective media content of at least some of the document components of the layout include two-dimensional (2D) images depicting one or more elements of the complex system, and are produced from a digital 3D model of the complex system.

20. The computer-readable storage medium of claim 19, wherein the respective media content of one or more document components of the layout depict one or more elements of the complex system, and are not produced from the 3D model.

21. The computer-readable storage medium of claim 15, wherein the respective media content of at least one of the document components of the layout depicts an element of the complex system and has associated metadata that includes structured information reflecting the 3D geometry of the respective depicted element within the coordinate system of the complex system, and

wherein the respective media content of at least one other of the document components of the layout depicts an element of the complex system and has associated metadata absent information reflecting the 3D geometry of the respective depicted element within the coordinate system of the complex system.