Integrated visualization and analysis of a complex system

• US 9,841,870 B2
• Filed: 08/21/2014
• Issued: 12/12/2017
• Est. Priority Date: 08/21/2014
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Hadley, Brent L., Eames, Patrick J., Floyd, Joseph F.
• Primary Examiner(s): PHANTANA ANGKOOL, DAVID
• Application Number:
  US14/464,904
• Publication Number:
  US 20160054889A1
• Time in Patent Office: 1,209 Days
• Field of Search: 715859
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06F 30/15 Vehicle, aircraft or watercraft design
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

An apparatus is provided for implementation of a system for integrated visualization and analysis of a complex system such as an aircraft composed of a plurality of elements. The apparatus may receive data for a physical instance of the complex system, and this data may include a list of the plurality of elements that reflects a relationship between at least some of them. The apparatus may generate a visual presentation of a digital three-dimensional (3D) model of the complex system. The apparatus may generate a layout of document components depicting elements of the plurality of elements, with the layout including one or more two-dimensional (2D) images depicting at least some of the elements. And the apparatus may integrate the data, visual presentation of the 3D model and layout to produce a visual definition of the complex system.

Claims

1. An apparatus for implementation of a system for integrated visualization and analysis of a complex system, the apparatus comprising a processor and a memory storing executable instructions that in response to execution by the processor cause the apparatus to at least:

receive data for a physical instance of a complex system composed of a plurality of elements, the data including a list of the plurality of elements that reflects a relationship between at least some of the plurality of elements;

generate a visual presentation of a digital three-dimensional (3D) model of the complex system, the visual presentation depicting elements of the plurality of elements;

generate a layout of document components depicting elements of the plurality of elements, the layout including a panoptic arrangement of images of the document components at least some of which include two-dimensional (2D) images depicting at least some of the elements, the panoptic arrangement being two-dimensional and the relationship between at least some of the plurality of elements being expressed in the panoptic arrangement by a difference in size or depth of at least some of the 2D images relative to others of the 2D images; and

integrate the data, visual presentation of the 3D model and layout of document components to produce a visual definition of the complex system, wherein at least the visual presentation and layout in the visual definition are navigable through interaction with the data.

2. The apparatus of claim 1, wherein the visual presentation of the 3D model depicts one or more elements absent from the document components of the layout, or the document components of the layout depict one or more elements absent from the visual presentation of the 3D model.

3. The apparatus of claim 1, wherein the memory stores further executable instructions that in response to execution by the processor cause the apparatus to further navigate the visual presentation of the 3D model or layout of document components, including the apparatus being caused to at least:

receive selection of an element from the list of the plurality of elements; and

manipulate the visual presentation of the 3D model or layout of document components in the visual definition to draw attention to the element therein, including being caused to at least one of focus on a portion of the visual presentation including the element or apply a visual effect to the element, or focus on or apply a visual effect to one or more document components of the layout that depict the element.

4. The apparatus of claim 1, wherein the memory stores further executable instructions that in response to execution by the processor cause the apparatus to further receive additional data for elements of the plurality of elements,

wherein the apparatus being caused to integrate the data, visual presentation of the 3D model and layout of document components further includes the additional data, the additional data in the visual definition being navigable through interaction with the data, visual presentation or layout.

5. The apparatus of claim 4, wherein the memory stores further executable instructions that in response to execution by the processor cause the apparatus to further navigate the additional data, including the apparatus being caused to at least:

receive selection of an element of the plurality of elements, the element being selectable from each of the list of the plurality of elements, visual presentation of the 3D model and layout of document components; and

manipulate the additional data in the visual definition to draw attention to additional data for the element.

6. The apparatus of claim 1, wherein the memory stores further executable instructions that in response to execution by the processor cause the apparatus to further at least:

receive and process additional data for the physical instance of the complex system, the additional data being received and processed to identify a topical element from the plurality of elements, the elements depicted by the visual presentation of the 3D model including the topical element and one or more other elements of the plurality of elements; and

apply a visual effect to the topical element depicted by the visual presentation to distinguish the topical element from the one or more other elements depicted by the visual presentation.

7. The apparatus of claim 1, wherein the memory stores further executable instructions that in response to execution by the processor cause the apparatus to further at least:

receive user-selection of a particular point on the visual presentation of the 3D model in the visual definition of the complex system; and

translate the particular point to a particular location within a coordinate system of the complex system, and

wherein the apparatus being caused to generate the layout of document components includes being caused to at least;

identify a first document component that depicts an element of the complex system at the particular location or within a volume about or at least in part defined by the particular location, the first document component having and being identified from associated metadata that provides structured information reflecting a 3D geometry of the element within the coordinate system of the complex system; and

generate the layout of document components including the first document component, and including one or more second document components identified from the associated metadata that provides structured information further identifying a link between the first document component and one or more second document components.

22. The apparatus of claim 6, wherein the additional data is also processed to calculate a level-of-interest value for the topical element, the topical element being one for which a number of issues has been identified, and the level-of-interest value being proportionate to the number of issues identified for the topical element, and

wherein the apparatus being caused to apply the visual effect includes being caused to apply a color to the topical element, the color having a property that varies with the level-of-interest value that is proportionate to the number of issues identified for the topical element, and the one or more other elements being depicted by the visual presentation without any color applied thereto.

8. A method comprising:

receiving data for a physical instance of a complex system composed of a plurality of elements, the data including a list of the plurality of elements that reflects a relationship between at least some of the plurality of elements;

generating a visual presentation of a digital three-dimensional (3D) model of the complex system, the visual presentation depicting elements of the plurality of elements;

generating a layout of document components depicting elements of the plurality of elements, the layout including a panoptic arrangement of images of the document components at least some of which include two-dimensional (2D) images depicting at least some of the elements, the panoptic arrangement being two-dimensional and the relationship between at least some of the plurality of elements being expressed in the panoptic arrangement by a difference in size or depth of at least some of the 2D images relative to others of the 2D images; and

integrating the data, visual presentation of the 3D model and layout of document components to produce a visual definition of the complex system, wherein at least the visual presentation and layout in the visual definition are navigable through interaction with the data.

9. The method of claim 8, wherein the visual presentation of the 3D model depicts one or more elements absent from the document components of the layout, or the document components of the layout depict one or more elements absent from the visual presentation of the 3D model.

10. The method of claim 8 further comprising navigating the visual presentation of the 3D model or layout of document components, including:

receiving selection of an element from the list of the plurality of elements; and

manipulating the visual presentation of the 3D model or layout of document components in the visual definition to draw attention to the element therein, including at least one of focusing on a portion of the visual presentation including the element or applying a visual effect to the element, or focusing on or applying a visual effect to one or more document components of the layout that depict the element.

11. The method of claim 8 further comprising receiving additional data for elements of the plurality of elements,

wherein integrating the data, visual presentation of the 3D model and layout of document components further includes the additional data, the additional data in the visual definition being navigable through interaction with the data, visual presentation or layout.

12. The method of claim 11 further comprising navigating the additional data, including:

receiving selection of an element of the plurality of elements, the element being selectable from each of the list of the plurality of elements, visual presentation of the 3D model and layout of document components; and

manipulating the additional data in the visual definition to draw attention to additional data for the element.

13. The method of claim 8 further comprising:

receiving and processing additional data for the physical instance of the complex system, the additional data being received and processed to identify a topical element from the plurality of elements, the elements depicted by the visual presentation of the 3D model including the topical element and one or more other elements of the plurality of elements; and

applying a visual effect to the topical element depicted by the visual presentation to distinguish the topical element from the one or more other elements depicted by the visual presentation.

14. The method of claim 8 further comprising:

receiving user-selection of a particular point on the visual presentation of the 3D model in the visual definition of the complex system; and

translating the particular point to a particular location within a coordinate system of the complex system, and

wherein generating the layout of document components comprises;

identifying a first document component that depicts an element of the complex system at the particular location or within a volume about or at least in part defined by the particular location, the first document component having and being identified from associated metadata that provides structured information reflecting a 3D geometry of the element within the coordinate system of the complex system; and

generating the layout of document components including the first document component, and including one or more second document components identified from the associated metadata that provides structured information further identifying a link between the first document component and one or more second document components.

23. The method of claim 13, wherein the additional data is also processed to calculate a level-of-interest value for the topical element, the topical element being one for which a number of issues has been identified, and the level-of-interest value being proportionate to the number of issues identified for the topical element, and

wherein applying the visual elect includes applying a color to the topical element, the color having a property that varies with the level-of-interest value that is proportionate to the number of issues identified for the topical element, and the one or more other elements being depicted by the visual presentation without any color applied thereto.

15. A computer-readable storage medium that is non-transitory and has computer-readable program code portions stored therein that, in response to execution by a processor, cause an apparatus to at least:

receive data for a physical instance of a complex system composed of a plurality of elements, the data including a list of the plurality of elements that reflects a relationship between at least some of the plurality of elements;

generate a visual presentation of a digital three-dimensional (3D) model of the complex system, the visual presentation depicting elements of the plurality of elements;

generate a layout of document components depicting elements of the plurality of elements, the layout including a panoptic arrangement of images of the document components at least some of which include two-dimensional (2D) images depicting at least some of the elements, the panoptic arrangement being two-dimensional and the relationship between at least some of the plurality of elements being expressed in the panoptic arrangement by a difference in size or depth of at least some of the 2D images relative to others of the 2D images; and

integrate the data, visual presentation of the 3D model and layout of document components to produce a visual definition of the complex system, wherein at least the visual presentation and layout in the visual definition are navigable through interaction with the data.

16. The computer-readable storage medium of claim 15, wherein the visual presentation of the 3D model depicts one or more elements absent from the document components of the layout, or the document components of the layout depict one or more elements absent from the visual presentation of the 3D model.

17. The computer-readable storage medium of claim 15 having further readable program code portions stored therein that, in response to execution by the processor, cause the apparatus to further navigate the visual presentation of the 3D model or layout of document components, including the apparatus being caused to at least:

receive selection of an element from the list of the plurality of elements; and

manipulate the visual presentation of the 3D model or layout of document components in the visual definition to draw attention to the element therein, including being caused to at least one of focus on a portion of the visual presentation including the element or apply a visual effect to the element, or focus on or apply a visual effect to one or more document components of the layout that depict the element.

18. The computer-readable storage medium of claim 15 having further readable program code portions stored therein that, in response to execution by the processor, cause the apparatus to further receive additional data for elements of the plurality of elements,

wherein the apparatus being caused to integrate the data, visual presentation of the 3D model and layout of document components further includes the additional data, the additional data in the visual definition being navigable through interaction with the data, visual presentation or layout.

19. The computer-readable storage medium of claim 18 having further readable program code portions stored therein that, in response to execution by the processor, cause the apparatus to further navigate the additional data, including the apparatus being caused to at least:

receive selection of an element of the plurality of elements, the element being selectable from each of the list of the plurality of elements, visual presentation of the 3D model and layout of document components; and

manipulate the additional data in the visual definition to draw attention to additional data for the element.

20. The computer-readable storage medium of claim 15 having further readable program code portions stored therein that, in response to execution by the processor, cause the apparatus to further at least:

receive and process additional data for the physical instance of the complex system, the additional data being received and processed to identify a topical element from the plurality of elements, the elements depicted by the visual presentation of the 3D model including the topical element and one or more other elements of the plurality of elements; and

apply a visual effect to the topical element depicted by the visual presentation to distinguish the topical element from the one or more other elements depicted by the visual presentation.

21. The computer-readable storage medium of claim 15 having further readable program code portions stored therein that, in response to execution by the processor, cause the apparatus to further at least:

receive user-selection of a particular point on the visual presentation of the 3D model in the visual definition of the complex system; and

translate the particular point to a particular location within a coordinate system of the complex system, and

wherein the apparatus being caused to generate the layout of document components includes being caused to at least;

identify a first document component that depicts an element of the complex system at the particular location or within a volume about or at least in part defined by the particular location, the first document component having and being identified from associated metadata that provides structured information reflecting a 3D geometry of the element within the coordinate system of the complex system; and

generate the layout of document components including the first document component, and including one or more second document components identified from the associated metadata that provides structured information further identifying a link between the first document component and one or more second document components.

24. The computer-readable storage medium of claim 20, wherein the additional data is also processed to calculate a level-of-interest value for the topical element, the topical element being one for which a number of issues has been identified, and the level-of-interest value being proportionate to the number of issues identified for the topical element, and

wherein the apparatus being, caused to apply the visual effect includes being caused to apply a color to the topical element, the color having a property that varies with the level-of-interest value that is proportionate to the number of issues identified for the topical element, and the one or more other elements being depicted by the visual presentation without any color applied thereto.