Immersive Object Testing System

• US 20170076495A1
• Filed: 09/16/2015
• Published: 03/16/2017
• Est. Priority Date: 09/16/2015
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Gabrys, Jonathan Wayne, Bowen, David William, Montalbano, Anthony Mathew, Choi, Chong
• Granted Patent: US 9,852,237 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06Q 50/04 Manufacturing
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An object testing system and method for testing an object. A three-dimensional environment is displayed with a model of an object and an avatar from a viewpoint relative to the avatar on a display system viewed by a human operator. The object is under testing in a live environment. Information about motions of the human operator that are detected is generated. Live information about the object that is under testing in the live environment is received. A change in the object from applying the live information to the model of the object is identified. The change in the model of the object is displayed on the display system as seen from the viewpoint relative to the avatar in the three-dimensional environment.

Claims

1. An object testing system comprising:

a motion capture system that detects motions of a human operator and generates information about the motions; and

a model manager that creates an avatar representing the human operator;

places the avatar in a three-dimensional environment with a model of an object;

displays the three-dimensional environment with the model of the object and the avatar from a viewpoint relative to the avatar on a display system;

receives live information about the object that is under testing in a live environment;

identifies a change in the object from applying the live information to the model of the object; and

displays the change in the model of the object on the display system as seen from the viewpoint relative to the avatar in the three-dimensional environment.

2. The object testing system of claim 1 further comprising:

a sensor system that generates the live information about the object under testing in the live environment.

3. The object testing system of claim 2, wherein the sensor system is selected from at least one of a laser scanner, a strain gauge, an accelerometer, a force sensing resistor, a vibration sensor, a temperature sensor, an impact detector, a gyroscopic sensor, or an inertial measurement unit.

4. The object testing system of claim 1, wherein when the change is displacement, the model is a finite element method model, and in identifying the change in the object from applying the live information to the model of the object, the model manager performs a finite element analysis on the model using the live information about the object, and identifies stress in the object from the finite element analysis.

5. The object testing system of claim 1, wherein in displaying the change in the model of the object as seen from the viewpoint relative to the avatar, the model manager displays a group of colors on the model of the object as seen from the viewpoint relative to the avatar in which the group of colors indicates amounts of a group of parameters of the object, wherein the group of parameters is selected from at least one of stress, strain, displacement, acoustics, computational fluid dynamics, or temperature of the object.

6. The object testing system of claim 1, wherein the live information about the object includes at least one of modulation data, temperature, acceleration, velocity, translation, vibration data, force, or acoustic data.

7. The object testing system of claim 1, wherein the human operator is in a first location and the object that is under testing is in a second location.

8. The object testing system of claim 1, wherein the model is selected from one of a computer-aided design model, a finite element method model, or a computer-aided model.

9. The object testing system of claim 1, wherein the display system is selected from at least one of a display device, a computer monitor, glasses, a head-mounted display device, a tablet computer, a mobile phone, a projector, a heads up display, a holographic display system, or a virtual retinal display.

10. The object testing system of claim 1, wherein the object is selected from one of a mobile platform, a stationary platform, a land-based structure, an aquatic-based structure, a space-based structure, an aircraft, a surface ship, a tank, a personnel carrier, a train, a spacecraft, a space station, a satellite, a submarine, an automobile, a power plant, a bridge, a dam, a house, a manufacturing facility, a building, a wing, a beam, an engine housing, a seat, and a stabilizer.

11. A method for testing an object, the method comprising:

displaying a three-dimensional environment with a model of the object and an avatar from a viewpoint relative to the avatar on a display system viewed by a human operator, wherein the object is under testing in a live environment;

generating information about motions of the human operator that are detected;

receiving live information about the object that is under testing in the live environment;

identifying a change in the object from applying the live information to the model of the object; and

displaying the change in the model of the object on the display system as seen from the viewpoint relative to the avatar in the three-dimensional environment.

12. The method of claim 11 further comprising:

generating the live information about the object under testing in the live environment with a sensor system.

13. The method of claim 12, wherein the sensor system is selected from at least one of a laser scanner, a strain gauge, an accelerometer, a force sensing resistor, a vibration sensor, a temperature sensor, an impact detector, a gyroscopic sensor, or an inertial measurement unit.

14. The method of claim 11, wherein when the change is a displacement, the model is a finite element method model, wherein the identifying step comprises:

performs a finite element analysis on the model using the live information about the object; and

identifying stress in the object from the finite element analysis.

15. The method of claim 11, wherein displaying the change in the model of the object as seen from the viewpoint relative to the avatar comprises:

displaying a group of colors on the model of the object as seen from the viewpoint relative to the avatar in which the group of colors indicates amounts of a parameter of the object, wherein the parameter is selected from one of stress, strain, displacement, acoustics, computational fluid dynamics, and temperature of the object.

16. The method of claim 11 further comprising:

changing a test process used in testing the object based on the change identified in the model of the object as seen from the viewpoint relative to the avatar.

17. The method of claim 16, wherein the change in the test process is made during a time selected from at least one of during a test of the object or after the test of the object.

18. The method of claim 11, wherein the live information about the object includes at least one of modulation data, temperature, acceleration, velocity, translation, vibration data, force, or acoustic data.

19. The method of claim 11, wherein the human operator is in a first location and the object that is under testing is in a second location.

20. The method of claim 11, wherein the model is selected from one of a computer-aided design model, a finite element method model, and a computer-aided model.

21. The method of claim 11, wherein the display system is selected from at least one of a display device, a computer monitor, glasses, a head-mounted display device, a tablet computer, a mobile phone, a projector, a heads up display, a holographic display system, or a virtual retinal display.

22. The method of claim 11, wherein the object is selected from one of a mobile platform, a stationary platform, a land-based structure, an aquatic-based structure, a space-based structure, an aircraft, a surface ship, a tank, a personnel carrier, a train, a spacecraft, a space station, a satellite, a submarine, an automobile, a power plant, a bridge, a dam, a house, a manufacturing facility, a building, a wing, a beam, an engine housing, a seat, and a stabilizer.