Multi-component and mixed-reality simulation environments

• US 10,311,171 B2
• Filed: 03/02/2015
• Issued: 06/04/2019
• Est. Priority Date: 03/02/2015
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Inventors: Greenspan, Steven L., Danielson, Debra J., Reno, James D., Kane, John P., Parikh, Prashant
• Primary Examiner(s): Chad, Aniss
• Assistant Examiner(s): Wechselberger, Alfred H B
• Application Number:
  US14/636,134
• Publication Number:
  US 20160259870A1
• Time in Patent Office: 1,555 Days
• CPC Class Codes: G06F 30/20 Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]

Abstract

Systems and methods may include receiving real-time data about a real component operating in a real-world environment. The systems and methods may further include generating a virtual representation of the real component based on the real-time data about the real component and historical data associated with the real component. In addition, the systems and methods may include receiving injected data from a lab. The injected data may provide data about a lab component operating in the lab. The systems and methods may also include simulating, in a virtual environment, a real-time interaction in the real-world environment between the real component and the lab component using the virtual representation of the real component and the injected data. Moreover, the systems and methods may include determining a real-time performance characteristic of at least one of the lab component and the real component based on the simulated real-time interaction in the real-world environment.

Claims

1. A method comprising:

receiving real-time data about a real component operating in a real-world environment;

generating a virtual representation of the real component based on the real-time data about the real component and historical data associated with the real component;

receiving injected data from a lab, the injected data providing data about a lab component operating in the lab;

wherein the lab component comprises a physical device and the lab comprises a laboratory environment that subjects the physical device to controlled interactions with the physical device;

generating a virtual representation of the lab component using the injected data;

combining three domains in order to collectively monitor the real component, the lab component, the virtual representation of the real component and the virtual representation of the lab component;

the three domains including the real-world environment, the laboratory environment, and a virtual environment;

wherein combining the three domains includes simulating, in the virtual environment, a real-time interaction between the virtual representation of the real component and the virtual representation of the lab component;

continuously updating the real-time interaction between the virtual representation of the real component and the virtual representation of the lab component based upon real-time environmental data regarding the real-world environment and real-time environmental data regarding the lab; and

determining a real-time performance characteristic of at least one of the lab component and the real component based on the simulated real-time interaction in the real-world environment.

2. The method of claim 1, further comprising:

receiving real-time data about another real component operating in the real-world environment, the other real component being different from the real component,

using the injected data as a substitute for the real-time data about the other real component in the process of simulating the real-time interaction in the real-world environment between the real component and the lab component, and

wherein determining the real-time performance characteristic of the at least one of the lab component and the real component comprises determining the real-time performance characteristic of the lab component relative to a real-time performance characteristic of the other real component determined from the real-time data about the other real component.

3. The method of claim 2, further comprising:

determining whether the real-time performance characteristic of the lab component is improved relative to the real-time performance characteristic of the other real component; and

in response to determining that the real-time performance characteristic of the lab component is improved relative to the real-time performance characteristic of the other real component, transmitting a notification to an operator of the other real component in the real-world environment, the notification notifying the operator of an available upgrade incorporating features of the lab component.

4. The method of claim 1,

wherein the lab component is used as a substitute for another real component interacting with the real component in the real-world environment,

wherein the injected data simulates a malfunction of the other real world component,

wherein determining the real-time performance characteristic of the at least one of the lab component and the real component comprises determining the real-time performance characteristic of the real component, and

wherein the method further comprises modifying the virtual representation of the real component to establish the real-time performance characteristic of the real component determined when the injected data simulates the malfunction of the other real world component as a predicted response of the real component to the malfunction of the other real world component.

5. The method of claim 1, wherein the real component comprises a first component of a vehicle under evaluation and the lab component is a second component of the vehicle that is used with the real component.

6. The method of claim 1, further comprising:

determining a predicted performance characteristic of the at least one of the lab component and the real component based on the simulated real-time interaction in the real-world environment and expected future data about the real component operating in the real-world environment.

7. The method of claim 1, wherein the historical data associated with the one or more real components comprises:

information describing a history of interaction between a plurality of components similar to the real component and the real-world environment; and

information describing a history of interaction between the real component and the real-world environment.

8. A system comprising:

a processing system configured to;

receive real-time data about a real component operating in a real-world environment;

generate a virtual representation of the real component based on the real-time data about the real component and historical data associated with the real component;

receive injected data from a lab, the injected data providing data about a lab component operating in the lab;

wherein the lab component comprises a physical device and the lab comprises a laboratory environment that subjects the physical device to controlled interactions with the physical device;

generate a virtual representation of the lab component using the injected data;

combine three domains in order to collectively monitor the real component, the lab component, the virtual representation of the real component and the virtual representation of the lab component;

the three domains including the real-world environment, the laboratory environment, and a virtual environment;

wherein combine three domains includes simulate, in the virtual environment, a real-time interaction between the virtual representation of the real component and the virtual representation of the lab component;

continuously update the real-time interaction between the virtual representation of the real component and the virtual representation of the lab component based upon real-time environmental data regarding the real-world environment and real-time environmental data regarding the lab; and

determine a real-time performance characteristic of at least one of the lab component and the real component based on the simulated real-time interaction in the real-world environment.

9. The system according to claim 8,

wherein the processing system is further configured to receive real-time data about another real component operating in the real-world environment, the other real component being different from the real component,

wherein, when the processing system simulates the real-time interaction in the real-world environment between the real component and the lab component, the processing system is configured to use the injected data as a substitute for the real-time data about the other real component, and

wherein, when determining the real-time performance characteristic of the at least one of the lab component and the real component, the processing system is configured to determine the real-time performance characteristic of the lab component relative to a real-time performance characteristic of the other real component determined from the real-time data about the other real component.

10. The system according to claim 9,

wherein the processing system is further configured to:

determining whether the real-time performance characteristic of the lab component is improved relative to the real-time performance characteristic of the other real component; and

in response to determining that the real-time performance characteristic of the lab component is improved relative to the real-time performance characteristic of the other real component, transmit a notification to an operator of the other real component in the real-world environment, the notification notifying the operator of an available upgrade incorporating features of the lab component.

11. The system according to claim 8,

wherein, when simulating the real-time interaction in the real-world environment between the real component and the lab component, the processing system is configured to use the lab component as a substitute for another real component interacting with the real component in the real-world environment,

wherein the injected data simulates a malfunction of the other real world component,

wherein, when determining the real-time performance characteristic of the at least one of the lab component and the real component, the processing system is configured to determine the real-time performance characteristic of the real component, and

wherein the processing system is further configured to modify the virtual representation of the real component to establish the real-time performance characteristic of the real component determined when the injected data simulates the malfunction of the other real world component as a predicted response of the real component to the malfunction of the other real world component.

12. The system according to claim 8, wherein the real component comprises a first component of a vehicle under evaluation and the lab component is a second component of the vehicle that is used with the real component.

13. The system according to claim 8, wherein the processing system is further configured to determine a predicted performance characteristic of the at least one of the lab component and the real component based on the simulated real-time interaction in the real-world environment and expected future data about the real component operating in the real-world environment.

14. The system according to claim 8, wherein the historical data associated with the one or more real components comprises:

information describing a history of interaction between a plurality of components similar to the real component and the real-world environment; and

information describing a history of interaction between the real component and the real-world environment.

15. A computer program product comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to receive real-time data about a real component operating in a real-world environment;

computer readable program code configured to generate a virtual representation of the real component based on the real-time data about the real component and historical data associated with the real component;

computer readable program code configured to receive injected data from a lab, the injected data providing data about a lab component operating in the lab;

wherein the lab component comprises a physical device and the lab comprises a laboratory environment that subjects the physical device to controlled interactions with the physical device;

computer readable program code configured to generate a virtual representation of the lab component using the injected data;

computer readable program code configured to combine three domains in order to collectively monitor the real component, the lab component, the virtual representation of the real component and the virtual representation of the lab component;

the three domains including the real-world environment, the laboratory environment, and a virtual environment;

wherein computer readable program code configured to combine three domains includes computer readable program code configured to simulate, in a virtual environment, a real-time interaction the virtual representation of the real component and the virtual representation of the lab component;

computer readable program code configured to continuously update the real-time interaction between the virtual representation of the real component and the virtual representation of the lab component based upon real-time environmental data regarding the real-world environment and real-time environmental data regarding the lab; and

computer readable program code configured to determine a real-time performance characteristic of at least one of the lab component and the real component based on the simulated real-time interaction in the real-world environment.

16. The computer program product of claim 15, further comprising:

computer readable program code configured to receive real-time data about another real component operating in the real-world environment, the other real component being different from the real component,

computer readable program code configured to use the injected data as a substitute for the real-time data about the other real component, and

wherein the computer readable program code configured to determine the real-time performance characteristic of the at least one of the lab component and the real component comprises;

computer readable program code configured to determine the real-time performance characteristic of the lab component relative to a real-time performance characteristic of the other real component determined from the real-time data about the other real component.

17. The computer program product of claim 16, further comprising:

computer readable program code configured to determine whether the real-time performance characteristic of the lab component is improved relative to the real-time performance characteristic of the other real component; and

computer readable program code configured to, in response to determining that the real-time performance characteristic of the lab component is improved relative to the real-time performance characteristic of the other real component, transmit a notification to an operator of the other real component in the real-world environment, the notification notifying the operator of an available upgrade incorporating features of the lab component.

18. The computer program product of claim 15,

the computer readable program code configured to simulate the real-time interaction in the real-world environment between the real component and the lab component comprises:

computer readable program code configured to use the lab component as a substitute for another real component interacting with the real component in the real-world environment,

wherein the injected data simulates a malfunction of the other real world component,

wherein the computer readable program code configured to determine the real-time performance characteristic of the at least one of the lab component and the real component comprises;

computer readable program code configured to determine the real-time performance characteristic of the real component, and

wherein the computer program product further comprises;

computer readable program code configured to modify the virtual representation of the real component to establish the real-time performance characteristic of the real component determined when the injected data simulates the malfunction of the other real world component as a predicted response of the real component to the malfunction of the other real world component.

19. The computer program product of claim 15, further comprising:

computer readable program code configured to determine a predicted performance characteristic of the at least one of the lab component and the real component based on the simulated real-time interaction in the real-world environment and expected future data about the real component operating in the real-world environment.

20. The computer program product of claim 15, wherein the historical data associated with the one or more real components comprises:

information describing a history of interaction between a plurality of components similar to the real component and the real-world environment; and

information describing a history of interaction between the real component and the real-world environment.