Network error detection using virtual reality display devices

• US 10,620,981 B2
• Filed: 03/09/2018
• Issued: 04/14/2020
• Est. Priority Date: 03/09/2018
• Status: Active Grant

• Current Assignee: Bank of America Corp.
• Original Assignee: Bank of America Corp.
• Inventors: Thomas, James M., Shields, Alan W.
• Primary Examiner(s): Richer, Aaron M
• Assistant Examiner(s): Vu, Khoa
• Application Number:
  US15/916,706
• Publication Number:
  US 20190278621A1
• Time in Patent Office: 767 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/0823 Errors, e.g. transmission errors

Abstract

The device that includes a normalization engine configured to populate data fields in a normalized data structure with network information in accordance with normalization rules. The device further includes a virtualization engine configured to generate virtual data defining one or more virtual objects in accordance with virtualization rules that map data fields from the normalized data structure to physical attributes of virtual objects and to transmit the virtual data defining the one or more virtual objects for display on a user device. The virtualization engine is further configured to receive user feedback that identifies a selected virtual object, to identify data field values in the normalized data structure for the physical attributes of the selected virtual object, and to generate an error report comprising at least a portion of the identified data fields values. The virtualization engine is further configured to send the error report to the user device.

Claims

1. A network error detection device, comprising:

a network interface configured to receive network information from a plurality of data sources;

a memory configured to store;

a normalized data structure comprising a plurality of data fields;

normalization rules configured to map received network information to data fields in the normalized data structure; and

virtualization rules configured to map data fields from the normalized data structure to physical attributes of virtual objects within a virtual reality environment, wherein;

the virtual reality environment is a geographical map comprising a landmark; and

the virtualization rules map;

internal IP addresses to a virtual object type and a starting location on the landmark within the virtual reality environment;

external IP addresses to a target location on the landmark within the virtual reality environment; and

hardware settings to virtual objects configured to obstruct the movement of other virtual objects within the virtual reality environment;

a normalization engine implemented by one or more processors, configured to populate data fields in the normalized data structure with received network information in accordance with the normalization rules;

a virtualization engine implemented by the one or more processors, configured to generate virtual data defining a virtual object for the virtual reality environment, wherein generating the virtual object comprises;

selecting an entry linked with a plurality of data fields in the normalized data structure;

identifying an internal IP address associated with the entry;

determining a virtual object type based on the identified internal IP address, wherein the virtual object type identifies a physical representation of a physical object;

determining a starting location on the landmark within the virtual reality environment based on the identified internal IP address;

identifying an external IP address associated with the entry;

determining a target location on the landmark within the virtual reality environment based on the identified external IP address; and

configuring the virtual object to traverse the landmark within the virtual reality environment from the determined starting location to the target location;

the network interface further configured to;

transmit the virtual data defining the virtual object for display on a user device in response to generating the virtual object;

receive user feedback from the user device, wherein the user feedback identifies a selected virtual object;

the virtualization engine further configured to;

identify physical attributes of the selected virtual object;

identify data fields and corresponding data field values in the normalized data structure for the identified physical attributes of the selected virtual object based on the virtualization rules;

generate an error report comprising at least a portion of the identified data fields values; and

the network interface further configured to send the error report to the user device.

2. The device of claim 1, further comprising the user device, wherein the user device comprises:

a display configured to present virtual objects in the virtual reality environment;

a virtual overlay engine implemented by the one or more processors, configured to;

receive the virtual data defining the virtual object;

render the virtual object in the virtual reality environment on the display;

a camera configured to capture gestures performed by a user; and

a gesture recognition engine configured to;

detect a gesture performed by the user interacting with the virtual object in the virtual reality environment using the camera; and

send the user feedback to the virtualization engine identifying the virtual object as the selected virtual object.

3. The device of claim 2, wherein detecting the gesture performed by the user interacting with the virtual object in the virtual environment comprises detecting the user repositioning the virtual object within the virtual reality environment.

4. The device of claim 1, further comprising the user device, wherein the user device comprises:

a display configured to present virtual objects in the virtual reality environment;

a virtual overlay engine implemented by the one or more processors, configured to;

receive the virtual data defining the virtual object;

render the virtual object in the virtual reality environment on the display;

a microphone configured to capture audio signals; and

a voice recognition engine configured to;

detect an audio signal identifying the virtual object in the virtual reality environment using the microphone; and

send the user feedback to the virtualization engine identifying the virtual object as the selected virtual object.

5. The device of claim 1, wherein receiving network information from the plurality of data sources comprises:

receiving a first portion of network information from a first data source in a first format; and

receiving a second portion of the network information from a second data source in a second format, wherein the second format is different than the first format.

6. The device of claim 1, wherein the user device is configured to display at least a portion of the error report in the virtual reality environment.

7. A network error detection method, comprising:

receiving, by a network interface, network information from a plurality of data sources;

populating, by a normalization engine, data fields in a normalized data structure with received network information in accordance with normalization rules, wherein;

the normalized data structure comprising a plurality of data fields; and

the normalization rules are configured to map received network information to data fields in the normalized data structure;

generating, by a virtualization engine, virtual data defining one or more virtual objects for a virtual reality environment using virtualization rules, wherein;

the virtual reality environment is a geographical map comprising a landmark;

the virtualization rules are configured to map data fields from the normalized data structure to physical attributes of virtual objects, wherein the virtualization rules map;

internal IP addresses to a virtual object type and a starting location on the landmark within the virtual reality environment;

external IP addresses to a target location on the landmark within the virtual reality environment; and

hardware settings to virtual objects configured to obstruct the movement of other virtual objects within the virtual reality environment; and

generating the virtual object comprises;

selecting an entry linked with a plurality of data fields in the normalized data structure;

identifying an internal IP address associated with the entry;

determining a virtual object type based on the identified internal IP address, wherein the virtual object type identifies a physical representation of a physical object;

determining a starting location on the landmark within the virtual reality environment based on the identified internal IP address;

identifying an external IP address associated with the entry;

determining a target location on the landmark within the virtual reality environment based on the identified external IP address; and

configuring the virtual object to traverse the landmark within the virtual reality environment from the determined starting location to the target location;

transmitting, by the network interface, the virtual data defining the virtual object for display on a user device in response to generating the virtual object;

receiving, by the network interface, user feedback from the user device, wherein the user feedback identifies a selected virtual object;

identifying, by the virtualization engine, physical attributes of the selected virtual object;

identifying, by the virtualization engine, data fields and corresponding data field values in the normalized data structure for the identified physical attributes of the selected virtual object based on the virtualization rules;

generating, by the virtualization engine, an error report comprising at least a portion of the identified data fields values; and

sending, by the network interface, the error report to the user device.

8. The method of claim 7, further comprising:

receiving, by a virtual overlay engine, the virtual data defining the virtual object;

rendering, by the virtual overlay engine, the virtual object in the virtual reality environment on a display;

detecting, by a gesture recognition engine, a gesture performed by the user interacting with the virtual object in the virtual reality environment; and

sending, by the virtual overlay engine, the user feedback to the virtualization engine identifying the virtual object as the selected virtual object.

9. The method of claim 8, wherein detecting the gesture performed by the user interacting with the virtual object in the virtual environment comprises detecting the user repositioning the virtual object within the virtual reality environment.

10. The method of claim 7, further comprising:

receiving, by a virtual overlay engine, the virtual data defining the virtual object;

rendering, by the virtual overlay engine, the virtual object in the virtual reality environment on the display;

detecting, by a voice recognition engine, an audio signal identifying the virtual object in the virtual reality environment; and

sending, by the virtual overlay engine, the user feedback to the virtualization engine identifying the virtual object as the selected virtual object.

11. The method of claim 7, wherein receiving network information from the plurality of data sources comprises:

receiving a first portion of network information from a first data source in a first format; and

receiving a second portion of the network information from a second data source in a second format, wherein the second format is different than the first format.

12. The method of claim 7, further comprising displaying, by the virtual overlay engine, at least a portion of the error report in the virtual reality environment.

13. A network error detection device, comprising:

a network interface configured to receive network information from a plurality of data sources;

a memory configured to store;

a normalized data structure comprising a plurality of data fields;

normalization rules configured to map received network information to data fields in the normalized data structure; and

virtualization rules configured to map data fields from the normalized data structure to physical attributes of virtual objects within virtual reality environment, wherein;

the virtual reality environment is a geographical map comprising a landmark;

the virtualization rules map;

internal IP addresses to a virtual object type and a starting location on the landmark within the virtual reality environment;

external IP addresses to a target location on the landmark within the virtual reality environment; and

hardware settings to virtual objects configured to obstruct the movement of other virtual objects within the virtual reality environment;

a normalization engine implemented by one or more processors, configured to populate data fields in the normalized data structure with received network information in accordance with the normalization rules;

a virtualization engine implemented by the one or more processors, configured to;

generate virtual data defining a virtual object for the virtual reality environment, wherein generating the virtual object comprises;

selecting an entry linked with a plurality of data fields in the normalized data structure;

identifying an internal IP address associated with the entry;

determining a virtual object type based on the identified internal IP address, wherein the virtual object type identifies a physical representation of a physical object;

determining a starting location on the landmark within the virtual reality environment based on the identified internal IP address;

identifying an external IP address associated with the entry;

determining a target location on the landmark within the virtual reality environment based on the identified external IP address; and

configuring the virtual object to traverse the landmark within the virtual reality environment from the determined starting location to the target location;

a virtual overlay engine implemented by the one or more processors, configured to;

receive the virtual data defining the virtual object;

render the virtual object in the virtual reality environment on a display; and

receive user feedback, wherein the user feedback identifies a selected virtual object from among one or more virtual objects;

the virtualization engine further configured to;

identify physical attributes of the selected virtual object;

identify data fields and corresponding data field values in the normalized data structure for the identified physical attributes of the selected virtual object based on the virtualization rules;

generate an error report comprising at least a portion of the identified data fields values; and

the virtual overlay engine further configured to display the error report.

14. The device of claim 13, further comprising:

a camera configured to capture gestures performed by a user; and

a gesture recognition engine configured to;

detect a gesture performed by the user interacting with the virtual object in the virtual reality environment using the camera; and

send the user feedback to the virtualization engine identifying the virtual object as the selected virtual object.

15. The device of claim 14, wherein detecting the gesture performed by the user interacting with the virtual object in the virtual environment comprises detecting the user repositioning the virtual object within the virtual reality environment.