Unmanned vehicle (UV) movement and data control system

• US 10,067,510 B2
• Filed: 02/11/2015
• Issued: 09/04/2018
• Est. Priority Date: 02/14/2014
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Mullan, Pramila, Mui, Michael, Fenney, Robert
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Kong, Sze-Hon
• Application Number:
  US14/619,749
• Publication Number:
  US 20180224854A1
• Time in Patent Office: 1,301 Days
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/104 involving a plurality of aircrafts, e.g. formation flying traffic control systems for aircraft G08G5/00
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/38 for collecting sensor information

Abstract

Unmanned vehicle (UV) movement and data control may include controlling a UV according to a movement plan. Formatted movement and status metadata may be received from a sensor of the UV during movement of the UV. The movement and status metadata may include time and location information for the UV during the movement of the UV. An unformatted data stream may be received from the sensor of the UV. The time and location information may be injected into metadata of the unformatted data stream to generate a time and location correlated (TLC) stream. The TLC stream may be analyzed to identify an event related to the UV, and a notification related to the event may be generated.

Claims

1. An unmanned vehicle (UV) movement and data control system comprising:

a mission controller, executed by at least one hardware processor, to control a UV according to a movement plan;

a telemetry data analyzer, executed by the at least one hardware processor, to

receive, from the mission controller, formatted movement and status metadata from at least one sensor of the UV during movement of the UV according to the movement plan, and

buffer the movement and status metadata for forwarding to a data integrator, wherein the movement and status metadata includes time and location information for the UV during the movement of the UV according to the movement plan;

a stream data analyzer, executed by the at least one hardware processor, to receive an unformatted data stream from the at least one sensor of the UV, wherein

the unformatted data stream includes a plurality of frames of data, and

the data integrator is executed by the at least one hardware processor to inject the time and location information into metadata of each of the plurality of frames of the data of the unformatted data stream to generate a time and location correlated (TLC) stream, wherein the data integrator comprises;

a time and location service analyzer, executed by the at least one hardware processor, to generate, based on the movement and status metadata, a model of a state of the UV during the movement of the UV according to the movement plan; and

a stream integrator, executed by the at least one hardware processor, to;

extract the plurality of the frames of the data from the unformatted data stream;

retrieve the time and location information from the model of the state of the UV; and

inject the time and location information into the metadata of each of the plurality of the frames of the data from the unformatted data stream to generate TLC frames, wherein a collection of the TLC frames represents the TLC stream;

a TLC stream data analyzer, executed by the at least one hardware processor, to analyze the TLC stream to identify an event related to the UV; and

an event orchestrator, executed by the at least one hardware processor, to generate a notification related to the event.

2. The UV movement and data control system according to claim 1, wherein the mission controller is to convert the movement and status metadata to a JavaScript Object Notation (JSON) format for processing by the data integrator.

3. The UV movement and data control system according to claim 1, wherein the at least one sensor includes a video camera, a gas detector, an infrared (IR) camera, and a pressure sensor.

4. The UV movement and data control system according to claim 1, wherein the data integrator comprises:

the stream integrator to;

extract, from the plurality of the frames of the data, a frame of the data from the unformatted data stream; and

pre-process the data from the unformatted data stream for detecting a leak in a pipeline by passing the frame of the data through hue-saturation-value (HSV) based clustering to segment environment into distinct color patches, wherein the leak in the pipeline is detected by using a transform to extract the pipeline spanning the frame.

5. The UV movement and data control system according to claim 4, wherein the TLC stream data analyzer is to utilize a histogram-based sliding window to identify pixels of interest in the frame of the data.

6. The UV movement and data control system according to claim 4, wherein the TLC stream data analyzer is to utilize Teh-Chin chain approximation to extract a blob that represents the leak in the pipeline.

7. The UV movement and data control system according to claim 6, wherein the TLC stream data analyzer is to utilize a naï

ve-Bayes classifier to classify the blob as a leak or not a leak.

8. The UV movement and data control system according to claim 7, wherein the TLC stream data analyzer is to retain the classification of the blob based on corroboration of the classification with the plurality of the frames of the data including the frame of the data.

9. The UV movement and data control system according to claim 1, wherein the TLC stream data analyzer further comprises:

a stream processing unit (SPU) including a plurality of event analyzers, executed by the at least one hardware processor, to analyze the TLC stream to identify the event related to the UV.

10. The UV movement and data control system according to claim 1, wherein the UV is an autonomous UV.

11. The UV movement and data control system according to claim 1, wherein the UV is an unmanned aerial vehicle (UAV).

12. The UV movement and data control system according to claim 1, wherein the TLC stream data analyzer is to analyze the TLC stream to identify the event that includes a potential leak or an intruder related to a pipeline.

13. A method for unmanned vehicle (UV) movement and data control, the method comprising:

receiving, at a telemetry data analyzer that is executed by at least one hardware processor, formatted movement and status metadata from at least one sensor of a UV during movement of the UV, wherein the movement and status metadata includes time and location information for the UV during the movement of the UV;

receiving, at a stream data analyzer that is executed by the at least one hardware processor, an unformatted data stream from the at least one sensor of the UV, wherein the unformatted data stream includes a plurality of frames of data;

injecting, by a data integrator that is executed by the at least one hardware processor, the time and location information into metadata of each of the plurality of frames of the data of the unformatted data stream to generate a time and location correlated (TLC) stream;

generating, by a time and location service analyzer that is executed by the at least one hardware processor, based on the movement and status metadata, a model of a state of the UV during the movement of the UV;

extracting, by a stream integrator that is executed by the at least one hardware processor, the plurality of the frames of the data from the unformatted data stream;

retrieving, by the stream integrator, the time and location information from the model of the state of the UV;

injecting, by the stream integrator, the time and location information into the metadata of each of the plurality of the frames of the data from the unformatted data stream to generate TLC frames, wherein a collection of the TLC frames represents the TLC stream; and

analyzing, by a TLC stream data analyzer that is executed by the at least one hardware processor, the TLC stream to identify an event related to the UV.

14. The method for UV movement and data control according to claim 13, further comprising:

extracting, by the stream integrator, from the plurality of the frames of the data, a frame of the data from the unformatted data stream; and

pre-processing, by the stream integrator, the data from the unformatted data stream to detect a leak in a pipeline by passing the frame of the data through hue-saturation-value (HSV) based clustering to segment environment into distinct color patches, wherein the leak in the pipeline is detected by using a transform to extract the pipeline spanning the frame.

15. The method for UV movement and data control according to claim 13, further comprising:

utilizing, by the TLC stream data analyzer, Teh-Chin chain approximation to extract a blob that represents the leak in the pipeline.

16. The method for UV movement and data control according to claim 13, further comprising:

utilizing, by the TLC stream data analyzer, a naï

ve-Bayes classifier to classify the blob as a leak or not a leak.

17. A non-transitory computer readable medium having stored thereon machine readable instructions for UV movement and data control, the machine readable instructions when executed cause at least one hardware processor to:

receive, at a telemetry data analyzer that is executed by at least one hardware processor, formatted movement and status metadata from at least one sensor of a UV during movement of the UV, wherein the movement and status metadata includes time and location information for the UV during the movement of the UV;

receive, at a stream data analyzer that is executed by the at least one hardware processor, an unformatted data stream from the at least one sensor of the UV, wherein the unformatted data stream includes a plurality of frames of data;

inject, by a data integrator that is executed by the at least one hardware processor, the time and location information into metadata of each of the plurality of frames of the data of the unformatted data stream to generate a time and location correlated (TLC) stream;

generate, by a time and location service analyzer that is executed by the at least one hardware processor, based on the movement and status metadata, a model of a state of the UV during the movement of the UV;

extract, by a stream integrator that is executed by the at least one hardware processor, the plurality of the frames of the data from the unformatted data stream;

retrieve, by the stream integrator, the time and location information from the model of the state of the UV;

inject, by the stream integrator, the time and location information into the metadata of each of the plurality of the frames of the data from the unformatted data stream to generate TLC frames, wherein a collection of the TLC frames represents the TLC stream;

analyze, by a TLC stream data analyzer that is executed by the at least one hardware processor, the TLC stream to identify an event related to the UV; and

generate, at an event orchestrator that is executed by the at least one hardware processor, a notification related to the event.