Systems and methods of capturing and distributing imaging content captured through unmanned aircraft systems

• US 10,298,664 B2
• Filed: 12/15/2016
• Issued: 05/21/2019
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Inventors: High, Donald R., Thompson, John P., Fenton, Timothy M., Rye, Erik
• Primary Examiner(s): Wasel, Mohamed A
• Application Number:
  US15/380,086
• Publication Number:
  US 20170180460A1
• Time in Patent Office: 887 Days
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/142 Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/40 for vehicles, e.g. vehicle-to-pedestrians [V2P]
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
• CPC Class Codes: H04W 84/042 Public Land Mobile systems, e.g. cellular systems
• CPC Class Codes: H04W 84/12 WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/12 Access point controller devices

Abstract

In some embodiments, systems and methods are provided to capture and distribute imaging content. Some embodiments, provide remote inspection systems, comprising: an unmanned aircraft system (UAS) base station control system that wirelessly communicates with an UAS, and comprises: a wireless transceiver; a control circuit; and a memory wherein the control circuit: receives imaging content, captured by a camera of the UAS; establishes a network connection with a content distribution system and activate a distribution session; and communicates the imaging content to the content distribution system that enables multiple remote authorized rendering systems to access the networked content distribution system over the Internet, join the distribution session, and receive over the Internet in real time the imaging content allowing each of the rendering systems to visually play back the imaging content such that a user at each of the multiple rendering systems can watch the imaging content in real time.

Claims

1. A remote inspection system, comprising:

an unmanned aircraft system (UAS) base station control system of a plurality of separate and distributed base station control systems, wherein the base station control system is configured to wirelessly communicate with an UAS to provide control signals to the UAS in controlling flight of the UAS, wherein the base station control system comprises;

a wireless transceiver;

a control circuit communicatively coupled with the transceiver; and

a memory coupled to the control circuit and storing computer instructions that when executed by the control circuit cause the control circuit to perform the steps of;

receive imaging content captured by a camera of the UAS and processed by the UAS while the UAS is in flight and performing an assigned task;

receive sensor data captured in real time by sensor systems of the UAS while performing the task;

determine, based on at least some of the imaging content, a basic shape of an area for which imaging content is to be captured by the UAS, determine based on the determined basic shape a route relative to the basic shape along which the UAS is to travel while capturing the imaging content, and direct the UAS to implement controls to travel along the determined route while continuing to capture the imaging content;

establish a network connection over a distributed communication network with a remote networked content distribution system and activate a distribution session through the content distribution system causing the remote networked content distribution system, which is separate and remote from the base station control system, to establish the distribution session, which is accessible over the Internet and through the networked content distribution system by numerous different and remote devices and to contact at least one user of interest associated with the distribution session, wherein the activated distribution session is specifically associated with the imaging content received from the base station control system and is distinct from one or more other distribution sessions accessible through the networked content distribution system, and to cause a three-dimensional (3D) model to be generated based on the imaging content and the sensor data; and

communicate over the distributed communication network the imaging content to the remote networked content distribution system that enables multiple different and remote authorized rendering systems to simultaneously access the networked content distribution system over the Internet, join the distribution session, and receive over the Internet in real time the imaging content allowing each of the rendering systems to visually play back the imaging content such that a user at each of the multiple rendering systems can watch the imaging content in real time.

2. The system of claim 1, wherein the plurality of base station control systems comprise:

hundreds of additional base station control systems each in wireless communication with a separate corresponding UAS, wherein each of the hundreds of base station control systems are configured to;

establish a separate network connection over the distributed communication network with the remote networked content distribution system and activate separate distribution sessions through the content distribution system that are exclusively associated with that corresponding additional base station control system and are separately accessible with different authorizations by authorized rendering systems; and

communicate over the distributed communication network additional imaging content captured by the corresponding UAS to the remote networked content distribution system that enables hundreds of sets of one or more remote authorized additional rendering systems to access one of the additional imaging content that the rendering system has been authorized to access over the Internet, join the corresponding distribution session, and receive over the Internet in real time the authorized one of the additional imaging content allowing each of the additional rendering systems to visually play back the authorized one of the imaging content and a user at each of the multiple rendering systems to watch the corresponding additional imaging content in real time.

3. The system of claim 2, wherein the imaging content comprises imaging content captured by the UAS while the UAS is in a process of delivering a previously purchased product to a customer, and a first rendering system associated with the customer is authorized to access the distribution session enabling the customer to view the imaging content of the delivery of the product in real time through the first rendering system.

4. The system of claim 1, wherein the imaging content, captured by the camera of the UAS, comprises imaging content of a structure being inspected by a remote expert as the UAS flies about the structure capturing the imaging content;

wherein the remote inspector views the imaging content through the distribution session enabling remote inspection of the structure in real time.

5. The system of claim 4, wherein the control circuit is further configured to receive a remote navigation instruction from a first rendering system associated with the remote inspector, and to communicate, to the UAS while the UAS is flying about the structure, a navigation command corresponding to the navigation instruction to cause the UAS to deviate from an intended flight path in accordance with at least the navigation command.

6. The system of claim 5, wherein the control circuit in communicating the navigation command comprises causing a communication of the navigation command that causes the UAS to deviate from a preprogrammed flight path and capture additional imaging content while deviating from the preprogrammed flight path.

7. The system of claim 4, wherein the control circuit is further configured to receive, from the UAS, additional sensor data captured by a sensor of the UAS;

and

communicate the additional sensor data over the distributed communication network to the content distribution system to be distributed as part of the distribution session, and causing a first remote rendering system associated with the inspector to provide the inspector with access to additional sensor data as part of the inspection of the at least the portion of the structure.

8. The system of claim 7, wherein the control circuit in receiving the additional sensor data receives hyperspectral sensor data detected by a hyperspectral sensor on the UAS, and communicates the hyperspectral sensor data to the networked content distribution system to be accessible through the distribution session.

9. The system of claim 1, wherein the control circuit is configured to cause the UAS to process original imaging content captured by the camera of the UAS to generate the imaging content at a lower resolution than the original imaging content, cause the UAS to wirelessly communicate the imaging content, cause the UAS to locally store the higher resolution original imaging content on the UAS for later retrieval and distribution, receive the original imaging content in the higher resolution from the UAS after the UAS has completed capturing the original imaging content and after the UAS previously communicated the lower resolution imaging content.

10. The system of claim 1, wherein the control circuit in is configured to communicate an identifier of the base station control system utilized by the networked content distribution system to identify the distribution session and associate the imaging content with the distribution session.

11. The system of claim 1, further comprising the UAS wherein the UAS autonomously implements a preprogrammed flight path that causes the UAS to travel along a length of the structure at a first elevation, change to a second elevation based on a field of view of the camera, and travel along the length of the structure while at the second elevation.

12. A method of capturing and distributing imaging content, comprising:

receiving, through a wireless transceiver of a base station control system of a plurality of separate and distributed base station control systems and from an unmanned aircraft system (UAS), imaging content captured by a camera of the UAS and processed by the UAS while the UAS is in flight and performing an assigned task;

receiving sensor data captured in real time by sensor systems of the UAS while performing the task;

determining, based on at least some of the imaging content, a basic shape of an area for which imaging content is to be captured by the UAS;

determining based on the determined basic shape a route relative to the basic shape along which the UAS is to travel while capturing the imaging content;

directing the UAS to implement controls to travel along the determined route while continuing to capture the imaging content;

establishing a network connection over a distributed communication network with a remote networked content distribution system and activating a distribution session through the content distribution system causing the remote networked content distribution system, which is separate and remote from the base station control system, to establish the distribution session, which is accessible over the Internet and through the networked content distribution system by numerous different and remote devices and to contact at least one user of interest associated with the distribution session, wherein the activated distribution session is specifically associated with the imaging content received from the base station control system and is distinct from one or more other distribution sessions accessible through the networked content distribution system;

causing a three-dimensional (3D) model to be generated based on the imaging content and the sensor data; and

communicating over the distributed communication network the imaging content to the remote networked content distribution system and enabling multiple different and remote authorized rendering systems to simultaneously access the networked content distribution system over the Internet, join the distribution session, and receive over the Internet in real time the imaging content allowing each of the rendering systems to visually play back the imaging content such that a user at each of the multiple rendering systems can watch the imaging content in real time.

13. The method of claim 12, further comprising:

establishing, through each of hundreds of additional base station control systems of the plurality of base station control systems each in wireless communication with a separate corresponding UAS, a separate network connection over the distributed communication network with the content distribution system and activating separate distribution sessions through the content distribution system that are exclusively associated with that corresponding additional base station control system and are separately accessible with different authorizations by authorized rendering systems; and

communicating, by each of the hundreds of additional base station control systems and over the distributed communication network, additional imaging content captured by each of the corresponding UASs to the remote networked content distribution system which enables hundreds of sets of one or more remote authorized additional rendering systems to access one of the additional imaging content that the rendering system has been authorized to access over the Internet, join the corresponding distribution session, and receive over the Internet in real time the authorized one of the additional imaging content allowing each of the additional rendering systems to visually play back the authorized one of the imaging content and a user at each of the multiple rendering systems to watch the corresponding additional imaging content in real time.

14. The method of claim 13, wherein communicating the imaging content comprises communicating the imaging content comprising imaging content captured by the UAS while the UAS is in a process of delivering a previously purchased product to a customer, wherein a first rendering system associated with the customer is authorized to access the distribution session enabling the customer to view the imaging content of the delivery of the product in real time through the first rendering system.

15. The method of claim 12, wherein the activating the distribution session comprises enabling access by a remote rendering system associated with a remote inspection expert to the imaging content comprising imaging content of a structure being inspected by the remote expert as the UAS flies about the structure capturing the imaging content;

wherein the remote inspector views the imaging content through the distribution session enabling remote inspection of the structure in real time.

16. The method of claim 15, further comprising:

receiving a remote navigation instruction from the remote rendering system associated with the remote inspector; and

communicating, to the UAS while the UAS is flying about the structure, a navigation command corresponding to the navigation instruction to cause the UAS to deviate from an intended flight path in accordance with at least the navigation command.

17. The method of claim 16, wherein the communicating the navigation command comprises communicating the navigation command that causes the UAS to deviate from a preprogrammed flight path and to capture additional imaging content while deviating from the preprogrammed flight path.

18. The method of claim 15, further comprising:

receiving, from the UAS, additional sensor data captured by a sensor of the UAS;

communicating the additional sensor data over the distributed communication network to the content distribution system to be distributed as part of the distribution session; and

causing a remote rendering system associated with the inspector to provide the inspector with access to the additional sensor data as part of the inspection of the at least the portion of the structure.

19. The method of claim 18, wherein the receiving the additional sensor data comprises receiving hyperspectral sensor data detected by a hyperspectral sensor on the UAS;

and

the communicating the additional sensor data comprises communicating the hyperspectral sensor data to the networked content distribution system to be accessible through the distribution session.

20. The method of claim 15, further comprising:

communicating an identifier of the base station control system utilized by the networked content distribution system to identify the distribution session and associate the imaging content with the distribution session.

21. The method of claim 12, further comprising:

causing the UAS to process original imaging content captured by the camera of the UAS to generate the imaging content at a lower resolution than the original imaging content;

causing the UAS to wirelessly communicate the imaging content;

causing the UAS to locally store the higher resolution original imaging content on the UAS for later retrieval and distribution; and

receiving the original imaging content in the higher resolution from the UAS after the UAS has completed capturing the original imaging content and after the UAS previously communicated the lower resolution imaging content.

22. The method of claim 12, further comprising:

autonomously implementing, through the UAS, a preprogrammed flight path that causes the UAS to travel along a length of the structure at a first elevation, change to a second elevation based on a field of view of the camera, and travel along the length of the structure while at the second elevation.