Swarm approach to consolidating and enhancing smartphone target imagery by virtually linking smartphone camera collectors across space and time using machine-to machine networks

• US 10,419,657 B2
• Filed: 10/26/2015
• Issued: 09/17/2019
• Est. Priority Date: 10/26/2014
• Status: Active Grant

• Current Assignee: Galileo Group, Inc.
• Original Assignee: Galileo Group, Inc.
• Inventors: Barnes, Donald Michael, Grichnik, James Michael
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US15/521,578
• Publication Number:
  US 20180278827A1
• Time in Patent Office: 1,422 Days
• Field of Search: 345418
• CPC Class Codes: G01C 19/5776 Signal processing not specific to any of the devices covered by groups G01C19/5607 - G01C19/5719
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/2137 based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]

Abstract

At a first computer-enabled imaging device in a plurality of computer-enabled imaging devices, a first workflow including time-stamped images of a region of interest is obtained during a first time interval, and first meta data is associated with the second workflow. The first meta data includes positional and orientation data, the first time interval, and an identity of the first computer-enabled imaging device. Control signals are generated based on a characteristic of the first workflow, and then communicated to a second computer-enabled imaging device. In accordance with the control signals, the second computer-enabled imaging device obtains a second workflow including time-stamped images of the region of interest, and associates second meta data with the second workflow. A central system, or any computer-enabled imaging device in the plurality, consolidates the first and second workflows into a consolidated workflow for the region of interest using the first and second meta data.

Claims

1. A computer-implemented method for consolidating imagery using a plurality of computer-enabled imaging devices:

at a first computer-enabled imaging device in the plurality of computer-enabled imaging devices, the first computer-enabled imaging device having one or more processors, a first two-dimensional pixilated detector, and memory for storing one or more programs for execution by the one or more processors;

obtaining a first workflow in accordance with one or more capture parameters, the first workflow comprising a first plurality of time-stamped two-dimensional pixilated images of a region of interest of a subject during a first time interval; and

associating first meta data with the first workflow, wherein the first meta data comprises (i) a plurality of first positions and a plurality of first orientations of the first computer-enabled imaging device indicating first positional and orientation data for the first computer-enabled imaging device during the first time interval, (ii) an indication of the first time interval, and (iii) an identity of the first computer-enabled imaging device;

generating one or more control signals, wherein the one or more control signals are based at least in part on a characteristic of the first workflow or a characteristic of the region of interest captured by the first workflow,

communicating the one or more control signals to a second computer-enabled imaging device in the plurality of computer-enabled imaging devices across a network; and

at the second computer-enabled imaging device, the second computer-enabled imaging device having one or more processors, a second two-dimensional pixilated detector, and memory for storing one or more programs for execution by the one or more processors, the one or more programs executed in accordance with the one or more control signals received across the network;

in accordance with the one or more control signals;

obtaining a second workflow in accordance with one or more capture parameters, the second workflow comprising a second plurality of time-stamped two-dimensional pixilated image of the region of interest at a second time interval; and

associating second meta data with the second workflow, wherein the second meta data comprises (i) a plurality of second positions and a plurality of second orientations of the second computer indicating second positional and orientation data for the second computer-enabled imaging device during the second time interval, (ii) an indication of the second time interval, and (iii) an identity of the second computer-enabled imaging device; and

formatting the second workflow for combination with the first workflow; and

at a central system or any computer-enabled imaging device in the plurality of computer-enabled imaging devices;

generating a consolidated workflow for the region of interest by consolidating the first and second workflows using the first and second meta data, wherein the consolidated workflow comprises a visualization of the data of the first and second workflows.

2. The computer-implemented method of claim 1, wherein:

generating the one or more control signals at the first computer-enabled imaging device is performed while obtaining the first workflow during the first time interval; and

a portion of the first workflow and the second workflow are obtained concurrently in accordance with the one or more control signals.

3. The computer-implemented method of claim 1, wherein the one or more control signals are further dictated by a third workflow acquired by a third computer-enabled imaging device in the plurality of computer-enabled imaging devices, wherein the third workflow is acquired during or before the first time interval.

4. The computer-implemented method of claim 1, wherein the one or more control signals are communicated to two or more computer-enabled imaging devices in the plurality of computer-enabled imaging devices in order to initiate a plurality of workflows for the region of interest.

5. The computer-implemented method of claim 1, wherein the consolidating takes place at the central system at a time after the first and second workflows are completed.

6. The computer-implemented method of claim 1, wherein the first computer-enabled imaging device is airborne during the first time interval and the second computer-enabled imaging device is airborne during the second time interval.

7. The computer-implemented of claim 1, further comprising:

at the first computer-enabled imaging device, before communicating the one or more control signals;

synchronizing with the second computer-enabled imaging device, wherein the synchronizing comprises;

discovering one or more active computer-enabled imaging devices of the plurality of computer-enabled imaging devices, wherein the one or more active computer-enabled imaging devices includes the second computer-enabled imaging device; and

establishing a wireless network connection with at least the second computer-enabled imaging device,

wherein the one or more control signals are communicated to at least the second computer-enabled imaging device via the wireless network connection.

8. The computer-implemented method of claim 7, wherein the synchronizing further comprises determining respective identities of the one or more active computer-enabled imaging devices discovered.

9. The computer-implemented method of claim 7, wherein the synchronizing further comprises synchronizing respective clocks of the first computer-enabled imaging device and at least the second computer-enabled imaging device.

10. The computer-implemented of claim 1, wherein the one or more control signals include instructions for causing the obtaining of the second workflow to commence.

11. The computer-implemented of claim 1, wherein the one or more control signals include instructions for causing the obtaining of the second workflow and the associating of the second meta data at the second computer-enabled imaging device to continue after the first computer-enabled imaging device ceases the obtaining of the first workflow and the associating of the first meta data.

12. The computer-implemented of claim 1, wherein the one or more control signals include respective modified capture parameters for one or more computer-enabled imaging devices of the plurality of computer-enabled imaging devices, including the second computer-enabled imaging device,

wherein obtaining respective workflows and associating respective meta data by the one or more computer-enabled imaging devices are performed in accordance with the respective modified capture parameters.

13. The computer-implemented method of claim 12, wherein the respective modified capture parameters include:

respective frequencies at which the obtaining of respective workflows is performed,

respective frequencies at which the associating of respective meta data is performed; and

/or

respective resolutions at which the respective workflows are obtained.

14. The computer-implemented method of claim 12, wherein the respective modified capture parameters include:

respective start times at which the obtaining of respective workflows and/or the associating of respective meta data commence;

respective end times at which the obtaining of respective workflows and/or the associating of respective meta data terminate; and

/or

respective capture durations for which the obtaining of respective workflows and/or the associating of respective meta data are performed.

15. The computer-implemented of claim 12, wherein the respective modified capture parameters indicate one or more characteristics of the first and/or second workflows, and/or one or more characteristics of the region of interest, with respect to which the obtaining of respective workflows and the associating of respective meta data are to be performed.

16. The computer-implemented method of claim 1, wherein the one or more control signals include instructions for causing the obtaining of respective workflows and the associating of respective meta data by the first computer-enabled imaging device and the second computer-enabled imaging device to be performed synchronously.

17. The computer-implemented of claim 1, wherein the one or more control signals include instructions for causing the obtaining of respective workflows and the associating of respective meta data by the first computer-enabled imaging device and/or the second computer-enabled imaging device to be performed in accordance with one or more additional regions of interest of the subject corresponding to the characteristic upon which the one or more control signals were generated.

18. The computer-implemented method of claim 17, wherein the one or more additional regions of interest include regions of interest for which insufficient image data and/or meta data was captured by the first computer-enabled imaging device.

19. The computer-implemented method of claim 17, wherein the instructions for causing the obtaining of respective workflows and the associating of respective meta data include instructions for obtaining the respective workflows of the one or more additional regions of interest at an increased image resolution.

20. The computer-implemented method of claim 1, wherein the first and second positional and orientation data are acquired by using respective compass, gyroscope, barometer, global positioning, and/or accelerometer readings of the first and second computer-enabled imaging devices.