METHODS AND SYSTEMS FOR DETECTING AND ANALYZING A REGION OF INTEREST FROM MULTIPLE POINTS OF VIEW

• US 20190042868A1
• Filed: 08/03/2017
• Published: 02/07/2019
• Est. Priority Date: 08/03/2017
• Status: Active Grant

• Current Assignee: Streaming Global, Inc.
• Original Assignee: Shouty, LLC
• Inventors: Oesterreicher, Richard, Hessing, Jonathan, Ukani, Zunair, Schmidt, Austin
• Granted Patent: US 10,586,124 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49

Abstract

Disclosed are devices, systems, methods, techniques, and computer program products for estimating a Region Of Interest (ROI) corresponding to a plurality of content streams. The method may include receiving, by a system having at least one processor, a plurality of sensor data associated with a plurality of mobile devices. The plurality of sensor data corresponds to a plurality of content streams captured by the plurality of mobile devices. Further, each of the plurality of mobile devices may include at least one recorder configured to capture a corresponding content steam. Further, a sensor data associated with a mobile device may include a location data and an orientation data of the mobile device during capturing of a content stream. The method may further include, analyzing, by the system, the plurality of sensor data and determining, by the system, a ROI based on at least the plurality of sensor data, thereby generating an estimation of the ROI.

Claims

1. A method of estimating a Region Of Interest (ROI) corresponding to a plurality of content streams, the method comprising:

receiving, by a system having at least one processor, a plurality of sensor data associated with a plurality of mobile devices, wherein the plurality of sensor data corresponds to a plurality of content streams captured by the plurality of mobile devices, wherein each of the plurality of mobile devices comprises at least one recorder configured to capture a corresponding content steam, wherein a sensor data associated with a mobile device comprises a location data and an orientation data of the mobile device during capturing of a content stream;

analyzing, by the system, the plurality of sensor data; and

determining, by the system, a ROI based on at least the plurality of sensor data, thereby generating an estimation of the ROI.

2. The method of claim 1, further comprising receiving, by the system, a plurality of optical characteristics corresponding to the plurality of mobile devices;

and wherein determining the ROI is based at least on the plurality of optical characteristics.

3. The method of claim 2, wherein an optical characteristic of a mobile device comprises a Field of View (FOV) of a lens comprised in the mobile device.

4. The method of claim 2, further comprising receiving a plurality of device identifiers from the plurality of mobile devices, wherein the plurality of optical characteristics is identified based on at least the plurality of device identifiers.

5. The method of claim 1, wherein analyzing the plurality of sensor data comprises:

determining a plurality of origin points based at least on a plurality of location data corresponding to the plurality of mobile devices; and

determining a plurality of vectors associated with the plurality of origin points based at least on a plurality of orientation data corresponding to the plurality of mobile devices, a vector associated with an origin point originating at the origin point.

6. The method of claim 5, wherein determining the ROI is based at least on the plurality of origin points and the plurality of vectors.

7. The method of claim 6, wherein the plurality of origin points is configured to be used as coordinates in a mapping coordinate system.

8. The method of claim 7, further comprising determining a plurality of vector intersections of the plurality of vectors;

and wherein determining the ROI is based at least on the plurality of vector intersections.

9. The method of claim 1, further comprising:

receiving a plurality of Field of Views (FOVs) associated with the plurality of mobile devices, a Field of View (FOV) associated with a mobile device being associated with a lens comprised in the mobile device;

determining a plurality of origin points based at least on a plurality of location data corresponding to the plurality of mobile devices, the plurality of origin points configured as coordinates in a mapping coordinate system;

determining a plurality of vectors associated with the plurality of origin points based at least on a plurality of orientation data corresponding to the plurality of mobile devices, a vector associated with an origin point originating at the origin point;

generating a plurality of viewing frustums based at least on the plurality of FOVs, wherein an apex of a viewing frustum associated with a mobile device is coincident with the origin point associated with the mobile device, wherein a view direction of the viewing frustum is aligned with the vector associated with the mobile device; and

determining a plurality of intersections of the plurality of viewing frustums, wherein determining the ROI is based at least on the plurality of intersections of the plurality of viewing frustums.

10. The method of claim 9, wherein determining a plurality of origin points comprises converting a plurality of location data corresponding to the plurality of mobile devices into a plurality of origin points in a mapping coordinate system;

and wherein determining a plurality of vectors comprises converting a plurality of orientation data corresponding to the plurality of mobile devices into a plurality of vectors associated with the plurality of origin points, wherein a vector associated with an origin point originates at the origin point.

11. The method of claim 1, further comprising:

receiving the plurality of content streams from the mobile devices;

analyzing the plurality of content streams;

determining at least one object associated with the ROI; and

determining at least one spatial dimension associated with the at least one object;

wherein determining the ROI is based at least on the at least one spatial dimension.

12. The method of claim 1 further comprising:

receiving the plurality of content streams from the mobile devices;

analyzing the plurality of content streams;

determining at least one object associated with the ROI based at least on the plurality of content streams;

determining at least one spatial dimension associated with the at least one object based at least on a predetermined association between a plurality of objects and a plurality of spatial dimensions; and

determining a plurality of distances corresponding to the plurality of mobile devices and the ROI, wherein a distance between a mobile device and the ROI is estimated based on a spatial dimension of the at least one object determined in a content stream captured by the mobile device;

wherein determining the ROI is based at least on the plurality of distances.

13. The method of claim 12, wherein determining the plurality of distances comprises determining a plurality of distances corresponding to the plurality of mobile devices, wherein a distance between a mobile device and the ROI is determined based at least on a spatial dimension of the at least one object determined in a content stream captured by the mobile device.

14. The method of claim 1, wherein the sensor data further comprises a distance data representing a distance of the mobile device from at least one object comprised in the ROI, wherein the mobile device comprises a ranging sensor configured to determine the distance.

15. The method of claim 14, wherein the ranging sensor comprises at least one of a LIDAR sensor and an image sensor.

16. The method of claim 1, further comprising:

executing a search on a venue database based on the location data;

determining the venue from a venue database based at least on the location data; and

retrieving at least one spatial characteristic of the venue;

wherein determining the ROI is based at least on the at least one spatial characteristic.

17. The method of claim 1, wherein the plurality of mobile devices comprises a drone configured for flight.

18. The method of claim 16, further comprising determining at least one trajectory associated with at least one mobile device based at least on analysis of the plurality of sensor data;

wherein determining the ROI is based at least on the at least one trajectory.

19. The method of claim 1, further comprising:

generating a visualization of the ROI and the plurality of mobile devices; and

transmitting the visualization to at least one mobile device of the plurality of mobile devices.

20. The method of claim 19, further comprising:

receiving a selection associated with the visualization;

identifying one or more mobile devices associated with the selection; and

transmitting one or more content streams captured by the one or more mobile devices.