Automatic imaging device selection for video analytics

• US 9,871,998 B1
• Filed: 01/30/2017
• Issued: 01/16/2018
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Curlander, James Christopher, Graybill, Jules Cook, Molello, Michael Anthony
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Brown, Jr., Howard D
• Application Number:
  US15/419,968
• Time in Patent Office: 351 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 25/08 using communication transmission lines G08B13/19658, G08B21/0286, G08B25/016 take precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A suitable imaging device for capturing images of an object or an area of interest may be automatically selected based on information regarding the locations and/or other operational properties of a plurality of imaging devices, and a location of the object or the area of interest. One or more geometric models of the fields of view of the imaging devices may be generated, and whether the object or the area of interest falls within one or more of the fields of view may be determined using such models. Where multiple imaging devices may include the object or the area of interest, within a field of view, the imaging device having the most suitable image, e.g., the largest or highest resolution image of the object or the area of interest, may be selected.

Claims

1. A method comprising:

capturing a plurality of video files from a scene by a plurality of digital cameras over a period of time, wherein each of the digital cameras defines a field of view including at least a portion of the scene, and wherein the period of time includes a first time;

storing at least some of the video files in at least one data store at a second time, wherein the second time follows the period of time;

after the second time,

determining that an event occurred at a location within the scene at approximately the first time;

determining, for each of the digital cameras, a coverage zone based at least in part on each of;

a position of the digital camera;

an orientation of the digital camera; and

a focal length of the digital camera;

identifying a first one of the coverage zones including the location within the scene;

identifying a first one of the digital cameras corresponding to the first one of the coverage zones; and

retrieving a first one of the video files captured by the first one of the digital cameras from the at least one data store.

2. The method of claim 1, further comprising:

generating a model of the field of view of the first one of the digital cameras at the first time based at least in part on the first one of the video files;

generating at least one back-projection of the model into at least a second one of the video files captured by at least a second one of the digital cameras according to an optical transfer function; and

reconstructing the event based at least in part on the model and the at least one back-projection.

3. The method of claim 1, wherein the location within the scene includes at least a portion of a secure facility, and

wherein determining that the event has occurred comprises:

determining that access to the secure facility was obtained via at least one entryway at approximately the first time, and

wherein identifying the one of the coverage zones including the location within the scene comprises;

determining that the one of the coverage zones includes at least a portion of the at least one entryway.

4. The method of claim 1, wherein determining the coverage zone for each of the digital cameras further comprises:

processing at least one of the video files to recognize at least one of an outline, a movement or a figure associated with an object or an area of interest within at least one frame;

determining a number of pixels or locations of the pixels covering the outline, the movement or the figure associated with the object or the area of interest within at least one frame of each of the video files; and

selecting the first one of the video files based at least in part on the number of pixels or the location of the pixels within the at least one frame of each of the video files.

5. The method of claim 1, wherein determining the number of pixels or the locations of the pixels comprises:

determining coordinates of the object or the area of interest in three-dimensional space based at least in part on the at least one of the outline, the movement or the figure within the at least one frame; and

converting the coordinates of the object or the area of interest to coordinates within the at least one frame of each of the video files according to an optical transfer function.

20. The monitoring system of claim 1, wherein the operational characteristics of at least one of the plurality of imaging devices comprise:

a position of the at least one imaging device;

a roll angle of the at least one imaging device;

a pitch angle of the at least one imaging device;

a yaw angle of the at least one imaging device;

or a focal length of the at least one imaging device, and wherein the at least one computer processor is configured to at least;

determine, for each of the plurality of imaging devices, a geometric model of the field of view of the imaging device according to a coordinate system, wherein the geometric model is based at least in part on the operational characteristics of the imaging device;

identify a set of coordinates associated with the position of the event of interest according to the coordinate system; and

determine, for each of the plurality of imaging devices, whether the set of coordinates associated with the position of the event of interest is within the geometric model of the field of view of the imaging device.

6. A method comprising:

after a first time, determining that an event occurred at a location within a field of view of at least one of a plurality of cameras, wherein the event is determined to have occurred at the first time;

identifying a plurality of video files captured by the plurality of cameras, wherein each of the video files includes at least one image frame captured at the first time, and wherein each of the video files is stored in at least one data store;

determining information regarding positions of each of the plurality of cameras at the first time;

determining information regarding focal lengths of each of the plurality of cameras at the first time;

determining information regarding angular orientations of each of the plurality of cameras at the first time; and

selecting one of the video files based at least in part on the information regarding the positions, the information regarding the focal lengths and the information regarding the angular orientations of one of the plurality of cameras at the first time.

7. The method of claim 6, wherein determining that the event occurred comprises:

determining that an alarm was issued at the location at the first time, wherein the alarm includes at least one of;

an audible signal;

a video signal;

or

a transmission of an electronic message.

8. The method of claim 6, wherein determining that the event occurred comprises at least one of:

determining that a portal was opened or closed at the location;

determine that a security barrier surrounding the location was breached;

or

determining that an object moved at the location.

9. The method of claim 6, wherein determining that the object moved at the location comprises:

determining that an object was present within a first video frame captured by one of the plurality of cameras prior to the first time; and

determining that the object was not present within a second video frame captured by one of the plurality of cameras following the first time.

10. The method of claim 6, wherein selecting the one of the video files comprises:

defining, for each of the plurality of cameras, a coverage zone for the field of view of the camera at the first time, wherein each of the coverage zones is defined with respect to free space at the location, and wherein each of the coverage zones comprises a substantially pyramidal section defined based at least in part on the information regarding the positions, the information regarding the focal lengths and the information regarding the angular orientations of each of the plurality of cameras;

identifying one of the coverage zones including the location at the first time; and

determining the one of the plurality of cameras corresponding to the one of the coverage zones,

wherein the selected one of the video files was captured by the one of the plurality of cameras corresponding to the one of the coverage zones.

11. The method of claim 10, wherein each of the coverage zones comprises a subset of pixels of the field of view of the one of the plurality of cameras.

12. The method of claim 10, wherein selecting the one of the video files further comprises:

determining locations of pixels covering each of the coverage zones; and

initiating video analytics of the video files based at least in part on the locations of the pixels by the at least one computer processor.

13. The method of claim 10, wherein at least one of the plurality of cameras has at least one of an adjustable focal length, an adjustable location, an adjustable roll angle of rotation, an adjustable pitch angle of rotation or an adjustable yaw angle of rotation, and

wherein each of the coverage zones is defined as a function of time.

14. The method of claim 6, wherein selecting one of the video files comprises:

determining a position of an object associated with the event in three-dimensional space at the first time;

mathematically transforming the position of the object associated with the event in three-dimensional space into each of the video files; and

identifying, in each of the video files, a subset of pixels corresponding to the mathematically transformed position of the object at the first time,

wherein the one of the video files is selected based at least in part on the subsets of pixels.

15. The method of claim 14, wherein identifying the subset of pixels corresponding to the mathematically transformed position of the object at the first time comprises at least one of:

determining a number of pixels covering at least a portion of the object within each of the video files;

or

determining locations of the pixels covering at least the portion of the object within each of the video files,

wherein the one of the video files is selected based at least in part on the number of the pixels or the locations of the pixels.

16. The method of claim 15, wherein the number of the pixels or the locations of the pixels are determined according to an optical transfer function.

17. The method of claim 6, wherein selecting the one of the video files comprises:

identifying, in a first one of the plurality of video files, pixels corresponding to an object associated with the event at the location;

defining, in the first one of the plurality of video files, an area of interest including at least the pixels corresponding to the object; and

back-projecting the area of interest into each of the plurality of video files other than the first one of the plurality of video files,

wherein the one of the video files is selected based at least in part on the area of interest or the back-projected areas of interest.

18. The method of claim 6, wherein the information regarding the angular orientations of the plurality of imaging devices comprises an angle of view of a lens of the imaging device and an axis of orientation of the imaging device defined by at least one of a roll angle, a pitch angle or a yaw angle.

19. A monitoring system comprising:

a plurality of imaging devices, wherein each of the imaging devices is configured to capture media files at a location; and

a computing device in communication with each of the plurality of imaging devices,

wherein the computing device comprises at least one data store and at least one computer processor, and

wherein the at least one computer processor is configured to at least;

cause each of the plurality of imaging devices to capture media files at the location over a period of time;

cause each of the plurality of imaging devices to store the media files in the at least one data store;

determine that an event of interest has occurred at a first time, wherein the first time is within the period of time;

identify a position of the event of interest;

identify operational characteristics of each of the plurality of imaging devices at the first time;

identify a field of view for each of the plurality of imaging devices at the first time;

determine whether the field of view of at least one of the plurality of imaging devices included the position of the event of interest at the first time based at least in part on the operational characteristics of each of the plurality of imaging devices at the first time; and

in response to determining that a field of view of a first one of the plurality of imaging devices included the position of the event of interest at the first time,

identifying a media file captured by the first one of the plurality of imaging devices, wherein the media file includes at least one frame captured at the first time; and

retrieving the media file from the at least one data store.