Bulk searchable geo-tagging of detected objects in video

• US 10,025,992 B1
• Filed: 09/05/2017
• Issued: 07/17/2018
• Est. Priority Date: 07/07/2015
• Status: Active Grant

• Current Assignee: Ambarella International LP (Ambarella Incorporated)
• Original Assignee: Ambarella Incorporated
• Inventors: Fink, Alexander, Pertsel, Shimon
• Primary Examiner(s): Adams, Eileen M
• Application Number:
  US15/695,237
• Time in Patent Office: 315 Days
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An apparatus comprising a sensor, an interface and a processor. The sensor may be configured to generate a video signal based on a targeted view of an environment. The interface may be configured to receive status information of the apparatus at a time of generation of the video signal. The processor may be configured to (i) detect one or more objects in the video signal, (ii) determine a location of the one or more objects relative to the sensor and (iii) generate metadata. The location of the one or more objects may be based on (i) the status information of the apparatus and (ii) a field of view of the sensor. The metadata may correspond to the location of the one or more objects.

Claims

1. An apparatus comprising:

a first input configured to receive a video signal from a sensor;

a second input configured to receive status information about said apparatus at a time of generation of said video signal; and

a processor configured to (i) detect one or more objects in said video signal, (ii) determine absolute coordinates of said one or more objects in said video signal and (iii) generate metadata, wherein (A) said absolute coordinates of said one or more objects are based on (i) said status information, (ii) a relative location of said detected objects in said video signal, (iii) an orientation of said apparatus, and (iv) a field of view of said sensor, (B) said metadata corresponds to said absolute coordinates of said one or more objects, and (C) said orientation of said apparatus is added to the relative location of the detected objects to determine said absolute coordinates.

2. The apparatus according to claim 1, wherein said metadata comprises searchable data about said detected objects corresponding to a video file generated by said processor from said video signal.

3. The apparatus according to claim 1, wherein (i) a distance of said one or more detected objects to said apparatus is determined using a depth-sensing technology of said apparatus and (ii) said distance is used to determine said relative location.

4. The apparatus according to claim 1, wherein said metadata comprises at least one of (a) information related to said one or more detected objects, (b) information related to said sensor and (c) information related to said apparatus.

5. The apparatus according to claim 1, wherein said sensor is installed separate from said apparatus.

6. The apparatus according to claim 1, wherein said metadata is uploaded to a searchable database via at least one of (a) wireless communications and (b) hard-wired communications.

7. The apparatus according to claim 6, wherein an alert is issued by said searchable database when said metadata stored in said searchable database matches against data provided by an external source.

8. The apparatus according to claim 7, wherein said data provided by said external source comprises at least one of wanted fugitives and stolen vehicles.

9. The apparatus according to claim 1, wherein said metadata is appended as a text track to a video file generated by said processor from said video signal.

10. The apparatus according to claim 1, wherein said status information comprises absolute coordinates and azimuth orientation.

11. The apparatus according to claim 1, wherein said one or more detected objects comprise license plates.

12. The apparatus according to claim 11, wherein said metadata generated for said license plates is based on optical character recognition.

13. The apparatus according to claim 1, wherein said one or more detected objects comprise human faces.

14. The apparatus according to claim 13, wherein said processor is further configured to perform facial detection to detect said human faces in a region of a video frame generated by said processor from said video signal.

15. The apparatus according to claim 13, wherein (i) said human faces are compared with face information stored in a database using facial recognition and (ii) said human faces are associated with metadata based on said face information.

16. The apparatus according to claim 1, wherein said field of view of said sensor is used by said processor to calculate (i) a distance and (ii) an angle each used to determine said relative location and said orientation of said detected objects relative to said apparatus.

17. The apparatus according to claim 1, wherein said relative location of said detected objects in said video signal is determined based on (i) a comparison of (a) a respective size of said detected objects and (b) a size of a reference object in a first mode and (ii) a comparison of (a) a number of pixels said detected object occupies in a video frame generated by said processor from said video signal and (b) said field of view in a second mode.

18. An apparatus comprising:

a sensor configured to generate a video signal based on a targeted view of an environment;

a plurality of sensors configured to determine status information of said apparatus at a time of generation of said video signal; and

a processor configured to (i) detect one or more objects in said video signal, (ii) determine absolute coordinates of said one or more objects in said video signal and (iii) generate metadata, wherein (A) said absolute coordinates of said one or more objects are based on (i) said status information, (ii) a relative location of said detected objects in said video signal and (iii) an orientation of said apparatus, and (iv) a field of view of said sensor, (B) said metadata corresponds to said absolute coordinates of said one or more objects and (C) said orientation of said apparatus is added to the relative location of the detected objects to determine said absolute coordinates.

19. The apparatus according to claim 18, wherein (i) a first of said plurality of sensors comprises a Global Positioning System (GPS) sensor configured to determine absolute coordinates of said apparatus and (ii) a second of said plurality of sensors comprises a magnetometer configured to determine an azimuth orientation of said apparatus.

20. An apparatus comprising:

a sensor configured to generate a video signal based on a targeted view of an environment;

an interface configured to receive status information of said apparatus at a time of generation of said video signal; and

a connection to an external processor, said external processor configured to (i) detect one or more objects in said video signal, (ii) determine absolute coordinates of said one or more objects in said video signal and (iii) generate metadata, wherein (A) said absolute coordinates of said one or more objects are based on (i) said status information, (ii) a relative location of said detected objects in said video signal, (iii) an orientation of said apparatus and (iv) a field of view of said sensor, (B) said metadata corresponds to said absolute coordinates of said one or more objects and (C) said orientation of said apparatus is added to the relative location of the detected objects to determine said absolute coordinates.